What was I saving all these examples for?

Archive collection and preservation

Working

Per-item text and image signals

Working

Clustering, taxonomy and typed graph

Working

Retrieval and category prediction against hand labels

Benchmarked on held-out samples (self-reported)

Generalisation of induced concepts

Hypothesis · not shown

Claim/evidence modelling

Planned

04Situation

I had years of saved images and videos, collected because something in each one seemed worth keeping. Sometimes two examples belonged together even though they looked nothing alike. I built InstaGraph to preserve that material and explore whether I could make those connections easier to see.

The working hypothesis is that the pattern of what I save contains ideas I would struggle to name directly. The system collects text and visual information, groups items, and builds a graph of possible connections. I compare its retrieval and category predictions with examples I have labeled by hand.

05Research

The work proceeds in stages, each of which had to exist before the next could be tested: ^[I06]

1. Collect and preserve. Pull the saved items into a local archive with their provenance (source, date, collection), so that nothing downstream depends on a platform that may change or disappear.
2. Extract signals. For each item, derive text (captions, transcripts, on-screen text) and visual features, and store them alongside the original.
3. Organise. Cluster items, build a taxonomy, and construct a typed graph in which items connect to candidate concepts and concepts to each other.
4. Evaluate. Hold out a set of items I have labelled by hand, and test whether the system retrieves the right neighbours and predicts the right categories.
5. Model claims and evidence — the planned stage: moving from "these items are about the same thing" to "this item is evidence for this claim".

The research question lives in stages 3 and 4. Stages 1 and 2 are infrastructure, and most of the code is there.

06Design decisions

Preserve first, analyse second. The archive layer was built before any modelling, and it is the part I would keep if everything else were thrown away. A personal archive that depends on an external platform's goodwill is not an archive. This also set a constraint for the whole project: every derived signal points back to a preserved original.

Make the connections readable. Similarity search can find things that look or sound alike. I also wanted to examine why two items might belong together. The graph gives item-to-concept and concept-to-concept links explicit types, so I can inspect and question a relationship.

Give the system something it can get wrong. I labeled a held-out set myself rather than treating the system's own clusters as the answer. The benchmark asks whether it agrees with my labels. Those labels are still one person's interpretation of one archive.

07Artifact

A working pipeline on my own archive: collection and preservation, per-item text and visual signals, clustering, a taxonomy, a typed graph, and a retrieval/category benchmark against hand-labelled held-out items, with a test suite exercising the stages. ^[I06] The repository's own count of more than 600 tests is a self-report of the suite's size, not an evaluation of the method.

The hero figure above is a synthetic graph: generic items attached to invented concept names, laid out by a force simulation, showing the kind of structure the pipeline produces — including items that belong to two concepts, which are the interesting cases. A real export, cleared of third-party media, replaces it when ready.

08Outcomes

The pipeline runs end to end on the archive, and held-out benchmarks for retrieval and category prediction exist. ^[I06] Their numbers are not reported here until a dated run with its exact command can be attached, and even then they measure agreement with one person's labels on one archive.

I still need to test whether the proposed concepts explain anything useful about my thinking, whether the method transfers to another person's archive, and whether the planned claim-and-evidence layer works.

09Reflection

The archive has already been worth making: years of saved material are now preserved outside the platform they came from. The research part is harder. A graph can look convincing long before I know whether its connections are useful.

That is the connection to SecondThought for me. In both projects, I want a system to show enough of its interpretation that a person can disagree with it.