Work/How it learnsTerms explained in the pattern
SPINES: every shelf makes the next one easier to read.
SPINES turns a photo of a bookshelf into a list of identified books. Every shelf a user approves adds its books, and which books sit together, to a shared catalogue. The next photo is matched against that catalogue first, and every discovery feature is computed from it. Nothing here is a trained model yet. It is counting, done at scale, and it gets better because the counts grow. Every approved shelf also labels spine images for the classifier that comes next.
Under the hood
The loop, stage by stage
Each stage is a mechanism you could build yourself: what is stored, what is counted, and what changes.
-
A shelf comes in
One photo becomes one upload row. The uploader says whether it is their own collection or just a shelf they saw. Every detected spine is stored with what the model read, its confidence, and the book it resolved to.
How it is built- The upload row carries the image hash for dedup, the spine map, privacy level, tags, and where it came from: the web app, an API key, or an agent over MCP.
- One row per spine: detected title, author, ISBN, confidence, position, the resolved book, and feedback columns.
- Every model run is logged.
-
Corpus first, catalogues second
A new photo is matched against books already found on other shelves before any outside catalogue is asked. Editions collapse to one canonical work, so a paperback and a hardback count as the same book. Each shelf added raises the odds of an instant match for the next.
How it is built- A database-only pass runs before any catalogue call, and its hit count is logged.
- Editions resolve to a canonical work id from an open catalogue, or a minted one, and link together.
- Quick-match hit rate: 40% at baseline, 80% as the target. One re-upload matched 59 of 88 spines instantly.
- A nightly sweep spends leftover catalogue quota before it expires, re-searching spines it could not identify. Collections that keep coming up empty are visited less often.
-
Pairs are counted across shelves
Every pair of books that sits together on a shelf gets a count of the distinct approved shelves holding both. That one number drives pair pages, book pages and leaderboards.
How it is built- Book pairs across the whole corpus, grouped by canonical work.
- Rebuilt nightly into ranking slices, one per genre group.
- Pair tiers by shared shelves: Distant Cousins, No Strangers, Shelf Soulmates, a Natural Match.
-
The uploader gets something back
The Quick Insights overlay shows picks popular with like-minded readers, similar collections, and a one-line thesis about the shelf. Public profile and persona pages follow. And when a newly approved shelf shares five or more books with yours, you hear about it.
How it is built- Picks are the strongest co-occurring books, minus what is already on the viewer's reading list, with the “via” book shown.
- Similar collections need a minimum of shared books. The one-line thesis is rule-based: a genre share above a threshold fires it.
- Owners whose shelves share five or more books with the new one are notified. View milestones mark a shelf's reach.
-
Only approved shelves count
Every shelf passes moderation before it can add to the corpus, so the pair counts are built from real, checked shelves. Approval is also the moment the loop notifies neighbours, and the point where a shelf owner's own-collection choice starts to matter.
How it is built- Moderation status gates every query that counts pairs or matches quickly; moderators can approve in batches.
- “My own reads” is kept apart from “a shelf I saw”, so personas and recommendations lean on the former.
- Neighbour notifications fire on approval, not on upload.
-
Every approved shelf is a labelled training set
Today a detector finds each spine as a box and a language model reads the text on it; the two are merged by position. Every approved shelf therefore stores spine images with a verified title attached: the labelled data for a vision classifier that will know a book by its spine alone, without reading it. The classifier is the next step, not a shipped one.
How it is built- Two sources per spine: a visual detector for the boxes, a language model for the text, merged by spatial proximity.
- Each approved detection keeps its box, its resolved book and any user correction: a label, a crop and a ground truth.
- The training export and the classifier are on the roadmap. The labels accumulate with every shelf in the meantime.
-
Every discovery surface is instrumented
Each place a reader meets the corpus emits an event, so the loop can be measured end to end: did the insight render, did the pair get clicked, did the reader take the intent gate, did a book get re-researched.
How it is built- Events on every discovery surface: overlay rendered, link clicked, pair card clicked, pair page viewed, intent gate chosen, own-collection checked, re-research completed, edition changed, retailer click.
- Aggregate stats rebuild nightly, and a digest goes out daily.
-
Why the loop keeps turning
Insights are computed live on every page load, so they change as shelves arrive. The free public API, the SDK and the MCP tools exist to bring shelves in from other agents, not just from the web app.
How it is built- Insights recompute on request. As the database grows, they change.
- A design record makes the free API a deliberate choice to grow the data.
- The MCP tools include shelf recognition and book pairings, so an agent can read a shelf and ask what sits beside a book.