Building Scenes

Scenes is a discovery platform for films and books, and the core challenge is cross-modal recommendation — how do you suggest a book to someone based on a film they loved, when the two media share no surface features? A film has cinematography, pacing, performances. A book has prose style, narrative structure, voice. The connection lives in the thematic space between them, and that's where embeddings help.

I built multi-view embeddings for each item. Rather than a single vector per film or book, each gets multiple embeddings capturing different aspects: narrative themes, emotional tone, aesthetic sensibility, cultural context. A Tarkovsky film and a Sebald novel might be distant in a single-embedding space but close in the "meditative pacing + memory as theme" subspace. Multi-view lets the system find these diagonal connections.

The recommendation pipeline works in stages. First, a broad retrieval using IVFFlat indexes on pgvector — fast approximate search across the full catalogue. Then Voyage AI reranking to sort the candidates by actual relevance. The reranker is the crucial step because the initial retrieval is intentionally loose. You want to cast a wide net and then filter intelligently, rather than doing precise retrieval upfront and missing interesting lateral connections.

Cross-modal is the key innovation. Most recommendation systems stay within their medium — film-to-film, book-to-book. Scenes deliberately crosses the boundary. The interface asks "If you loved Stalker, read..." and surfaces literary fiction that shares something essential with Tarkovsky's vision. Not adaptations or novelisations, but works that resonate in the same emotional frequency.

The embedding pipeline processes text descriptions, reviews, and metadata for each item. Films get additional signal from director filmography and cinematographic style descriptions. Books get signal from prose excerpts and critical reception. All of this feeds into Voyage AI to produce the multi-view vectors.

Discovery is the product, not search. You don't come to Scenes knowing what you want — you come with a starting point and follow the connections outward. The interface is designed for browsing: pick a film, see the constellation of related works across media, follow a thread into unfamiliar territory. The best recommendation is the one that surprises you while still feeling inevitable.