Recommendation as Retrieval
A catalogue of fifty million items and a response budget of fifty milliseconds rule out the obvious approach: you cannot score every item on every request. Recommendation as retrieval treats that constraint as the problem statement rather than an engineering detail.
The reframing
Instead of ranking the catalogue, the system treats the request as a query. The user becomes a vector, every item becomes a vector in the same space, and recommendation becomes nearest-neighbour lookup: fetch the few hundred items closest to the user and pass them on. The two-tower model is the canonical form — separate encoders for user and item, trained so that the dot product of their outputs predicts engagement, with in-batch negative sampling supplying the items the user did not interact with.
What this buys
- Item vectors are computed offline and loaded into an index such as FAISS or ScaNN, so serving cost grows with the logarithm of the catalogue rather than linearly.
- Only the user tower runs at request time, which is what makes conditioning on fresh session context affordable.
- A new item becomes recommendable as soon as its features produce a vector — no interaction history needed, which softens item cold start.
What it gives up
A dot product cannot model interactions between user and item features, because the two towers never meet before the final multiplication. That is the price of the index, and the reason retrieval is always paired with a ranking stage: retrieval optimises Recall@k over a vast space cheaply, then a heavier model reorders the survivors with every feature available.
Recommendation as retrieval frames candidate selection as nearest-neighbour search: user and item are encoded into one vector space, and the top candidates are those closest to the user's vector. Item vectors are indexed offline with tools such as FAISS, so a catalogue of millions is searched in milliseconds instead of being scored exhaustively. It is the first stage of the retrieve-then-rank pipeline, measured by Recall@k.
Think of how you find a book in a huge library. You do not inspect every shelf — you walk to the section where books like the one you want live.
Recommendation as retrieval works the same way. Software arranges every item on a kind of map, so similar things sit near each other, and places you on that map too, close to what you tend to enjoy. Suggesting something is then just a matter of looking at what is nearby. This is how a service with millions of items answers in a fraction of a second: it never considers most of the catalogue at all.