Multi-Stage Recommender
A catalogue of ten million items and a latency budget of fifty milliseconds rule out the obvious design: you cannot score every item with your best model and sort the results. A multi-stage recommender is the architecture that falls out of that constraint — spend almost nothing per item early, spend a lot per item late.
The stages
- Candidate generation — several cheap retrieval sources (a two-tower model queried through a vector index, co-visitation lists, trending items) each return a few hundred items from the full catalogue.
- Ranking — one heavy model scores the few thousand survivors with the full feature set, including cross features far too expensive to compute catalogue-wide.
- Re-ranking — diversification, deduplication and business rules shape the final slate from the ranked list.
Why the split works
Each stage only has to be good at its own job. Retrieval optimises recall, and losing a relevant item there is unrecoverable — no later stage can see what was never fetched. Ranking optimises precision over whatever survived, with a model it could never afford to run catalogue-wide. YouTube’s published two-stage design is the canonical example, and nearly every large system now looks like it.
Where it breaks
The stages train separately on different distributions, so a ranker learns only from items the current retrieval policy chose to send it and scores unfamiliar candidates badly. Retrieval recall is also easy to leave unmeasured, which hides a ceiling no amount of ranker tuning can lift.
A multi-stage recommender serves recommendations through a pipeline instead of one model: cheap retrieval sources narrow millions of items to a few thousand candidates, a heavy ranker scores those with the full feature set, and a re-ranking pass applies diversity and business rules. The split is what makes an expensive model affordable inside a latency budget. Its weak point is retrieval recall — anything missed early can never be recovered.
Hiring for one job out of ten thousand applicants, nobody interviews everyone. A quick filter cuts the pile to a few hundred, a proper screen gets it to a dozen, and only those few get the long interview.
A multi-stage recommender does the same with a catalogue. A fast, rough method picks a few hundred possibilities out of millions, a much more careful model ranks just those, and a final pass tidies the list so it isn’t five near-identical items. All of it happens while the page is loading. The catch is the same as in hiring: a good candidate dropped by the first quick filter never gets another look.