Item Tower

A retrieval model cannot afford to think about you and a candidate item at the same time — with ten million items in the catalogue, that is ten million forward passes per request. The item tower is half of the trick that avoids it.

What it is
In a two-tower model, two separate neural networks map into one shared embedding space: the user tower encodes the request — user id, recent history, context — and the item tower encodes the candidate — item id, category, text and image features, age. The towers share no weights and never see each other's inputs. Their only meeting point is the final dot product, or cosine, that scores the pair.

Why the separation is the whole point
Because the item tower depends on nothing about the user, every item embedding can be computed in a nightly batch job and indexed in FAISS or ScaNN. At request time only the user tower runs, once, and retrieval becomes approximate nearest-neighbour search over that index. This is how candidate generation returns a few hundred items out of millions inside a few milliseconds.

What it costs
The constraint that buys the speed also caps the accuracy:

• No feature crosses between user and item are possible, so the subtle interactions are left to the ranking stage downstream.
• The index goes stale between refreshes, and retraining the tower invalidates every stored vector at once.
• Training needs negatives the logs do not contain, supplied by in-batch negatives or sampled softmax.