User Tower

In a two-tower retriever, one half of the model is the only part that has to run when a request arrives. That half is the user tower, and almost every serving decision in embedding-based retrieval follows from the asymmetry.

What it encodes
The user tower is a neural network mapping everything known about the requester into a single vector in the shared embedding space: a user-id embedding, aggregated history such as the mean or attention-pooled embeddings of recently watched items, profile attributes, and request-time context like device, hour or country. Nothing about the candidate item enters it — that restriction is precisely what makes the item tower's output cacheable.

Why it runs online
Item embeddings are precomputed into an ANN index such as FAISS or ScaNN. The user vector cannot be, because it depends on the session that just happened — so the user tower is executed per request, and its forward pass sits inside the retrieval latency budget of a few milliseconds. This is why user towers stay comparatively shallow while item towers can afford depth: one runs millions of times offline, the other once, right now.

What it does not see
Because no layer mixes user and item features, the tower cannot represent “this user clicks this brand on mobile”; such crosses are left to the ranking stage. Train it with in-batch negatives or sampled softmax, and keep the training-time feature pipeline identical to the serving one — a skew between them quietly moves the user vector away from the indexed item space, and recall falls with no error raised anywhere.