In-Batch Negative Sampling
Training a retrieval model means teaching it to prefer the item a user actually chose over the millions they did not. You cannot score millions of items per training example, so you need a handful of convincing stand-ins — and the cheapest convincing ones are already sitting in the batch.
Other people's positives, reused
A training batch for a two-tower model holds, say, 1,024 (user, item) pairs that really happened. Encode both towers once, then score every user in the batch against every item in the batch: the diagonal holds the true pairs, and the other 1,023 items act as negatives for each user. One softmax over the batch buys 1,024 comparisons per example at the cost of 1,024 item encodings rather than a million.
The popularity skew it introduces
Because the negatives are drawn from logged traffic, popular items appear as negatives far more often than rare ones, and the model over-penalises them. The standard fix is a logQ correction — subtracting the log of an item's estimated sampling frequency from its score — which is why production two-tower recipes pair in-batch negatives with a running frequency estimate.
Why it decides retrieval quality
This is the training-side counterpart to ANN retrieval: the embedding space that FAISS later searches is shaped entirely by the negatives it was trained against. Too few, or too easy, and the retrieval stage hands the ranker plausible-looking neighbours that no amount of ranking can rescue. Large batches are valuable here for a reason that has nothing to do with gradient noise.
In-batch negative sampling trains a retrieval model by treating the other examples in the same minibatch as negatives: every user in the batch is scored against every batch item, with the true pairs on the diagonal. It supplies many comparisons per step without encoding extra items, which is what makes two-tower training affordable at catalogue scale. Because the negatives come from logged traffic, popular items are over-sampled and need a logQ correction.
To learn what you like, a system also has to see things you did not pick. Checking your choice against every product in a catalogue of millions would be impossibly slow, so it needs a small, realistic set of alternatives instead.
In-batch negative sampling borrows them from other people. While the system works through a group of examples together, each person's choice doubles as a "someone didn't pick this" example for everyone else in the group. It costs almost nothing and still gives the system plenty to compare against. The catch is that whatever is already popular turns up in those comparisons constantly, so the system has to be told not to punish popular things unfairly.