Notes
🔍

Recall@k

If a recommender shows you ten items, one question comes before any question about ordering: did the thing you actually wanted make it into those ten at all? Recall@k measures exactly that.

How it is computed
For one user, take the items they are known to have engaged with in the held-out period, count how many appear in the top k recommendations, and divide by that user's total number of relevant items; then average over users. Recall@10 of 0.3 means the list captured three in ten of the items the user went on to interact with. Position inside the list is ignored — rank 1 and rank 10 count the same.

What it is good for

  • It is the natural metric for the retrieval stage, whose only job is to not lose good candidates: an item missing from the retrieved few hundred cannot be recovered downstream.
  • Paired with Precision@k it separates two distinct failures — missing relevant items, and padding the list with irrelevant ones.
  • It stays stable under sparse ground truth, which is why MovieLens leave-one-out benchmarks report Recall@10 and Recall@20 by default.

How it misleads
Recall rises automatically as k grows, so numbers are comparable only at a fixed k that matches the real slate size. It also inherits the central bias of offline evaluation: an item the logging policy never showed is counted as irrelevant, which quietly rewards the model that agrees most with the system already in production.

Recall@k is the fraction of a user's relevant items that appear anywhere in their top k recommendations, averaged across users; order within the list does not count. It is the primary metric for the retrieval stage, since a relevant item absent from the candidate set can never be ranked back in. It is reported at a fixed k only, because recall rises mechanically as the list grows.

Suppose you pack a bag for a friend and only ten things will fit. Afterwards you check how many of the things they actually needed made it in. That score is the idea behind Recall@k.

Here the bag is the short list of suggestions a system shows you, and the k is simply how many slots it has. Recall asks what share of the things you genuinely wanted were included. It does not care whether your favourite sat first or last, only whether it got in at all. That makes it the right question for the earliest stage of a recommender, whose whole job is to avoid leaving good things out.