Notes
🔍

NDCG

Precision@10 treats a hit in position one and a hit in position ten as the same event. Readers of a recommendation list do not. NDCG — normalized discounted cumulative gain — is the metric that takes position seriously, and it is the number most recommendation papers report.

How the number is built

  • Gain — each recommended item contributes a relevance value: 1 for an observed interaction, or a graded value such as a rating.
  • Discounted — the gain at rank i is divided by log2(i+1), so position 1 counts fully and position 10 counts about a third.
  • Cumulative — the discounted gains are summed over the top k, giving DCG.
  • Normalized — DCG is divided by the DCG of the perfect ordering, putting every user on a 0–1 scale so a user with one relevant item and a user with fifty can be averaged together.

Why it matters
Ranking quality is what retrieval and ranking stages are actually tuned for, and NDCG is the metric that notices when a model finds the right items but orders them badly — the failure a top-k set metric scores as a success.

Two cautions
Computing it against a sampled candidate set instead of the full catalogue distorts the discount and can reorder which model looks best — the documented sampled-metrics problem. And logged feedback only covers what the previous system chose to show, so a higher offline NDCG routinely fails to turn into more clicks.

NDCG (normalized discounted cumulative gain) scores a recommendation list by summing each item’s relevance after dividing it by a logarithmic position discount, then normalizing by the best possible ordering so every user lands on a 0–1 scale. It is the standard rank-aware accuracy metric because it penalises burying a relevant item deep in the list — something Precision@k ignores entirely. Sampled candidate sets distort it badly.

Two search results pages can contain exactly the same good links and still feel completely different — one puts the useful link first, the other buries it at the bottom.

NDCG is a score that captures that difference. It rewards a system for putting the things you actually wanted near the top, and it counts an item for less and less the further down the list it sits. The score is then adjusted so that a person with one favourite and a person with fifty can be compared fairly. It is the usual way researchers answer a simple question: did the system get the order right, not just the contents?