Notes

Hit Rate

A list of ten recommendations either contains something the person actually wanted or it does not. Hit rate is that yes-or-no question, asked once per user and averaged.

How it is computed
Hold out some of each test user's interactions, generate a top-k list from what remains, and record a hit if any held-out item appears in it. HR@k is the fraction of users who got a hit. Under the common leave-one-out protocol — one held-out item per user — hit rate coincides with Recall@k. The metric is binary and blind to position: a hit at rank one and a hit at rank ten score identically, which is why it is reported next to NDCG or MRR rather than alone.

Reading it honestly
The number means nothing without its protocol. HR@10 measured against a hundred sampled negatives is a far easier test than HR@10 over a catalogue of a million items, and sampled candidate sets can reorder which model looks best. A popularity-only baseline also scores higher than newcomers expect, so report it alongside.

Why it matters
Hit rate maps directly onto the product question for a fixed shelf of recommendations: did the slot earn its place. It is cheap, it is interpretable, and it is the headline number across much of the implicit-feedback literature. It is also an offline metric computed on logged feedback the previous ranker shaped, so an improvement in HR is a reason to run an A/B test, not a substitute for one.

Hit rate is the fraction of test users whose top-k recommendation list contains at least one held-out interaction. It is binary and position-blind — a hit at rank one counts the same as one at rank ten — so it is read alongside rank-aware measures such as NDCG. Under leave-one-out evaluation it equals Recall@k. Its value depends entirely on k and on the candidate set, so a figure quoted without its protocol is not comparable.

Suppose you hide one film a person went on to watch, ask the system for ten suggestions, and check whether that film is in the list. It either is or it is not.

Do this for thousands of people and the share of times the answer is yes is the hit rate. It is popular because it asks the blunt question a shelf of ten recommendations really faces: was anything on it wanted. What it does not notice is where on the list the wanted thing turned up — top of the shelf and bottom of the shelf count the same — so it is always read next to measures that reward being first.