Relevance (RecSys)

"Relevant" sounds like a judgement about an item. In recommendation it is a judgement about a pair — this user, this item, in this moment — and almost never one anybody wrote down.

What the word means here
Relevance is the degree to which an item would satisfy a particular user if it were shown to them. Everything a recommender does is an estimate of it: a predicted rating, a click probability, an inner product between a user and an item embedding. The crucial property is that relevance is not a property of the item — a documentary is relevant to one account and noise on the next — and not binary, though evaluation usually flattens it that way.

Where the labels come from

Explicit signals, such as a rating or a thumbs-up, state relevance directly and arrive rarely.
Implicit signals — a click, a play, dwell time, a purchase — stand in for it, and each is a biased proxy: a click can mean a misleading thumbnail, and no click on an item never shown means nothing at all.
Human relevance judgements, borrowed from information retrieval, exist for some evaluation sets but do not scale to a personal catalogue.

Why the definition carries weight
Every offline metric — Precision@k, NDCG, hit rate — needs a relevance label to score against, so the proxy chosen quietly defines what "good" means. Treat a click as relevance and the system learns clickbait; treat completion or return visits as relevance and it learns something closer to satisfaction. Most arguments about a recommender's quality are really arguments about this substitution.

Relevance is the degree to which an item would satisfy a specific user in a specific context — a property of the user-item pair, not of the item, and almost always unobserved. Recommenders estimate it as a predicted rating, a click probability or an embedding similarity, and offline metrics such as NDCG and Precision@k score lists against a proxy label drawn from implicit feedback. Whichever proxy is chosen defines what the system actually optimises.

Ask whether a film is good and people will argue forever. Ask whether it is the right film for you tonight, and there is a real answer. Relevance is that second kind of judgement.

It is never about the item on its own. The same recipe is perfect for one person and useless to another, so relevance lives in the match between a person and a thing at a particular moment. The catch is that nobody tells the software what is relevant. It has to work it out from behaviour — what you clicked, what you watched to the end, what you quietly scrolled past — and those clues are imperfect stand-ins for what you really wanted.