Notes

Click-Through Rate (CTR)

Of all the numbers a recommender gets judged on, click-through rate is the one that needs no explaining: of the items you showed, what fraction did people click?

The definition lives in the denominator
CTR is clicks divided by impressions, and the care goes into what counts as an impression — an item rendered and actually in view, not merely returned by the ranker. Position matters enormously: the top slot can earn several times the clicks of the tenth for identical content, so CTR describes an item in a slot in a context, never an item on its own. Comparing it across layouts without correcting for position bias is the classic route to a confident wrong conclusion.

Two different jobs
The same ratio does duty as a measurement and as a target:

  • As an online metric it is the headline figure of an A/B test or an interleaving experiment — cheap, fast-moving, and usually what a launch decision rests on. Offline NDCG improving while live CTR falls is a routine outcome, not a surprising one.
  • As a prediction target, predicted CTR is the ranking score itself in feeds and advertising. DeepFM, DLRM and DIN are all trained on click labels with log loss, and calibration becomes essential once that prediction gets multiplied by a bid.

What it hides
CTR is trivially gamed. A lurid thumbnail lifts clicks and lowers satisfaction, and a system tuned on clicks alone drifts towards popular, safe, already-familiar items. That is why it is read alongside dwell time, conversion, or a composite engagement objective — a click is a vote cast before the user knows what they voted for.

Click-through rate is clicks divided by impressions — the fraction of shown items a user actually opens. It is the headline online metric for a recommender and, as predicted CTR, the ranking score itself in feeds and advertising. Because position strongly affects clicks, it is only comparable within a fixed layout. It is also easy to game, since clickbait raises it while satisfaction falls, so it is read alongside dwell time or conversion.

Click-through rate is a simple score: out of everything a system put in front of people, what share did they actually click? Show a thousand thumbnails, collect fifty clicks, and that is a five per cent click-through rate.

It is popular because it is easy to measure and moves within hours, so a company can tell quickly whether a change to its suggestions helped. It is also easy to cheat. A shocking headline or a misleading picture earns the click and leaves the person disappointed, so a click on its own proves nothing about whether the suggestion was any good. That is why it is usually read next to how long people actually stayed.