Position Bias

Put the same item in the first slot of a recommendation list, then in the tenth, and it collects very different numbers of clicks. Nobody changed their mind — the top of a list gets looked at and the bottom rarely does. Position bias is that effect: the chance a user examines a slot at all, separate from whether the item sitting there is any good.

Why logged clicks mislead
Implicit feedback is the only training signal most recommenders have, and it arrives already shaped by the ranker that produced it. A click means examined and liked. A missing click means disliked, or never seen, and the log cannot tell those apart. Train on raw clicks and the model learns the old layout instead of preference, re-promoting whatever already sat high.

Separating examination from relevance

Click models give each rank its own examination probability, fitted across millions of impressions.
Propensity estimation recovers those probabilities from small doses of result randomisation, or from naturally occurring swaps already in the logs.
Inverse propensity scoring down-weights clicks earned in prominent slots, which is the basis of unbiased learning to rank.

Why it matters
Every offline evaluation built on logged interactions inherits the skew, so a new ranker can win on NDCG and lose on click-through. Uncorrected, position bias compounds popularity bias: prominent items earn clicks, clicks earn prominence, and better items below the fold never get the exposure to prove it.

Position bias is the tendency of users to click items simply because they appear high in a list, independent of relevance. Because logged feedback records clicks but not whether a slot was examined, raw interaction data overstates the quality of top-ranked items. Correcting it — through click models and inverse propensity scoring — is what makes offline evaluation and unbiased learning to rank trustworthy rather than a measurement of the previous ranking.

Think about a supermarket shelf. Things at eye level sell more than things near the floor, even when they are no better. People reach for what is easiest to see.

The same thing happens with recommendations. Whatever sits at the top of the list gets clicked far more, just for being first. That creates a problem: the software learns from clicks, so it decides the top items must be the good ones — when really they were only the visible ones. Position bias is the name for this distortion, and good systems measure it deliberately so they can subtract it out before drawing any conclusion about what you actually like.