Propensity Score
Your logs contain only what the old ranker chose to show. If an item was never put in front of a user there is no feedback for it, and that absence is not neutral — the system caused it. The propensity score is the number that makes this tractable: the probability that the logging policy showed a given item to a given user in the first place.
Why a probability is the fix
Treat every logged impression as a sample drawn with a known but unequal chance of being drawn, and ordinary survey mathematics applies. Dividing each observed reward by its propensity — inverse propensity scoring — re-weights the log so that under-shown items count for more, producing an unbiased estimate of how a different policy would have performed.
Where the numbers come from
- Logged probabilities — the clean case: a stochastic policy such as a bandit or an epsilon-greedy ranker records its own action probability at serving time.
- Estimated propensities — a model fitted after the fact to predict what the old policy would have shown, standard when the logging policy was deterministic.
- Position propensities — in ranking, the examination probability of a slot plays the same role, which is how position bias is corrected.
The catch
Small propensities blow up: dividing by 0.001 turns one lucky click into a thousand, so variance can swallow the bias you removed. Clipping, self-normalisation and doubly robust estimation exist to control that. And anything outside the logging policy's support is unrecoverable — a policy that never showed an item cannot tell you what would have happened if it had.
A propensity score is the probability that the logging policy showed a particular item to a particular user. Because logged feedback only covers what a previous policy chose to display, weighting each observation by the inverse of its propensity corrects that selection effect and makes off-policy estimates unbiased. Small propensities inflate variance, so clipping and doubly robust estimation are used alongside it.
Imagine judging which songs a radio station's listeners love, using only the station's own play history. Songs it played every hour got plenty of reaction; songs it played once at 3 a.m. barely got any. You cannot compare the two fairly without knowing how often each was played.
The propensity score is that missing number: the chance a particular thing was ever put in front of a particular person. Knowing it lets the software give rare chances extra weight, so a song that was almost never played but did well every time it was is not written off as unpopular.