Notes
🔍

Doubly Robust Estimation

Judging a new ranker from an old one's logs leaves you with two imperfect options: reweight the logged rewards by how likely the old policy was to show each item, or fit a model that predicts the reward for anything. The first is unbiased but wildly noisy; the second is calm but wrong whenever the model is wrong. Doubly robust estimation uses both, so that either one being right is enough.

How the two halves combine
The estimator starts from a reward model prediction for every item the new policy would show — complete coverage, low variance. It then adds an inverse-propensity-weighted correction computed only on the logged items: the gap between the observed reward and the model's prediction for that item, divided by the logging propensity. When the reward model is accurate those residuals sit near zero and almost nothing gets reweighted, so variance stays small. When the model is biased, the correction term cancels that bias, because the propensity-weighted part is unbiased by itself. Hence doubly robust: consistent if the propensities are correct or the reward model is.

Why it matters in recommendation
Logged feedback is never a random sample — it covers only what the previous ranker chose to show, and a rarely-shown item carries an enormous propensity weight.

  • It makes counterfactual estimates usable at catalogue scale, where plain IPS variance over a long tail swamps the signal.
  • The same construction appears inside training as a debiasing loss for click models, not only in evaluation.
  • It still needs known or estimated, non-zero propensities. An item the logging policy never showed cannot be rescued by either half.

Doubly robust estimation combines a reward model with an inverse-propensity correction to estimate how a new ranking policy would have performed from logs produced by a different one. The model supplies a low-variance prediction for every item; the propensity-weighted residual on logged items removes its bias. The estimate stays consistent if either the propensities or the reward model is correct, which makes off-policy evaluation practical on long-tail recommendation logs where plain IPS is too noisy.

Suppose a shop wants to know whether a new way of arranging its shelves would sell more, but all it has is last year's sales from the old arrangement. There are two ways to guess. Correct the old figures for how often each item used to be on display, or build a model that predicts how well anything would sell. The first is honest but jumpy. The second is steady but can be plain wrong.

Doubly robust estimation uses both at once, so getting either one roughly right is enough to trust the answer. It is how a recommendation system estimates what a change to its suggestions would do before risking it on real people.