Off-Policy Evaluation (RecSys)
Every log a recommender leaves behind was written by the recommender you already had. It records what users did with the items that system chose to show — and nothing about the items it never showed. Off-policy evaluation answers the question that gap creates: how would a new ranking policy have performed, judged only from the old one's data?
Why logs are not a test set
A supervised test set is drawn from the same distribution as training. Recommendation logs are not. The logging policy picked every slate, so items it favoured are over-represented, and the outcome under a different ranking is absent rather than merely noisy. Scored naively, the log rewards whichever candidate agrees most with the old system.
Correcting for what was shown
- Inverse propensity scoring — divide each logged reward by the probability the logging policy gave that item, so rarely shown items count for more. Unbiased, but the variance explodes on small propensities.
- Clipped IPS — cap the weights, accepting bias in exchange for a usable estimate.
- Doubly robust estimation — pair propensity weights with a learned reward model, so either component being right keeps the estimate close.
What it buys you
It is the gate before an A/B test, not a replacement for one: candidates that estimate clearly worse never consume live traffic. Bandit and reinforcement-learning recommenders depend on it outright, since each policy update has to be judged from the previous policy's data. The catch is operational — propensities must be logged at serving time, and no amount of later analysis recovers them.
Off-policy evaluation estimates how a new recommendation policy would have performed using interaction logs generated by a different, earlier policy. Because logged feedback only covers items the old system chose to show, estimators reweight each observation by its propensity — the probability it was shown — through inverse propensity scoring or doubly robust variants. It is the screen that keeps weak candidate rankers off live traffic, and it fails outright if serving-time propensities were never logged.
Imagine judging a new film critic using only the reviews written by the old critic. You know what people thought of the films the old critic picked — and nothing about the ones she ignored. That is the puzzle off-policy evaluation solves. A company wants to know whether a new way of suggesting things would do better, but all it has is a record of how people reacted to the old suggestions. So it does the counting carefully: suggestions that were rare in the old records get weighted more heavily, to undo the old system's habits. It is a way of testing a change before any real user sees it.