Implicit Feedback Bias

A click looks like a vote. It is not. It records that the system put an item in front of someone and that they looked — and says nothing about the thousands of items they were never offered. Implicit feedback bias is the gap between those two readings, and it is baked into every interaction log.

Where the distortion enters

• Exposure — an item never shown cannot be clicked, so its absence from the log is evidence about the old ranker, not about the user.
• Position — the top slot collects clicks regardless of what sits in it.
• Presentation — a striking thumbnail buys the click, and the disappointment that follows is never logged.
• The loop — tomorrow's model trains on what today's model chose to display.

What it does to a model
Train on this log as though missing meant unwanted and the model learns the previous policy's exposure pattern: head items are confirmed, the tail stays unseen because it was unseen, and the system narrows what anyone is ever shown. The damage is invisible offline, because the held-out test data carries the same bias as the training data — which is the standard explanation for an offline metric that improves while click-through falls.

How it is handled
Inverse propensity scoring reweights each observed interaction by the reciprocal of its estimated chance of being shown. Exposure models treat visibility as a latent variable inferred alongside preference. A small randomised slice of traffic yields genuinely unbiased evaluation data, and interleaving or an A/B test settles what the log cannot.