Differential Privacy in RecSys
A recommender's quality comes from the specifics of what individual people did, and that is exactly what makes its outputs leakable. Differential privacy is the formal answer: a mathematical guarantee that whether any one user's history was in the training data cannot be determined from what the system produces.
What the guarantee says
An algorithm is differentially private if removing a single user's interactions changes the probability of any output by at most a small multiplicative factor, set by the privacy budget ε. A smaller budget means stronger protection and more injected noise. In recommendation the guarantee is usually obtained through DP-SGD — clipping per-user gradients and adding calibrated Gaussian noise while training the matrix-factorisation or neural model — or by randomising the interaction data itself before it reaches the server.
Why it is hard here
The guarantee fights the structure of the data:
- Interaction matrices are sparse, so a tail item's signal can rest on a handful of users and is destroyed by noise a popular item shrugs off.
- Protection must be defined per user rather than per record, and one person contributes thousands of events, which widens the noise required.
- The accuracy loss lands disproportionately on cold-start users and the long tail, so privacy trades directly against coverage.
Where it fits
It composes with federated recommendation, which keeps raw histories on the device but still leaks through the updates it sends; differential privacy is what bounds that residual leak. The de-anonymisation of the Netflix Prize dataset is the standard illustration of why removing names is not a defence and a formal budget is.
Differential privacy in recommendation is a formal guarantee that a model's outputs reveal almost nothing about whether any single user's interactions were in its training data, enforced by clipping per-user gradients and adding calibrated noise. It bounds the leakage that anonymisation alone cannot, including in federated setups, at a measurable accuracy cost that falls hardest on sparse long-tail items and new users.
Suppose a survey asks something embarrassing, and to protect you it tells everyone to flip a coin and lie if it comes up heads. Your individual answer is now deniable, but with enough people the overall pattern still comes through. Differential privacy applies that trick to recommendations.
Noise is added deliberately while the system learns from people's viewing and buying, in a carefully measured amount. The result is a provable promise: nobody can tell from the recommendations whether your history was used at all. The cost is accuracy. Blurring the data hurts the rare, niche items least known to the system, so stronger privacy means slightly duller suggestions — a trade a service chooses on purpose.