Wide & Deep
Two instincts pull against each other in a click model: memorise the exact feature combinations that have worked before, or generalise to combinations never yet seen. Wide & Deep refuses to choose and trains both at once.
Two halves, one loss
The wide half is a linear model over raw and cross-product features — “installed app is Netflix AND impressed app is Pandora” treated as a single binary feature. It memorises: if that exact pair converted historically, the weight says so. The deep half embeds each categorical feature into a dense vector and feeds the concatenation through an MLP, which generalises to pairs that never co-occurred in training. The two outputs are summed into one logit and trained jointly on the same objective, so each half learns only what the other cannot supply.
Why joint training matters
Stacking two separately trained models would leave each carrying the full burden of accuracy. Trained together, the wide part can stay small — a handful of hand-chosen crosses — because the deep part covers the rest. Google's 2016 paper ran it on Play Store app recommendation and reported a measurable lift in installs over either half alone.
Where it sits
This is a ranking-stage model: it scores a few hundred candidates with full cross-features, after retrieval has already narrowed the catalogue. Its cost is the manual feature engineering in the wide half, which DeepFM and DCN were designed to automate.
Wide & Deep is a click-prediction architecture that trains a linear model over cross-product features alongside a deep network over feature embeddings, summing both into a single prediction and optimising them jointly. The wide half memorises feature combinations present in the data; the deep half generalises to ones that are not. Used at the ranking stage, it captures cross-feature accuracy a plain MLP misses, at the cost of hand-designed crosses.
Think of a shop assistant with two kinds of knowledge. One is pure memory: customers who bought this particular brand of boots almost always bought that particular wax. The other is judgement: this customer looks like a hiker, so outdoor gear is probably a good bet — even for products nobody quite like them has bought before.
Wide & Deep gives a recommendation system both at once. The wide part remembers exact combinations that worked; the deep part makes educated guesses about combinations it has never seen. Training them together means the system can be precise where it has evidence and sensible where it does not.