DeepFM
Wide & Deep works, and the catch sits in its wide half: somebody has to decide which feature pairs belong there. DeepFM removes the person from that step and lets a factorization machine learn the pairings instead.
Two halves, one embedding layer
DeepFM runs an FM component and a deep component side by side and sums their outputs before the final sigmoid. The FM half models every second-order feature interaction as the dot product of two learned embeddings, so it handles sparse conjunctions that appear rarely in the training data. The deep half is an MLP over the same embeddings, picking up higher-order and smoother structure. The important detail is that both halves share one embedding layer and train jointly end to end — there is no separate feature-engineering stage and no pre-training.
What that buys
In CTR prediction the features are overwhelmingly high-cardinality categoricals, and the signal lives in combinations: app category with user age, publisher with device type. Learning a weight for each such pair directly is impossible, because most pairs are never observed, while the FM factorisation shares strength across them through the embeddings. Compared with Wide & Deep, DeepFM needs no hand-crafted crosses; compared with a plain FM, it is not limited to order two.
In practice
DeepFM remains a strong ranking-stage baseline on Criteo and Avazu, and the first thing to try before reaching for DCN or a transformer ranker. Implementations exist in DeepCTR and TorchRec, and it trains comfortably on a single GPU.
DeepFM is a click-through prediction model that sums a factorization machine component, which learns every second-order feature interaction as a dot product of embeddings, with a deep MLP that captures higher-order structure — both sharing one embedding layer and trained jointly. It delivers Wide & Deep's accuracy without hand-crafted cross features, which is why it is a standard ranking-stage baseline on sparse categorical ad data.
Imagine guessing whether someone will tap an advert. On its own, knowing they use an Android phone tells you little. Knowing they use an Android phone and are reading a cooking blog on a Sunday evening tells you much more. The useful clues are pairs and groups of facts, not single facts.
DeepFM is a prediction method built around that idea. One half of it systematically checks every pair of available facts for a useful connection. The other half looks for broader patterns that pairs alone cannot capture. Earlier systems needed engineers to guess which pairs mattered and write them down by hand; DeepFM finds them itself, which is why it became a common starting point for ranking adverts.