Factorization Machine (FM)

Matrix factorisation learns a vector for each user and each item and multiplies the two. Factorization Machines ask the obvious follow-up: what if every feature — the user, the item, the day of the week, the device in hand — got a vector of its own?

How the model is built
An FM takes one sparse feature vector per example and predicts a global bias, a weight per feature, and then a term for every pair of features, computed as the dot product of their learned latent vectors. That pairwise term is the whole idea. A plain quadratic model needs an independent parameter for each feature pair and never sees most pairs often enough to fit them; factorising the interaction into per-feature embeddings means evidence about Tuesday gathered from one item informs how Tuesday pairs with another. The sum rearranges so that training costs time linear in the number of non-zero features rather than quadratic.

Where it fits
Encode only user id and item id and an FM reproduces biased matrix factorisation exactly. Add fields and it extends smoothly to:

• Cold start, where a brand-new item has no interaction history but does have genre, brand and price features whose vectors are already trained.
• Context-aware ranking — time of day, location, device — folded in as ordinary fields instead of a separate model.
• Click-through-rate prediction on heavily categorical feed and ad data, the setting libFM was written for.

Why it matters
FMs are the bridge between factorisation and feature-based learning, and the direct ancestor of DeepFM and Deep & Cross Network, which keep the explicit second-order term and run a neural network alongside it.