Field-Aware Factorization Machine (FFM)

A factorisation machine gives every feature one latent vector and reuses it against every other feature. Field-Aware Factorization Machines question that single vector: should the value device = mobile really interact with a publisher the same way it interacts with an advertiser?

One vector per field, not per feature
FFM groups features into fields — every device value forms one field, every publisher another, every ad category a third. Each feature then learns a separate vector for each field it can be paired with, and the interaction between two features uses the vector each one keeps for the other's field. Where an FM stores k numbers per feature, an FFM stores k per field per feature, so the parameter count multiplies by the number of fields. The usual response is a far smaller k — four to eight, against the dozens an FM happily uses.

Where it earns the cost
FFM's home is click-through-rate prediction on categorical data, where it won the Criteo and Avazu competitions and became the reference implementation in libFFM. It pays off when:

• features fall into genuinely distinct fields whose interactions behave differently;
• there is enough data to fit the extra parameters — FFM overfits fast and needs early stopping and firm regularisation;
• the input is one-hot and sparse rather than dense.

Why it matters
FFM marks where modelling pairwise interactions explicitly reached its practical ceiling. Past that point the field moved to learned higher-order interactions in DeepFM and DLRM, which keep the factorised pairwise term and let a network discover the rest.