Neural Matrix Factorization

Matrix factorization scores a user-item pair with an inner product, which forces every dimension to contribute independently and additively. That is a strong assumption about how taste combines. Neural Matrix Factorization (NeuMF) was built to test what happens when the model learns the combining function instead.

Two paths, one score
The pair runs through two branches at once. The generalized matrix factorization branch takes the element-wise product of a user and item embedding and passes it through a learned weight vector — an inner product with per-dimension weights, so plain factorization is a special case of it. The multi-layer branch concatenates a second, separate pair of embeddings and pushes them through hidden layers, where arbitrary interactions between dimensions can form. The two outputs are concatenated into a single output unit, keeping the linear structure the model can trust and adding nonlinear capacity on top. Each branch holds its own embedding table — sharing one caps the fused model's accuracy.

Trained on implicit feedback
Fitting uses binary cross-entropy over observed interactions plus sampled negatives rather than squared error on ratings, because the target data is clicks and plays, where a missing entry is ambiguous rather than negative. Training from scratch converges poorly, so the standard recipe fits the two branches separately and initialises the fused model from both.

Where it belongs
NeuMF is the canonical demonstration that the scoring function is itself learnable, and the ancestor of the deep ranking models in production use. Its cost is the mirror image: without a plain inner product there is nothing for FAISS to index, so it scores a few hundred shortlisted candidates in the ranking stage rather than retrieving from millions.