Generalized Matrix Factorization (GMF)
Classic matrix factorization makes one decision for you before training even starts: once a user and an item each have a vector, you multiply the matching dimensions and add the results up. Generalized Matrix Factorization keeps the multiplying and lets the model learn the adding.
What the generalisation is
GMF takes the element-wise product of the user and item embeddings — a vector whose entries say how strongly the two agree on each latent dimension — and feeds it to a learned output layer with its own weight vector and activation, instead of collapsing it with a plain sum. Ordinary MF is the special case where every output weight is one and the activation is the identity, so the generalisation costs nothing: the model can down-weight a latent dimension that carries little signal and sharpen one that carries a lot.
How it is trained
The output passes through a sigmoid and the model is fit with log loss over observed interactions plus sampled negatives — commonly four negatives per positive on a dataset like MovieLens-1M — rather than squared error on ratings. It is an implicit-feedback model from the start, predicting the probability that an interaction happens at all.
Why it matters
- It is the linear half of NeuMF, fused with a multi-layer path that keeps its own separate embeddings, so each branch learns the kind of interaction it is good at.
- The learned weights fold back into the embeddings, so item vectors stay indexable for approximate nearest-neighbour retrieval with a library such as FAISS.
- Replacing a fixed inner product with a learned function is the step that opens the door to the rest of neural collaborative filtering.
Generalized Matrix Factorization replaces the fixed inner product of matrix factorization with a learned one: it takes the element-wise product of the user and item embeddings and passes it through a trained output layer with its own weights and activation. Plain MF is the special case of uniform weights. Fitted with log loss on observed interactions plus sampled negatives, it serves as the linear branch of neural collaborative filtering models such as NeuMF.
Picture a shop assistant comparing you with a product on a handful of tastes — how much you both lean comedic, how much you lean recent, how much you lean long-form — and getting an agreement score for each one.
Older methods simply add those scores together, treating every taste as equally important. Generalized Matrix Factorization lets the system learn the weighting instead, so a taste that reliably predicts what you click counts for more than one that rarely does. It is a small change with a large consequence: the rule for turning a comparison into a score stops being something a designer fixed in advance and becomes something the data decides.