SVD for Recommendation
Singular value decomposition is a clean piece of linear algebra: any matrix splits into user directions, item directions, and the strengths that connect them. Pointing it at a ratings matrix is where recommendation's dominant idea came from — and where its most persistent naming confusion started.
The textbook decomposition and its problem
Truncated SVD gives the best rank-k approximation of a complete matrix. A user-item matrix is not complete, it is mostly missing, and missing is not zero. Filling the gaps with zeros or row means and then decomposing distorts the geometry, and the work scales with the full grid you cannot hold in memory anyway. Applied literally, classical SVD is the wrong tool for this data.
What "SVD" means in practice
What the field has called SVD since the Netflix Prize is a different animal: a latent-factor model predicting the dot product of a user vector and an item vector plus user and item biases, fitted by gradient descent on the observed entries only. Nothing is imputed and no orthogonality is enforced — the name survived from Simon Funk's original write-up. Its extension, SVD++, adds the set of items a user interacted with as an implicit-feedback term. The Surprise library ships both under exactly these names.
Why it matters
- It established the representation modern systems still use — a dense embedding per user and item, compared by dot product, which is what makes approximate nearest-neighbour retrieval over an index like FAISS possible.
- It optimises squared error on ratings, so for implicit top-N ranking it gives way to weighted ALS and BPR.
SVD for recommendation covers two related things. Truncated singular value decomposition yields the optimal low-rank approximation of a complete matrix, but a user-item matrix is mostly missing and imputing zeros distorts it. What recommender work calls SVD is therefore a latent-factor regression — user and item embeddings plus biases, fitted by gradient descent on observed entries only. It established the embedding-and-dot-product representation that retrieval models still rely on.
Suppose you described every film with a handful of hidden dials — how funny it is, how dark, how fast-moving — and described every viewer with the same dials, showing how much they care about each quality. Multiply one person's dials by one film's and you get a guess at how much they would enjoy it.
SVD is the mathematics that works out those dials for you, purely from the ratings people have already given. Nobody tells it what the dials should mean; it invents whichever few qualities best explain the data. The trick made its name in the Netflix Prize and still underpins modern systems, which compare compact numerical sketches of people and things.