Notes
🔍

Funk SVD

The name is a misnomer worth clearing up first. Funk SVD is not a singular value decomposition. It is the gradient-descent matrix factorisation Simon Funk published during the Netflix Prize, and it is what most people actually mean when they say SVD in a recommendation context.

What it does
Each user gets a short vector of latent factors and so does each item; a predicted rating is their dot product plus a global mean, a user bias and an item bias. Training visits observed ratings one at a time, measures the error, and nudges both vectors along the gradient with an L2 penalty holding them small. The decisive property is that unobserved cells are simply skipped — a true decomposition would require the gaps be filled first, which on a matrix that is 99% empty means inventing most of the data.

What it set in motion
Funk's write-up established the template the field still follows:

  • Predict from observed entries only, never from an imputed matrix.
  • Keep biases separate from the interaction term, since much of the variance in ratings is this user is generous and this film is good.
  • Regularise and stop on a validation split, because a fifty-dimensional factorisation will cheerfully memorise.

Why it still matters
It is the baseline worth beating on MovieLens and what the Surprise library's SVD class implements. Its limits defined the next moves: squared error on ratings is the wrong objective for a top-N list, which led to BPR, and it cannot use implicit feedback without confidence weighting, which led to the ALS variant for implicit data.

Funk SVD is the gradient-descent matrix factorisation popularised during the Netflix Prize — despite the name, not a true singular value decomposition. It predicts a rating as the dot product of short user and item latent factor vectors plus global, user and item biases, trained by stochastic gradient descent over observed entries only with L2 regularisation. Skipping unobserved cells rather than imputing them is what makes it workable on an almost entirely empty matrix.

Suppose you had to describe every film with a handful of dials — how funny, how dark, how fast-moving — and describe every viewer with the same dials, meaning how much they enjoy each of those qualities. Multiply the two sets of settings and you get a guess at the rating that viewer would give that film.

Funk SVD is a way of discovering those dials automatically. It starts with random settings, compares its guesses against ratings people really gave, and adjusts a little after each one, over and over. Nobody tells it what the dials mean; they emerge from the ratings themselves. The idea came out of the Netflix Prize competition and is still the first thing most recommendation courses teach.