Embedding (RecSys)
A user id and an item id are only labels — nothing about user 41,208 tells a model anything. An embedding is the dense vector of numbers the model learns to stand in for that label, positioned so that distance in the vector space carries meaning about taste.
Where the numbers come from
The embedding table is a lookup: one row of perhaps 32 to 256 floats per user and per item, initialised at random and learned by gradient descent. In matrix factorization the training signal is the fit to observed interactions — a user vector dotted with an item vector should reproduce a rating, or rank an observed interaction above an unobserved one. The user factors and item factors are therefore embeddings, and the latent dimensions are simply their coordinates. Nothing names those dimensions; they fall out of co-occurrence, and while one may loosely track something like animation versus live action, most correspond to nothing describable.
Why it matters
Embeddings are what let recommendation generalise across a sparse matrix: two items nobody consumed together can still sit close if they attract similar users.
- Scoring collapses to a dot product, so millions of candidates can be ranked with one matrix multiply or an ANN lookup.
- The same representation carries into neural recommenders — Item2Vec, two-tower retrieval, sequence models — which replace the dot product but keep the table.
- Most of a production recommender's parameters live here, and a cold item is precisely one with too few interactions to have moved far from its random initialisation. That is why embeddings are where cold start bites and where side-information fixes are applied.
An embedding is a dense learned vector representing a user or item, trained so that closeness in the vector space reflects interaction behaviour. In matrix factorization the user factors and item factors are exactly these vectors, and a dot product between them predicts preference. Embeddings make generalisation across a sparse matrix possible and retrieval over a large catalogue tractable; items with too few interactions keep near-random vectors, which is cold start.
Software cannot do much with a name or an account number. So a recommendation system turns every person and every item into a short list of numbers — rather like plotting them on a map, where things that appeal to the same people end up near each other.
Those coordinates are an embedding. Nobody decides what each number means; the system works them out from what people watched, bought or skipped. Once everything sits on the map, making a suggestion is close to asking which items are near you. It also explains a familiar weakness: a brand-new item has no history, so it has not really been placed on the map yet, and the system honestly does not know where it belongs.