Categorical Features (RecSys)
Most of what a system knows about an item is not a number. A film's genre, a product's brand, a track's language — these are labels, and labels are the raw material a content-based recommender works from.
Turning labels into vectors
A categorical feature takes one value from a finite, unordered set, so arithmetic on it is meaningless and it has to be encoded. Small vocabularies become one-hot vectors. Multi-valued fields such as tags or genres become multi-hot vectors, often TF-IDF weighted so a rare tag counts for more than a ubiquitous one. Large vocabularies get learned embeddings, and open-ended ones — brand names appearing faster than any vocabulary can be rebuilt — get the hashing trick. Cardinality here is extreme: the item id is itself a categorical feature with millions of values, which is why industrial models such as DLRM and DeepFM are mostly embedding table by parameter count.
Why they carry the cold-start load
These features are what a recommender has before any interaction exists:
- A film released this morning has no viewers but still has a genre, a cast and a language, so content features are what let it be scored at all.
- Their value is usually in combination — thriller plus Korean plus short runtime — which is exactly what factorisation machines and cross networks learn without anyone enumerating the pairs by hand.
- Rare values are a trap: a brand seen three times earns a noisy embedding, so low-count values are normally bucketed into a shared slot.
The ceiling
Categories only discriminate as finely as they are defined. Twenty genres cannot separate two thousand dramas, and a profile-matching step built on them alone converges on recommending more of the same category. That is the structural reason content features serve as a cold-start bridge while interaction signal takes over once it exists.
Categorical features are item or user attributes taking one value from a finite, unordered set — genre, brand, language, or the item id itself. They enter a model as one-hot vectors, multi-hot vectors for multi-valued fields such as tags, or learned embeddings when the vocabulary is large. They are the only signal available for an item with no interaction history, which makes them the standard answer to new-item cold start.
Categorical features are the labels attached to something rather than measurements of it. A film has a genre, a language and a director. A jacket has a brand and a colour. None of these are numbers you can add up — comedy is not halfway between drama and horror.
Software needs numbers, so it quietly turns each label into a small set of figures it learns over time, which is how it works out that two unrelated-looking brands appeal to the same shoppers. These labels matter most when something is brand new. A film released this morning has no viewers at all, but it still has a genre and a cast — and that is often enough to guess who might enjoy it.