Notes

Contextual Modeling

There are two easy ways to handle the situation a request arrives in: filter the data before you model, or adjust the ranking afterwards. Contextual modelling takes the harder route and lets the context variables into the model itself, as first-class inputs alongside the user and the item.

Context as an input, not a filter
The predictor becomes a function of user, item and context, trained on all the data at once. In a factorisation machine the context is simply more feature fields — hour of day, device, weather — and the model learns interactions between those fields and the user and item fields, so this user, in the evening, on a television is representable without that exact combination having been seen often. Tensor factorisation is the dedicated form: the user-item matrix becomes a user by item by context tensor with latent factors on all three modes. Deep CTR models such as DeepFM and DIN do it by concatenating context embeddings into the input.

Why it beats the shortcuts

  • No data is discarded. Pre-filtering trains on one slice and starves; modelling shares statistical strength across every context at once.
  • It can represent context changing which of a user's tastes is active, not merely scaling scores up or down as post-filtering does.
  • Combinations never observed still receive a sensible prediction through the factorised parameters, instead of meeting an empty slice.

The cost
Capacity grows with every context field, and so does the chance of fitting noise in a rarely-seen combination. Regularisation and sensible binning — hour of day collapsed to part of day — do real work here. Get it wrong and the model learns randomness dressed up as situational preference.

Contextual modelling treats context — time, location, device, companion — as a direct input to the recommender rather than a filter applied before or after scoring, predicting from user, item and context in one model. Tensor factorisation and factorisation machines are the usual forms. Because it trains on all the data at once it shares strength across contexts instead of starving on a single slice, and can score combinations it never explicitly observed.

Suppose you wanted to allow for the fact that people want different things at different moments. The lazy way is to split your records into piles — a weekday pile, a weekend pile — and look only at the pile that matches. The trouble is that each pile is small, and thin evidence makes poor guesses.

Contextual modelling is the better way. The situation goes into the calculation itself, as just another thing the software knows alongside your viewing history. Nothing is split off and nothing is thrown away. So the system can still make a sensible suggestion for a Tuesday at dawn, even though it has barely ever seen one, by borrowing from everything else it has learned about you.