Notes
🔍

SASRec

Your last five interactions say more about what you want next than your average behaviour over two years. SASRec is the model that made self-attention the standard way of reading that recent order.

What it is
SASRec — Self-Attentive Sequential Recommendation, Kang and McAuley, 2018 — treats a user's interaction history as a sequence and predicts the next item with a causal Transformer decoder. Items become embeddings, learned position embeddings mark order, and stacked self-attention blocks let each step attend over earlier steps with learned weights. Training is next-item prediction at every position at once: the sequence shifted by one supplies the targets, and a mask stops any position seeing the future. Scoring is an inner product between the final hidden state and the item embeddings, so retrieval stays a nearest-neighbour lookup.

Why it displaced the RNN

  • Attention reaches a relevant item from twenty steps back directly, where GRU4Rec has to carry it through twenty recurrent updates.
  • All positions train in parallel, which makes a sequence model practical on MovieLens- and Amazon-scale logs.
  • The attention weights are inspectable, so you can see which past items drove a recommendation.

Where it sits
It is the strict left-to-right counterpart of BERT4Rec, which instead trains by masking items anywhere in the sequence. SASRec remains the baseline every sequential-recommendation paper reports against, and a hard one to beat once it is properly tuned and evaluated over the full catalogue rather than against sampled negatives.

SASRec (Self-Attentive Sequential Recommendation) predicts a user's next item from their interaction sequence with a causal Transformer decoder: item and position embeddings, masked self-attention over earlier steps, and an inner-product score against item embeddings. It matters because attention reaches long-range history directly and trains every position in parallel, unlike recurrent models such as GRU4Rec. It is the standard sequential-recommendation baseline and the left-to-right counterpart of BERT4Rec.

What you watched in the last ten minutes tells a recommender more than everything you watched last year. A cooking video followed by two knife reviews is a story, and the order of it matters.

SASRec is a well-known model built to read that story. It looks back over your recent activity and learns which earlier moments are worth paying attention to — probably the knife reviews, not the film you half-watched on a plane — then uses those to guess what comes next. It borrowed the same attention machinery that powers modern language models, which is why it copes with long histories better than older approaches did.