Leave-One-Out Evaluation

With most cells of the interaction matrix empty, a conventional eighty-twenty split throws away data you cannot spare. Leave-one-out keeps nearly all of it: hold back exactly one interaction per user and train on everything else.

How the split is built
For each user, one interaction goes to the test set and the rest stay in training. Picking that item at random is the convenient choice and the wrong one — the user's later behaviour then sits in training, and the model is scored on predicting a past event from a future it has already seen. Holding out each user's most recent interaction keeps the arrow of time pointing the right way, and that is the version worth running.

The sampled-candidate shortcut
Scoring the held-out item against the whole catalogue is expensive, so the widespread protocol ranks it against 99 or 100 randomly drawn items the user never touched and reports HR@10 and NDCG@10 over that short list. The distortion is well documented and reproduced: with few negatives the metrics compress and model orderings can invert relative to full-catalogue evaluation, so numbers from studies using different sample sizes are not comparable.

What to take from it

• One test interaction per user makes the metric noisy — it averages over users, not over events, so confidence intervals earn their place.
• It measures next-item accuracy only, and says nothing about diversity, novelty, or whether the list survives a real feed.
• Reported beside a chronological split over the full catalogue it is informative; reported alone with sampled negatives it is barely a number.