k-Nearest Items
An item-item similarity table for a million items holds a trillion entries, and almost all of them are near zero and useless. k-nearest items is the decision to keep only the part that carries information.
The truncation
Once similarities are computed, each item retains only its k most similar items — typically 20 to 200 — and every other pair is treated as exactly zero. Predicting a user's score for a candidate uses nothing but that list: intersect it with the items the user has interacted with, then take the similarity-weighted average of the user's ratings over the intersection. Anything outside the top k contributes nothing, by construction.
Why truncate at all
Two reasons, both load-bearing. Cost: a sparse top-k table fits in memory and makes scoring a few lookups, which is what keeps item-based retrieval inside a serving budget. Accuracy: low similarities are dominated by sampling noise from items with very few co-raters, so keeping them actively degrades predictions rather than merely diluting them.
Choosing k
k is tuned on a validation split, and the curve has a recognisable shape:
- Small k gives precise but fragile neighbourhoods, and the intersection with a short history is often empty, so coverage falls.
- Large k readmits weak, popular neighbours and pulls every user's list toward the catalogue average.
- On sparse implicit data the optimum sits lower than ratings data would suggest, because each similarity rests on fewer shared users.
k-nearest items is the truncation that makes item-based collaborative filtering servable: each item keeps only its k most similar items, and all remaining similarities are treated as zero. Scoring intersects that list with a user's history and takes a similarity-weighted average. Truncation cuts storage and latency and also improves accuracy, since weak similarities are mostly noise. Too small a k empties the intersection; too large a k readmits popular neighbours and flattens the list.
Comparing every product in a huge catalogue with every other product leaves you with an unusable mountain of numbers, and nearly all of them just say "these two have nothing to do with each other".
k-nearest items throws that mountain away. Each product keeps a short list — perhaps fifty — of the things most like it, and everything else is ignored. This saves an enormous amount of work, and it actually improves the suggestions, because faint resemblances are mostly accidents. The list has to be the right length: too short and there is nothing to work with, too long and the bestsellers creep back in.