Notes

Item Neighborhood

Ask why a shop suggested a particular paperback and the honest answer is usually "because of the other books you bought". The set of books doing that work is the item neighborhood.

What the neighborhood is
For a candidate item, its neighborhood is the set of catalogue items most similar to it under a stored similarity measure — adjusted cosine on ratings, cosine or conditional probability on implicit counts. The full similarity matrix is never kept: each item retains only its top k most similar items, and everything outside that list is treated as similarity zero. That truncated structure is what gets precomputed offline and loaded into the serving layer.

How a score comes out of it
To score an item for a user, the system intersects the item's neighborhood with the items that user has already touched, then takes a similarity-weighted average of the user's ratings over that intersection. No user-level parameters are involved anywhere, so an interaction from one minute ago influences the next request without any retraining.

Why its size decides everything
The neighborhood is the entire capacity of an item-based recommender, and k is the dial that sets it:

  • Too wide, and weakly related popular items leak in and flatten every list toward the bestseller shelf.
  • Too narrow, and the intersection with a short user history is frequently empty, so no prediction can be produced at all.
  • Sparsity, not taste, usually decides the outcome — where most items carry a handful of interactions, neighborhoods stay unreliable however k is tuned, and a learned latent factor model becomes the better answer.

An item neighborhood is the set of catalogue items most similar to a given item, truncated to its top k neighbours and precomputed offline into an item-item table. Scoring intersects that neighborhood with the items a user has already interacted with and takes a similarity-weighted average. It is the whole model in item-based collaborative filtering: an empty intersection yields no prediction, and an over-wide neighborhood drags every list toward popular items.

Picture a bookshop where every book has a small card pinned beside it, listing the handful of other books most like it. When the shop wants to suggest something, it reads the cards for the books you have already bought and sees which titles keep coming up.

That short list beside each book is its item neighborhood. Keeping it short matters more than you would think. Let it grow long and it fills up with whatever everybody buys, so you end up being offered bestsellers you have already heard of. Keep it tight and the suggestions stay genuinely close to the things you chose yourself.