Number of Neighbors (k)

Imagine trying to guess something about a new customer by looking at the most similar customers you’ve seen before. In k-nearest neighbors (k-NN), the number of neighbors, written as k, is simply how many “most similar” training examples you consult before making a prediction.

What k controls in k-NN

k-NN predicts by finding the k closest points to a new input in feature space (based on a distance like Euclidean distance). Then it aggregates their labels:

• Classification: take a majority vote among the k neighbors (often with optional distance-weighted voting).
• Regression: take the average (or distance-weighted average) of their target values.

So k is a “smoothing knob”: small k makes predictions very local; large k makes them more global.

Bias–variance intuition (why k matters)

With k = 1, the model can chase noise: it has low bias but high variance (predictions can flip due to tiny changes). As k grows, predictions become more stable (lower variance) but can blur important local structure (higher bias). In spam detection, a tiny k might overreact to quirky emails; a large k might miss a niche spam pattern because it gets “outvoted” by common ham emails.

How k is chosen in practice

k is a hyperparameter selected using validation, commonly cross-validation. In scikit-learn you’ll see it as n_neighbors in KNeighborsClassifier / KNeighborsRegressor. Choosing k interacts with scaling: without feature scaling, “closest” can be dominated by one large-scale feature, making any k a poor choice.