Notes
🔍

Majority Voting

When you ask a few people for an opinion, the simplest way to decide is to go with what most of them say. Majority voting is that same idea applied to machine learning predictions.

What it is in supervised learning

In classification, majority voting means predicting the class label that appears most frequently among a set of candidate labels. The most common place you’ll see it is in k-nearest neighbors (k-NN): to classify a new example, k-NN finds the k closest training points (its “neighbors”) and then assigns the class that the majority of those neighbors belong to. If 7 neighbors are considered and 4 are “spam” while 3 are “not spam,” the prediction is “spam.”

How it behaves (and why k matters)

Majority voting is simple, but its behavior depends heavily on which votes you include:

  • Small k makes predictions sensitive to noise (one mislabeled neighbor can flip the vote).
  • Large k smooths noise but can wash out local patterns and favor the most common class.
  • Ties can happen (especially with even k); implementations break ties by distance, class order, or random choice.

Practical examples and common variations

In disease diagnosis, a patient might be labeled based on the majority label among similar historical patients; in fraud detection, a transaction can inherit the majority class of nearby transactions in feature space. A frequent upgrade is distance-weighted voting, where closer neighbors count more than farther ones. In scikit-learn, this is controlled in KNeighborsClassifier via weights='uniform' (plain majority vote) vs weights='distance'.

Majority Voting is a decision rule for classification that predicts the class receiving the most votes from a set of label proposals, such as the labels of the k nearest neighbors in k-NN or the outputs of multiple models in an ensemble. It matters because it converts multiple local or model-level signals into a single discrete prediction, directly determining classification accuracy and robustness to individual noisy votes.

Imagine you’re trying to guess what a mystery fruit is, and you ask 5 friends who have seen it before. If 3 say “apple” and 2 say “pear,” you go with “apple.” That’s Majority Voting: you pick the answer that gets the most votes.

In supervised learning, it’s a simple way to make a decision when you have multiple “opinions.” A common example is k-nearest neighbors: the model looks at the closest past examples and lets them vote on the label. If most nearby emails were marked “spam,” the new one is labeled spam too. It’s popular because it’s intuitive and often reliable.