Notes
🔍

Hard Voting

When you ask several people to label an email as “spam” or “not spam,” you might just go with the majority opinion. Hard voting does the same thing for a group of classification models: it picks the class that gets the most votes.

What hard voting is doing

In a hard-voting ensemble, each base classifier outputs a discrete class label (not a probability). The ensemble collects these labels and predicts the majority class. With three models predicting {spam, spam, not spam}, the final prediction is spam. You can also use weighted voting, where stronger models count more (e.g., a model with better validation accuracy gets weight 2 while others get weight 1). If there’s a tie, libraries use a fixed rule (like choosing the smallest class label) unless you design your own tie-break.

Where it shows up in practice

  • Fraud detection: combine a linear model, a tree model, and an SVM; the final “fraud/not fraud” is the majority decision.
  • Disease diagnosis: mix models trained on labs, symptoms, and imaging features; hard voting gives a single diagnosis label.
  • Customer churn: if two out of three models flag a customer as likely to churn, the ensemble flags them too.

In scikit-learn, this is exactly what VotingClassifier(voting="hard") implements.

Why it matters (and what can go wrong)

Hard voting is simple, robust, and works well when models make different errors (their mistakes aren’t perfectly correlated). But it throws away confidence information: a 51% vs. 99% probability both become the same label. If your application needs calibrated risk scores (credit scoring, medical triage), soft voting (averaging probabilities) is usually a better fit.

Hard Voting is an ensemble classification rule that predicts the class receiving the most votes from a set of base classifiers, using each model’s final class label rather than its probability scores (optionally with model-specific vote weights). It matters because it provides a simple, robust way to reduce individual model errors and improve classification stability without requiring calibrated probabilities or an additional meta-learner.

Imagine you’re picking a restaurant with friends. Everyone votes for their top choice, and the place with the most votes wins. Hard voting in machine learning is the same idea: you ask several different AI models to make a yes/no (or category) decision, and the final answer is whichever class gets the most votes.

This is used in supervised learning for tasks like “spam or not spam,” “fraud or not fraud,” or “which disease is most likely.” It matters because a group decision is often more reliable than trusting a single model that might occasionally be wrong.