Complement Naive Bayes
When you’re classifying text—spam vs. not spam, or news topics—plain Naive Bayes can feel surprisingly strong for such a simple model. But it has a known weakness: it can get biased toward classes that have lots of training examples or lots of words, which shows up as uneven accuracy across classes.
What it is and the core idea
Complement Naive Bayes (CNB) is a tweak to the usual multinomial Naive Bayes designed mainly for imbalanced text classification. Instead of estimating word probabilities from documents inside a class, CNB estimates them from the complement of that class (all documents not in the class). For each class c, it builds statistics from “everything except c,” then uses those to form weights that penalize words that are common outside the class. Intuitively: to recognize “sports,” it learns what “not sports” looks like and down-weights those words.
How prediction works (intuition)
CNB still uses the Naive Bayes independence assumption and a bag-of-words representation. At prediction time, a document is scored by summing per-word weights for each class; the class with the best score wins. Key practical details:
- Complement counts reduce the dominance of frequent words from majority classes.
- Smoothing (like additive/Laplace smoothing) prevents zero probabilities for rare words.
- It pairs naturally with TF-IDF, where CNB often beats standard MultinomialNB.
Where you’ll see it and why it matters
CNB is common in spam detection, sentiment classification, and topic labeling when classes are skewed (e.g., “refund request” is rare). In scikit-learn you’ll encounter it as sklearn.naive_bayes.ComplementNB, a fast baseline that can deliver noticeably better minority-class performance than MultinomialNB without giving up Naive Bayes simplicity.
Complement Naive Bayes is a Naive Bayes text classifier that estimates each class using statistics from the complement (all other classes) rather than the class itself, typically with multinomial word-count features and smoothing. This reweighting reduces bias toward frequent classes and improves robustness on imbalanced datasets. It matters because it often yields higher accuracy and more stable decision boundaries than standard Multinomial Naive Bayes in document classification.
Imagine you’re sorting emails into folders. One trick is to learn what “spam” looks like. Another trick is to learn what “not spam” looks like, and use that as a strong clue too. Complement Naive Bayes is a simple text-classifying method that leans on this second idea: it learns each category by looking at the words that show up in all the other categories (the “complement”).
This often helps when some categories have far fewer examples than others, like rare types of customer complaints. It’s commonly used for tasks like spam filtering, news topic labeling, and sentiment detection.