Notes
🔍

Jaccard Index

When you’re judging a classifier, it’s easy to get fooled by “mostly right” predictions—especially when the positive class is rare. The Jaccard Index focuses your attention on how much your predicted positives really overlap with the true positives.

What it measures

The Jaccard Index (also called “intersection over union”) compares two sets: the items you predicted as positive and the items that are truly positive. It’s defined as:

Jaccard = |Predicted Positives ∩ True Positives| / |Predicted Positives ∪ True Positives|

In confusion-matrix terms for binary classification:

Jaccard = TP / (TP + FP + FN)

It ignores TN entirely, which is exactly why it’s useful when negatives are abundant and uninformative.

Why it matters in supervised learning

Accuracy can look great even if you miss most positives (e.g., fraud cases). The Jaccard Index penalizes both kinds of mistakes that harm “overlap”:

  • FP (predicting positive when it’s not) increases the union, lowering the score.
  • FN (missing a true positive) reduces the intersection and increases the union, also lowering the score.

It’s tightly related to the F1 score: F1 = 2J / (1 + J), so optimizing one tends to improve the other, but Jaccard is stricter about overlap.

Practical examples and where you’ll see it

  • Spam detection: how well your “spam” predictions match actual spam without being distracted by the huge “not spam” pile.
  • Churn prediction: overlap between customers flagged as churn risks and those who truly churn.
  • Multi-label classification (e.g., tagging support tickets): compute Jaccard per example between predicted and true label sets.

In scikit-learn, you’ll encounter it as sklearn.metrics.jaccard_score, with options for binary, multiclass, and multilabel averaging.

The Jaccard Index (also called Intersection over Union) measures similarity between two sets as |A ∩ B| / |A ∪ B|, ranging from 0 (no overlap) to 1 (identical). In supervised classification, it evaluates agreement between predicted and true positive label sets, especially in multilabel tasks or segmentation masks. It matters because it directly quantifies overlap quality and penalizes both false positives and false negatives in a single score.

Imagine you and a friend each make a list of movies you like. The Jaccard Index is a simple way to score how similar your lists are: it looks at how many movies you both picked, compared to how many movies appear on either list at all. If your overlap is big, the score is high; if you share only a few, it’s low.

In AI classification, it’s used the same way to compare a model’s “yes” predictions to the real “yes” answers. It’s especially handy when the “yes” cases are rare, like spotting fraud or detecting a disease.