Notes
🔍

Log-Probabilities

Probabilities can get tiny fast. In models like Naive Bayes, you might multiply hundreds or thousands of small numbers together—so small that your computer rounds them down to zero. Log-probabilities are the simple trick that keeps those calculations stable and workable.

What log-probabilities are

A log-probability is just the logarithm of a probability (usually the natural log). Because logs turn multiplication into addition, Naive Bayes typically works with log values. Instead of computing:

P(y) * Π_i P(x_i | y)

it computes:

log P(y) + Σ_i log P(x_i | y)

This gives the same ranking of classes (log is monotonic) but avoids numerical underflow and is faster and cleaner to implement.

How it shows up in Naive Bayes

In a spam filter, each word contributes evidence for “spam” or “not spam.” Naive Bayes combines that evidence by multiplying conditional probabilities across words; in log space it becomes a sum of per-word log-likelihoods plus a log prior. The model then picks the class with the largest log posterior (often called a log score).

  • Log prior: log P(y) (how common each class is)
  • Log likelihood: Σ_i log P(x_i | y) (feature evidence)
  • Log posterior (up to a constant): the quantity you compare across classes
Why it matters in practice

If you ignore log-probabilities, long documents or high-dimensional data can produce products that underflow to zero, making every class look equally impossible. Libraries bake this in: scikit-learn’s Naive Bayes exposes class_log_prior_ and feature_log_prob_, and prediction effectively compares these summed log terms rather than raw probability products.

Log-probabilities are probabilities expressed in logarithmic form (e.g., log p) rather than as raw values in [0,1]. In Naive Bayes, class scores are computed by summing log-likelihoods and a log prior (log P(y)+Σilog P(xi|y)) instead of multiplying many small probabilities. This matters because it prevents numerical underflow and turns products into stable, efficient additions while preserving the same class ranking.

Think of multiplying lots of tiny numbers, like the chances of several independent events all happening. The result can get so small that it’s hard to work with. Log-probabilities are a simple trick: instead of using the probability directly, you use its “log” version (a different scale) so extremely small numbers become manageable.

In AI systems like Naive Bayes spam filters, the model combines many clues—words, sender, links—each with its own probability. Using log-probabilities turns “multiply many probabilities” into “add many numbers,” which is more stable and reliable, while keeping the same final ranking of which label is most likely.