Notes
🔍

Law of Total Probability

When you’re unsure about something, a natural move is to break the situation into a few clear cases, figure out the chance in each case, and then “average” them back together. The Law of Total Probability is the formal rule that makes that idea precise.

What it says
Suppose the world can fall into one of several non-overlapping cases (events) B1, B2, …, Bk that cover all possibilities (a partition of the sample space). Then for any event A: 

P(A) = Σ_i P(A | B_i) P(B_i)

You’re weighting each conditional probability P(A | Bi) by how common that case is, P(Bi). This is why it feels like a probability “average,” but with weights.

A practical example
Imagine predicting whether a customer will buy (A). Customers come from either mobile (B1) or desktop (B2):

  • P(Buy | Mobile) = 0.03, P(Mobile) = 0.70
  • P(Buy | Desktop) = 0.06, P(Desktop) = 0.30

Then P(Buy) = 0.03·0.70 + 0.06·0.30 = 0.039. Even if desktop users buy more often, mobile dominates because it’s most traffic.

Why it matters in AI/ML
This rule is everywhere you marginalize over hidden or intermediate variables:

  • Naive Bayes and other probabilistic classifiers use it to get P(y) or P(x) by summing over classes.
  • Mixture models (e.g., Gaussian mixtures) compute overall likelihoods by summing component likelihoods weighted by mixing probabilities.
  • In Bayesian networks, it’s the workhorse for turning conditional pieces into unconditional probabilities.

Law of Total Probability states that if events B1, …, Bk form a partition of the sample space (mutually exclusive and exhaustive), then for any event A: P(A) = Σi P(A | Bi)P(Bi). It is fundamental for marginalizing over latent variables in AI/ML. Example: compute P(y) in a mixture model by summing P(y | z=i)P(z=i) over components.

Imagine you want to know the chance it will be late to work. That depends on what kind of day it is: sunny, rainy, or snowy. Each day type has its own “late” chance, and each day type also has its own chance of happening. The Law of Total Probability says you can find the overall chance of being late by adding up: (chance of each day type) × (chance of being late on that day type).

In statistics and AI/ML, this is used to combine probabilities across different possible situations or hidden “cases” (like different user groups or model states) to get one final probability.