Notes
🔍

Axioms of Probability

Probability can feel like “common sense with numbers,” but to make it reliable (and not just vibes), it needs a few ground rules. The axioms of probability are those rules: simple statements that everything else in probability is built from.

The three axioms (Kolmogorov)

We start with a set of possible outcomes (the “sample space”) and assign a probability to events (sets of outcomes). The axioms are:

  • Non-negativity: For any event A, P(A) ≥ 0. Probabilities can’t be negative.
  • Normalization: P(Ω) = 1, where Ω is the event “something in the sample space happens.” Total probability mass is 1.
  • Additivity (countable): If events A and B are mutually exclusive (can’t both happen), then P(A ∪ B) = P(A) + P(B). This extends to any countable list of disjoint events.

What these axioms give you

Many familiar “rules” are consequences, not extra assumptions:

  • P(∅) = 0 (the impossible event has probability 0).
  • Complement rule: P(Aᶜ) = 1 − P(A).
  • Inclusion–exclusion: P(A ∪ B) = P(A) + P(B) − P(A ∩ B).

Practical example

Suppose a model predicts whether a loan will default. If it assigns P(default)=0.2, then by the complement rule P(no default)=0.8. If “default” and “no default” weren’t mutually exclusive or didn’t sum to 1, your predictions wouldn’t behave like probabilities and would break downstream calculations.

Why it matters in AI/ML

Probabilistic ML—Naive Bayes, Bayesian networks, Hidden Markov Models, and modern probabilistic programming—relies on these axioms so that likelihoods, posteriors, and uncertainty estimates are coherent. Libraries like PyTorch distributions, TensorFlow Probability, and PyMC assume your “probabilities” obey them; otherwise training objectives and inference can become inconsistent.

Axioms of Probability are the foundational rules (Kolmogorov axioms) that define a valid probability measure: probabilities are nonnegative, the probability of the entire sample space is 1, and the probability of disjoint events adds (countable additivity). They matter in AI/ML because they ensure uncertainty calculations are consistent, enabling principled inference and learning. Example: for mutually exclusive classes, P(class A or B) = P(A) + P(B).

Think of probability like a game with clear house rules, so everyone agrees what “chance” means. The axioms of probability are those basic rules. They say: (1) every probability is between 0 and 1 (0 means impossible, 1 means certain), (2) something in the set of possible outcomes must happen, so the total probability is 1, and (3) if two events can’t happen at the same time (like flipping heads and tails on one coin), the chance of “either one” is the sum of their chances. AI systems rely on these rules to handle uncertainty consistently.