Sampling Distribution
When you take a sample and compute a number—like an average house price or a model’s accuracy—that number feels like “the” answer. But if you took a different sample, you’d get a slightly different answer. A sampling distribution is the way that answer would vary across many repeated samples.
What it is
The sampling distribution is the probability distribution of a statistic (such as the sample mean, median, proportion, regression coefficient, or AUC) computed from all possible samples of a fixed size drawn from the same population (or data-generating process). It’s not the distribution of your raw data; it’s the distribution of the summary you compute from the data.
Why it looks the way it does
Two key ideas shape sampling distributions:
- Center: many statistics are centered near the true population value (often approximately unbiased).
- Spread: larger samples usually give a tighter sampling distribution; this spread is captured by the standard error.
For the sample mean, the Central Limit Theorem explains why the sampling distribution often becomes approximately normal as sample size grows, even if the underlying data aren’t perfectly normal.
Concrete examples
- House prices: take 100 random recent sales and compute the mean price. Repeat this many times; those means form the sampling distribution, telling you how noisy your estimate is.
- Medical testing: estimate the fraction of positive tests from a sample of patients. The sampling distribution of that proportion drives confidence intervals.
Why it matters in AI/ML
Sampling distributions are the backbone of uncertainty in ML: confidence intervals for coefficients in linear/logistic regression, hypothesis tests, and the variability of metrics under resampling. Methods like bootstrap and cross-validation approximate sampling distributions when analytic formulas are hard, helping you avoid over-trusting a single train/test split.
Sampling Distribution is the probability distribution of a statistic (e.g., mean, variance, accuracy) computed over many hypothetical samples drawn from the same population. It quantifies how much the statistic varies due to random sampling, enabling standard errors, confidence intervals, and hypothesis tests. In ML, it underpins uncertainty estimates for metrics and parameters; for example, the sampling distribution of test accuracy across bootstrap resamples supports a confidence interval for model performance.
Imagine you want to know the “average height” of everyone in a city, but you can only measure 50 people at a time. If you repeat that many times—each time picking a new group of 50—you’ll get slightly different averages. The collection of all those possible averages forms a sampling distribution.
In statistics and machine learning, a sampling distribution describes how a number you compute from a sample (like an average, accuracy score, or error rate) would vary if you took many different samples from the same population. It helps us judge how “noisy” our estimate is and how confident we should be in it.