Stratified Sampling

If your dataset has a “rare but important” class—fraud cases, churners, positive diagnoses—randomly splitting the data can accidentally create a test set that barely contains any of them. Stratified sampling is the simple idea of splitting while keeping key proportions steady.

What it is and how it works

Stratified sampling means you first divide data into strata (groups) based on a variable you care about—most commonly the class label in classification—then you sample from each stratum so each split (train/validation/test) has roughly the same distribution as the full dataset. Mechanically, it’s like dealing cards from separate piles: you ensure each hand gets a fair share from every pile, instead of hoping a shuffled deck works out.

Why it matters in supervised learning

Model evaluation depends on the test set being representative. Without stratification, you can get:

Misleading metrics: accuracy looks great if the minority class barely appears; precision/recall become unstable.
Bad hyperparameter choices: validation results bounce around because the class mix changes between splits.
Training issues: the training set may underrepresent the minority class, making the model learn the wrong decision boundary.

Practical examples and common tooling

In spam detection (spam = 5%), stratification keeps ~5% spam in each split, so recall and precision reflect reality. In credit default prediction (default = 2%), it prevents a test set with too few defaults to judge risk models. In scikit-learn, you’ll see this in StratifiedKFold and by passing stratify=y to train_test_split:

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y
)

Stratified sampling is a data-splitting method that partitions a labeled dataset so each split preserves the original distribution of a chosen variable, typically the target class (or binned target for regression). It matters because it yields train/validation/test sets with comparable label proportions, reducing evaluation bias and improving reliability under class imbalance; for example, keeping a 95/5 class ratio consistent across splits.

Imagine you’re making a party playlist and you want it to represent everyone’s tastes. You wouldn’t pick 50 songs all from one person’s favorites. Instead, you’d take some songs from each group—rock fans, pop fans, jazz fans—so the sample feels balanced.

Stratified sampling does the same thing with data. When you split data into training and test sets, it makes sure each set keeps roughly the same mix of important categories (like “spam” vs “not spam,” or “disease” vs “healthy”). This matters because it helps you judge the model fairly, especially when one group is rare.