Random Forest

A single decision tree can feel like a confident “one-shot” guess: it learns clear rules from data, but small changes in the training set can make it swing to a very different set of rules. A Random Forest solves that by building many trees and letting them vote.

How it works

A Random Forest is an ensemble of decision trees trained with two key sources of randomness that make the trees diverse:

• Bootstrap sampling: each tree is trained on a different random sample of the training data drawn with replacement (so some rows repeat and some are left out).
• Random feature selection: at each split, a tree considers only a random subset of features, not all features, which prevents the same “strong” feature from dominating every tree.

For classification, the forest predicts the majority class across trees; for regression, it averages their numeric predictions. This aggregation sharply reduces variance, which is why Random Forests are usually far more stable than a single tree.

What it looks like in practice

Common supervised uses include:

• Spam detection or fraud detection: each tree captures different patterns of suspicious behavior; the forest combines them robustly.
• Credit scoring or churn prediction: handles nonlinear interactions (e.g., tenure × payment history) without manual feature engineering.
• House price prediction: averages many trees to avoid overreacting to quirks in the training data.

In scikit-learn you’ll meet RandomForestClassifier and RandomForestRegressor, with knobs like n_estimators (number of trees) and max_features (how many features each split can consider).

Why it matters

Random Forests are a go-to baseline because they deliver strong accuracy with minimal preprocessing, tolerate mixed feature types, and provide useful diagnostics like out-of-bag (OOB) error and feature importance. Ignoring the “forest” idea and using a single deep tree typically yields a model that fits noise, giving brittle predictions when the data shifts even slightly.