Normality of Residuals

When a model makes predictions, it rarely hits the exact true value. The little “misses” it makes—sometimes positive, sometimes negative—carry a lot of information about whether the model is behaving the way we think it is.

What “normality of residuals” means

Residuals are the differences between observed values and model predictions (often written as y − ŷ). Normality of residuals is the assumption that these residuals follow a normal (Gaussian) distribution, typically centered around zero. Intuitively, most errors should be small, large errors should be rare, and the pattern of errors should look roughly symmetric.

Why it matters (and what it doesn’t)

In linear regression, normal residuals are not required for the model to produce a line or even for predictions to be reasonable. The big reason we care is inference: normality supports the usual t-tests, F-tests, confidence intervals, and p-values for coefficients—especially in smaller samples. If residuals are strongly non-normal (heavy tails, skew), those uncertainty estimates can be misleading.

How you check it in practice

• Q–Q plot: points close to a straight line suggest approximate normality.
• Histogram of residuals: a rough bell shape is a good sign.
• Formal tests like Shapiro–Wilk (useful, but can overreact with large datasets).

Concrete example

Suppose you predict house prices. If residuals show a long right tail, it may mean the model systematically underpredicts very expensive homes—often a hint you need a log-transform of price, missing features (e.g., waterfront), or a non-linear model.

Where it shows up in ML workflows

You’ll see this assumption in regression diagnostics in statsmodels, scikit-learn pipelines (paired with residual plots), and classical models used for explainability. In many modern ML models, you may not assume normal residuals—but you still inspect residuals to detect bias, outliers, and mis-specified targets.