Notes
🔍

Underfitting

Underfitting is what happens when a model is too “simple-minded” for the pattern you’re asking it to learn. It’s like trying to summarize a complicated story with a single sentence: you’ll miss important details, even if you say that sentence perfectly.

What underfitting means

A model is underfitting when it cannot capture the real structure in the data, leading to poor performance not only on new data, but even on the data it was trained on. In statistical terms, the model has high bias: its assumptions are too restrictive, so its predictions are systematically off.

You’ll often see underfitting when:

  • The model is too constrained (e.g., a straight line for a curved relationship).
  • Important features are missing or overly simplified.
  • Regularization is too strong (e.g., very large λ in ridge/lasso).
  • Training is cut short (e.g., too few epochs in a neural network).

How it looks in practice

Imagine predicting house prices using only square footage with a linear model, while the true prices also depend heavily on neighborhood, condition, and school quality. The model may consistently undervalue homes in premium areas and overvalue others—errors that don’t go away with more data because the model can’t represent the needed relationships.

A common diagnostic clue is:

  • Training error is high and validation/test error is also high (and often similar).

Why it matters in AI/ML

Underfitting blocks learning: you can’t generalize well if you never learned the signal in the first place. It shows up in model selection (choosing too-simple models), feature engineering (not providing informative inputs), and tuning (over-regularizing). Tools like cross-validation in scikit-learn and learning curves help spot it by comparing performance as model complexity or training time increases.

Underfitting occurs when a model is too simple to capture the true structure in the data, leading to high bias and poor performance on both training and test sets. It matters in AI/ML because it limits predictive accuracy and can mask meaningful relationships, even with abundant data. Example: fitting a straight line to a clearly nonlinear relationship (e.g., quadratic trend) yields systematically wrong predictions across the range.

Imagine trying to describe a whole movie using just one sentence: you’ll miss most of what matters. That’s what underfitting is like in machine learning.

Underfitting happens when a model is too simple to capture the real patterns in the data. It doesn’t learn enough, so it performs poorly not only on new, unseen examples, but even on the training data it was given. For instance, using a straight line to predict house prices when prices depend on many factors (size, location, condition) can lead to consistently wrong guesses. The model is “not flexible enough” to fit the problem.