Notes
🔍

Simple Linear Regression

When you want to predict a number from just one measurable input—like house price from square footage—simple linear regression is the “straight line” model people reach for first. It’s small enough to understand fully, but powerful enough to be genuinely useful.

What it is

Simple linear regression models the relationship between one feature x and a continuous target y with a line: y = b0 + b1x. Here, b0 is the intercept (the predicted value when x=0) and b1 is the slope (how much y changes when x increases by 1). The model assumes the “signal” is roughly linear and that the remaining mismatch is captured by residuals (errors).

How it learns (ordinary least squares)

The most common fitting method is ordinary least squares (OLS), which chooses b0 and b1 to minimize the sum of squared residuals. Squaring makes large mistakes count more, and it produces a clean, stable solution. In scikit-learn, this is what you get with:

from sklearn.linear_model import LinearRegression
model = LinearRegression().fit(X, y)  # X has one column for simple linear regression

Why it matters in practice

Simple linear regression is a baseline and a diagnostic tool:

  • Interpretable effect size: the slope tells you the expected change in y per unit of x (useful in pricing, demand, and risk).
  • Sanity check: if a complex model barely beats this line, your features may be weak or your problem may be close to linear.
  • Failure modes are informative: strong curvature, outliers, or changing error spread (heteroscedasticity) show up clearly in residual plots and signal that you need transformations, robust methods, or a richer model.

Simple Linear Regression is a supervised regression model that predicts a continuous target y from a single input feature x using a linear relationship, typically written as y = β₀ + β₁x, with parameters fit to minimize squared prediction error. It matters because it provides a baseline for continuous prediction, supports interpretable effect estimates (slope and intercept), and underpins extensions like multiple linear regression and regularized linear models.

Imagine you’re trying to guess how much a taxi ride will cost based only on how far you travel. If you plot distance and price on a chart, you might notice the dots roughly follow a straight trend. Simple Linear Regression is the AI/ML tool that draws the best “straight line” through those dots so you can predict a number from one input.

It’s called “simple” because it uses just one factor (like distance) to predict one outcome (like price). In supervised learning, it learns from examples where the correct answers are known, then uses that learned relationship to make new predictions.