Polynomial Regression

Some relationships in data are clearly not straight lines: house prices can rise quickly with size at first and then level off, and demand can spike at moderate prices but drop at extremes. Polynomial regression is a simple way to capture that kind of smooth curve without abandoning the familiar tools of linear regression.

What it is (and why it’s still “linear”)

In polynomial regression, you predict a continuous target using polynomial features of the inputs—terms like x, x², x³, or interactions like x₁·x₂. The model is still a linear model in its parameters: it learns weights for each transformed feature and adds them up. For one feature, a degree-3 model looks like:

y ≈ w0 + w1*x + w2*x^2 + w3*x^3

Even though the curve in x is non-linear, estimating w can use ordinary least squares or regularized linear methods.

How it’s used in practice

House price prediction: allow diminishing returns as square footage grows (include x²).
Credit risk: capture a curved relationship between utilization ratio and default probability (in a regression setting).
Demand forecasting: model non-linear price elasticity with higher-degree terms.

In scikit-learn, this is commonly done with PolynomialFeatures plus LinearRegression (or Ridge to stabilize).

Why it matters (and common pitfalls)

The degree controls flexibility: higher degrees fit more complex curves but raise overfitting risk and can behave wildly outside the data range (extrapolation). Polynomial features also increase feature count quickly, making regularization (e.g., Ridge) and validation essential.

Polynomial Regression is a regression model that fits a target variable using a linear model over polynomial features of the inputs (e.g., x, x², x³), enabling curved relationships while remaining linear in parameters. It matters because it extends linear regression to capture nonlinearity with standard least-squares training, but requires careful degree selection or regularization to avoid overfitting and unstable extrapolation.

Imagine drawing a line on a chart to match how something changes, like how a car’s fuel use changes with speed. A straight line is often too simple, because real life curves. Polynomial Regression is a way to let that “line” bend smoothly, so it can follow curved patterns in data.

In supervised learning, you show the model many examples with the right answers (like past speeds and fuel use), and it learns a curve that best matches them. This is useful when the relationship isn’t straight—like predicting house prices that rise quickly in some ranges but level off in others.