Ordinal Regression
Some prediction problems don’t fit neatly into “classification” or “regression.” The target is a category (like a class label), but the categories have a natural order (like ratings), and that order carries real information.
What ordinal regression is
Ordinal regression models a target variable made of ordered categories, such as {low, medium, high} or {1, 2, 3, 4, 5}. It sits between classification and regression: unlike standard multiclass classification, it treats “1 vs 2” as closer than “1 vs 5”; unlike regression, it doesn’t assume the gaps between categories are numerically meaningful or equal. A common approach is the cumulative (proportional odds) model: the model learns a single scoring function plus a set of learned thresholds that split the score line into ordered bins, producing probabilities like P(y ≤ k).
Why the ordering matters
Ignoring order wastes structure and can hurt both accuracy and calibration. Ordinal-aware models:
- penalize mistakes by “distance” (predicting 4 instead of 5 is less wrong than 1 instead of 5),
- use fewer parameters than one-vs-rest multiclass setups,
- produce more sensible probability curves across adjacent categories.
Practical examples and tools
Typical targets include credit risk bands (A–E), customer satisfaction (1–5 stars), disease severity stages, or churn risk buckets (low/med/high). In Python, you’ll see ordinal models in statsmodels (OrderedModel) and in scikit-learn-compatible packages like mord (ordinal logistic regression). You’ll also see ordinal-friendly metrics such as mean absolute error on encoded ranks or quadratic weighted kappa, which reflect how far off an ordered prediction is.
Ordinal regression is a supervised learning task where the target is an ordered categorical label (ranked classes) rather than an unordered class or a continuous number, e.g., {low, medium, high} or 1–5 ratings. Models learn to respect this order when predicting. It matters because treating ordinal targets as standard classification discards ordering information, while treating them as numeric regression imposes arbitrary distances, both harming accuracy and calibration.
Think of rating something with stars: 1 star is worse than 2, and 5 is best. Those ratings are categories, but they also have a clear order. Ordinal regression is the kind of AI prediction used for targets like that—where the answer comes from a set of ordered choices.
It sits between “pick a label” and “predict a number.” For example, a model might predict a patient’s condition as mild, moderate, or severe; a customer’s satisfaction as low, medium, or high; or a loan risk as A, B, C, D. The key is that being off by one step is less wrong than being off by three.