Notes
🔍

Generalized Linear Models (GLM)

Generalized Linear Models (GLM) are a way to do “regression” when the thing you’re predicting isn’t a nice, bell-shaped number. They let you model outcomes like yes/no decisions, counts, or positive skewed measurements using the same core idea: explain a target using a weighted combination of features.

What a GLM is
A GLM extends ordinary linear regression by separating two pieces:

  • A random component: the target variable follows a distribution from the exponential family (common choices: Gaussian, Bernoulli, Poisson, Gamma).
  • A systematic component: a linear predictor η = Xβ (features times coefficients).
  • A link function that connects the mean of the target to the linear predictor: g(E[y|X]) = η.

This link is the “adapter” that makes a linear model work for non-linear-looking outcomes while keeping estimation and interpretation manageable.

Concrete examples
GLMs show up whenever the target’s scale or constraints matter:

  • Logistic regression (Bernoulli + logit link): predict whether an email is spam (probability between 0 and 1).
  • Poisson regression (Poisson + log link): predict number of customer support tickets per day (nonnegative counts).
  • Gamma regression (Gamma + log link): predict medical costs (positive and skewed).

Why GLMs matter in AI/ML
Many ML tasks are classification or count prediction, and GLMs provide strong baselines that are fast, interpretable, and statistically grounded. They’re trained by maximum likelihood, often via iteratively reweighted least squares (IRLS) or gradient methods, and they support regularization (e.g., L1/L2) in common libraries. When you ignore the right distribution/link, you can get invalid predictions (like negative counts) and miscalibrated uncertainty.

Generalized Linear Models (GLM) extend linear regression to handle non-Gaussian outcomes by combining (1) a linear predictor of features, (2) an outcome distribution from the exponential family (e.g., Bernoulli, Poisson), and (3) a link function that maps the mean to the linear predictor. GLMs are important in AI/ML for interpretable, statistically principled prediction and inference. Example: logistic regression (a GLM) models click/no-click probabilities.

Imagine you’re trying to predict different kinds of outcomes: how much someone will spend (a number), whether an email is spam (yes/no), or how many customers arrive in an hour (a count). A plain “straight-line” model works well for some of these, but not all.

Generalized Linear Models (GLM) are a flexible family of prediction models that let you keep the idea of “inputs influence an outcome,” while choosing a shape that fits the type of outcome you have. In statistics and machine learning, GLMs power common tools like logistic regression (for yes/no) and Poisson regression (for counts).