MAML (Model-Agnostic Meta-Learning)
Most training produces a model that is already good at its job. MAML chases something stranger and more useful: a set of starting weights that is only a few gradient steps away from being good at a job it has never seen.
Two nested loops
Training draws from a distribution of small tasks. In the inner loop, the current weights are copied and updated with a handful of gradient steps on one task's tiny support set. In the outer loop, that adapted copy is scored on fresh data from the same task, and the gradient of that score — a gradient taken through the inner update itself — moves the original weights. The name is literal: nothing in the recipe cares which architecture sits underneath, only that it is differentiable and trained by gradient descent.
The cost and the workarounds
- Second-order gradients — differentiating through the inner steps is memory-hungry, so first-order MAML drops that term and Reptile replaces it with a simple move toward the adapted weights.
- Few-shot benchmarks — the original results were 1-shot and 5-shot classification on Omniglot and miniImageNet, plus fast adaptation in reinforcement learning.
- Tooling — learn2learn and higher supply the differentiable inner loop in PyTorch.
Why it matters here
MAML is the cleanest statement of the idea AutoML keeps reaching for: experience across many datasets should make the next dataset cheap. A tuner that warm-starts from configurations that worked on similar data is doing a coarse, discrete version of what MAML does with gradients.
MAML is a meta-learning algorithm that trains a model's initial weights so that a few gradient steps on a small sample of a new task produce strong performance. It learns across a distribution of tasks, using an inner loop that adapts a copy of the weights and an outer loop that differentiates through that adaptation. It matters because it turns transfer into an explicit training objective rather than a side effect, underpinning few-shot learning and warm-start strategies in AutoML.
Think of two cooks. One has memorised a single recipe perfectly. The other has cooked across many cuisines and can taste an unfamiliar dish, make three adjustments, and get it right. The second cook is not better at one recipe — they are better at picking things up.
MAML trains that second kind of cook. Instead of tuning a model until it aces one task, it tunes the model's starting point so that a handful of examples from a brand-new task is enough to get good at it. Software that has to learn a new customer, a new language, or a new product from a dozen samples is leaning on exactly this idea.