Few-Shot Meta-Learning
Show a child one photograph of a giraffe and they will spot the next one without hesitation. A conventional classifier needs thousands of labelled examples to get there. Few-shot meta-learning closes that gap by training a model across many tasks, so that a new task can be learned from a handful of examples.
Learning across tasks
Training is organised into episodes drawn from a task distribution. Each episode supplies a tiny support set — five classes with one example each, in the 5-way 1-shot setting — and a query set used to score the adaptation. Updates go to parameters shared across all tasks, so the model is rewarded for being easy to adapt rather than for what it already knows. What gets meta-learned depends on the method:
- An initialisation. MAML and Reptile seek weights that sit a few gradient steps from good performance on any task in the distribution.
- A metric space. Prototypical networks embed examples so a class becomes the mean of its few samples, reducing classification to a nearest-neighbour test.
- An adaptation rule. Recurrent or attention-based meta-learners read the support set and predict directly, with no inner loop at all.
Why AutoML cares
Warm starting is few-shot learning in disguise. When Auto-sklearn computes meta-features of your dataset and begins its search from configurations that won on similar ones, a new dataset is a new task with almost no evaluations to spare. The same framing covers adapting a speech model to one voice from a minute of audio, or screening a drug target from a few dozen assay results. Omniglot and miniImageNet are the standard benchmarks; learn2learn and Torchmeta are the usual libraries.
Few-shot meta-learning trains a model across many related tasks so that an unseen task can be learned from only a handful of labelled examples. Rather than fitting one dataset, it optimises a shared initialisation, embedding space or adaptation rule over a task distribution, scored on held-out queries. It matters in AutoML because warm-starting a search from meta-features of similar past datasets is exactly this problem: a new dataset affords very few evaluations.
Think about how you learn card games. The first one took a while. The tenth took five minutes, because you already knew what rules, turns and scoring feel like.
Few-shot meta-learning gives software that same advantage. Normally a computer needs thousands of examples to learn anything — thousands of photos of a cat before it can reliably spot a cat. Here, the system is first shown a huge variety of small learning problems, so it picks up a general knack for picking things up. After that, three or four examples of something new are enough. It is the reason your phone can learn your face from a few pictures rather than a few thousand.