Notes
🔍

AmoebaNet

Evolution seems an odd tool for designing a neural network, yet one of the strongest image classifiers of its era was bred rather than drawn. AmoebaNet is the family of architectures that came out of that experiment.

How it was found

AmoebaNet was produced by regularized evolution (Real et al., 2019), an evolutionary search over the same cell-based space as NASNet: the algorithm designs a small repeatable cell, and the network is built by stacking copies of it. Search keeps a population of architectures, samples a few of them at random, mutates the best of that sample — rewiring a connection, swapping an operation — trains the child briefly, and returns it to the population. The regularizing twist is what gets removed: not the worst model, but the oldest. That aging rule keeps the population turning over and favours architectures that stay good when retrained, rather than ones that won a single lucky evaluation.

Why it mattered

  • Under an identical search space and compute budget, evolution matched and then beat the reinforcement-learning search behind NASNet — evidence that the elaborate controller was not what made NAS work.
  • The largest AmoebaNet models reached roughly 84% top-1 accuracy on ImageNet, state of the art at the time, and became standard baselines in later NAS papers.
  • The cost ran to thousands of GPU-days, which is precisely what motivated the weight-sharing and one-shot methods that followed.

What to take from it

AmoebaNet matters less as a model you would deploy today than as the result that made evolutionary architecture search respectable — and that showed how much of NAS's early success lived in the search space rather than the search algorithm.

AmoebaNet is a family of convolutional image-classification architectures discovered by evolutionary neural architecture search. It came from regularized evolution — tournament selection with mutation, where the oldest population member is discarded rather than the worst — applied to the cell-based NASNet search space. AmoebaNet matters as the first clear demonstration that simple evolution matches or beats reinforcement-learning search at equal cost, and its models remain common NAS baselines.

Think about how farmers improve a crop. They plant many slightly different varieties, keep the ones that grow best, let those produce the next generation, and repeat until the harvest is far better than what they started with.

AmoebaNet is an AI image recogniser that was created exactly that way. Researchers did not design it by hand. They started with many rough network designs, made small random changes to the better ones, kept whichever did best at recognising pictures, and repeated this thousands of times over. The design that survived that long contest could identify objects in photographs about as accurately as the best human-designed systems of its day — which is why it is remembered.