Notes
🔍

Cell-Based Search

Searching for a neural network layer by layer is a losing game: the space of possible wirings explodes long before the search finds anything good. Cell-based search cuts the problem down by changing what is being searched for — not a whole network, but one small repeatable block.

How the space is factored

The search designs one or two cells, each a tiny directed graph of a handful of operations — separable convolutions, pooling, identity skips — over a few intermediate nodes. A fixed macro-skeleton then stacks copies of that cell into the finished network. NASNet introduced the standard pair: a normal cell that preserves spatial resolution and a reduction cell that halves it, alternated to whatever depth you want.

What it buys

  • A space small enough to search — tens of decisions instead of hundreds, which is why DARTS, ENAS and AmoebaNet all adopt it.
  • Transferability — a cell found on CIFAR-10 can be restacked deeper and wider for ImageNet, so the expensive search runs on the cheap dataset.
  • A usable depth dial — the number of stacked cells becomes a deployment knob, trading accuracy against latency with no new search.

The cost of the shortcut

The restriction is a strong human prior. Repeating one block rules out architectures whose stages genuinely differ, and the hand-built skeleton — stem, strides, channel doubling — does real work the search never gets credit for. Later methods search the macro-structure as well, accepting a larger space to win back what cell-based search traded away.

Cell-based search restricts neural architecture search to designing a small repeatable block — a cell of a few operations over a handful of nodes — which a fixed outer skeleton then stacks into the full network. It shrinks the search space from hundreds of choices to tens and makes results transferable: a cell found on a small dataset is restacked deeper for a large one. Nearly every practical NAS method, DARTS and ENAS included, relies on it.

Think of building with Lego. Instead of designing a whole castle brick by brick, you design one really good module — a wall section with a window in it — and then repeat that module to whatever height you need.

Cell-based search does the same thing for neural networks. The computer hunts for one small, clever block rather than an entire network, and a simple rule stacks copies of that block to make the finished model. This keeps the hunt quick enough to be practical, and the block discovered on a small, cheap problem can be reused, stacked higher, on a much bigger one.