Neural Predictor (NAS)
Every architecture search spends most of its budget on one expensive question: how good would this network be if I trained it? A neural predictor answers with a second, much smaller model — one trained to look at an architecture and estimate its accuracy, so the search can skim thousands of candidates for the price of a few.
How it learns to judge a network
The architecture has to become data first. A cell is encoded as a graph — an adjacency matrix saying which nodes feed which, plus a label per node naming its operation — and that graph is fed to a graph neural network. The predictor is itself a small neural network, which is where the name comes from. Its training set is tiny and costly: a few hundred architectures that really were trained and scored. From those pairs it learns a mapping from structure to performance, and the search queries it instead of a GPU.
Ranking matters more than accuracy
A predictor wrong by three points everywhere is still useful, because search only needs to know which candidate is better. Papers therefore report Kendall tau or Spearman correlation against true accuracy rather than squared error. Real systems iterate: predict, train the top handful properly, add those results to the training set, refit — an active-learning loop that keeps the predictor sharp exactly where the search is looking.
Where you meet it
Predictors are standard on NAS-Bench-101 and NAS-Bench-201, where ground truth exists for every architecture in the space, and the same machinery predicts on-device latency so a multi-objective search can respect a hardware budget without benchmarking every candidate on the device.
A neural predictor is a surrogate model — typically a graph neural network — trained to estimate an architecture's accuracy from its encoded structure, so a search can rank thousands of candidates without training any of them. It learns from a few hundred architectures that were genuinely trained, and is judged by rank correlation rather than absolute error, since search only needs the ordering. The same approach predicts on-device latency for hardware-aware search.
A wine expert can often tell you roughly how a bottle will taste from the grape, the region and the year — no need to open every one. A neural predictor does that for AI models. It is a small program that has seen a few hundred network designs along with how well each one eventually performed, and from that experience it guesses the score of a design it has never seen. Because guessing is almost free while training is slow and expensive, a search can sift through thousands of candidate designs quickly, then properly train only the handful that look most promising.