Once-for-All Networks
The usual way to ship a model to ten different devices is to search and train ten times, once per phone, watch or camera. Once-for-All networks break that pattern: train one over-provisioned network a single time, then pull specialised sub-networks out of it for each deployment target, with no retraining at all.
Progressive shrinking
The trick is in how the parent network is trained. OFA starts from the largest configuration, then progressively allows smaller ones, shrinking along four elastic axes — kernel size, depth (layers per block), width (channels per layer) and input resolution. Sub-networks inherit the big network's weights, with channels sorted by importance so a narrowed layer keeps its most useful filters. Shrinking progressively, rather than sampling every configuration from the start, stops the sub-networks from destroying each other's weights — the interference that cripples naive weight sharing.
Deployment becomes a search, not a training job
What comes out is on the order of 10^19 valid sub-networks, each usable immediately. To serve a specific constraint — under 20 ms on this particular phone — OFA runs an evolutionary search over sub-networks, scoring them with a small accuracy predictor and a measured latency table for the device rather than training or timing anything. Targeting a new device takes minutes.
Why it matters
It decouples training cost from the number of deployment targets, which is the economics that made hardware-aware architecture search practical at all. The method won the Low Power Computer Vision Challenge, and its released supernets are still a common starting point for efficient vision backbones.
Once-for-All trains a single over-provisioned network once, then extracts specialised sub-networks for different hardware targets without retraining any of them. Progressive shrinking trains the parent along four elastic axes — kernel size, depth, width and input resolution — so sub-networks inherit usable weights instead of interfering with each other. Deployment becomes a cheap evolutionary search guided by an accuracy predictor and a device latency table, decoupling training cost from the number of devices served.
A tailor can either sew a brand-new suit for every customer, or cut one very generous garment that can be taken in to fit anybody. Once-for-All networks take the second approach to AI models. Instead of training a separate model for each phone, watch or camera, researchers train one large, flexible network a single time. Smaller models are then simply carved out of it — narrower, shallower, or working on smaller images — and each one works straight away, with no further training. Fitting a model to a brand-new device drops from days of computing to a few minutes of searching.