Compute Budget (AutoML)
Hand an AutoML system a dataset and it will keep proposing new pipelines for as long as you let it. The compute budget is how you decide when to stop — the resource ceiling a search may consume before it must hand back the best model it has found.
What the budget counts
A budget is stated in whichever resource is scarce for you:
- Wall-clock time — the time_left_for_this_task argument in Auto-sklearn, or time_budget in FLAML.
- Number of evaluations — a fixed count of candidate configurations, such as n_trials in Optuna.
- Hardware time — CPU-hours or GPU-hours, the honest unit once a search runs in parallel across workers.
- Fidelity units — epochs, boosting rounds or data subsamples, the currency that multi-fidelity methods such as Hyperband and BOHB allocate and reclaim.
Why it shapes the result
The budget is part of the problem definition, not a footnote on the run. It decides how much of the search space is reachable, and it changes which strategy is correct: under a tight budget, meta-learned warm starts and a small algorithm portfolio beat a cold Bayesian search, while a generous budget pays for full pipeline search and ensembling of the top models. Set it too low and you get a barely tuned default; set it too high and you buy the third decimal place of accuracy with real money.
Budgets make results comparable
Every AutoML score is the score of a search that was cut off somewhere, so a claim that one framework beats another means nothing unless both were given the same budget. This is why benchmark suites fix a per-task time limit and report anytime-performance curves instead of a single final number.
A compute budget is the resource ceiling placed on an AutoML search — wall-clock time, a number of trials, or CPU/GPU-hours — after which the system must return the best pipeline it has found. Because every AutoML result is the product of a search that was cut off somewhere, the budget belongs to the problem definition: it decides how much of the search space can be explored, and comparisons between AutoML systems are meaningless unless their budgets match.
Think of sending a friend out to find you the perfect pair of shoes. Give them one hour and they check two shops and bring back something decent. Give them a whole weekend and they compare dozens. A compute budget is that limit for a tool that builds machine-learning models on your behalf. You tell it how long it may run, or how many attempts it may make, and it hands back the best model it managed to find in that window. The limit matters because the answer genuinely depends on it — and because comparing two tools is only fair when both were given the same time.