Configuration Space
Before an AutoML run can optimise anything, somebody has to say what it is allowed to change. The configuration space is that declaration: the complete set of decisions available to the search, each with its type, its legal range, and the conditions under which it applies at all.
What the space contains
- Typed hyperparameters — continuous (learning rate, usually on a log scale), integer (tree depth), categorical (kernel choice) and ordinal values.
- Pipeline slots — which imputer, encoder, scaler and feature selector run before which estimator, so the space covers whole pipelines rather than one model's knobs.
- Conditions — n_estimators is active only when the model is a forest, gamma only when the kernel is RBF. The space is a tree of possibilities, not a rectangle.
- Forbidden clauses — combinations that are individually legal but invalid together, ruled out up front rather than discovered by a crash.
How it is written down
Two styles dominate. Auto-sklearn and SMAC declare the space as data with the ConfigSpace library, which lets the optimiser inspect its structure. Optuna takes the define-by-run route: the space is whatever the objective function asks for through trial.suggest_float and its siblings, so ordinary Python branching expresses the conditions.
Why it decides the outcome
The space is the strongest lever you hold over an AutoML result, because a search cannot return what the space does not contain. Draw it too narrowly and the winning model was never a candidate; draw it too widely and a fixed budget spreads so thin that nothing gets tuned properly. Its structure also constrains the method: a plain Gaussian-process optimiser wants continuous dimensions, and conditional, mixed-type pipeline spaces are exactly why random-forest surrogates and tree-structured samplers exist.
The configuration space defines everything an AutoML search may vary: the pipeline steps, each hyperparameter with its type and legal range, the conditions that make a parameter active, and combinations forbidden outright. It is the strongest lever on the outcome, since a search can only return something the space contains — too narrow and the best model is excluded, too wide and the budget spreads too thin to tune anything well. Its structure also dictates which optimisers apply.
Think of a kitchen where someone is about to cook for you, and you hand them a list of what they may use: these ingredients, these amounts, this oven — and no, not the deep fryer. The configuration space is that list for a tool that builds machine-learning models by itself. It spells out which choices the computer is free to make while hunting for a good model. The list matters enormously. Leave something off and the tool can never try it, even if it would have been the best answer. Make the list too long and it runs out of time before testing anything properly.