Search Space (AutoML)

Before an AutoML system can search for anything, somebody has to say what it is allowed to consider. That declaration is the search space: the complete set of candidate configurations — algorithms, their hyperparameters, preprocessing choices, sometimes whole network architectures — with the type and range of every knob written down.

What a space is made of
A search space is a typed, structured object rather than a flat list:

• Dimension types — continuous (learning rate), integer (tree depth), categorical (solver), ordinal (batch size);
• Ranges and scales — learning rates are searched log-uniformly, because the useful resolution is in orders of magnitude;
• Conditionals — a kernel coefficient exists only if the model is an SVM with an RBF kernel, so parts of the space switch on and off;
• Forbidden clauses — invalid or guaranteed-to-crash combinations, excluded so no trial is wasted on them.

The ConfigSpace library exists to express exactly this, and it is how Auto-sklearn declares its roughly hundred-hyperparameter space.

Why it matters in AutoML
The space sets both the ceiling and the difficulty of the search. If the optimum lies outside the declared ranges, no optimiser will find it: the result is bounded by the declaration, not by the algorithm. Make the space too wide and the budget drains into regions nobody would deploy. In practice a carefully bounded space with sensible scales and good defaults beats a sophisticated optimiser turned loose on a careless one — which is why search-space design is where experienced practitioners spend their judgement.