Configuration Sampling

An AutoML system cannot learn which pipelines are good until it has tried some, and it has to get those first candidates from somewhere. Configuration sampling is that step: drawing concrete configurations — a preprocessing choice, a model, a full set of hyperparameter values — out of the search space so they can be trained and scored.

How candidates are drawn

Random sampling — draw values independently, usually log-uniformly for scale parameters like learning rate. It is the baseline Bergstra and Bengio showed beats grid search at equal cost.
Model-based sampling — a surrogate fitted to past results proposes what to try next, as Optuna's TPE sampler, SMAC's random-forest model and Hyperopt all do.
Space-filling designs — Sobol or Latin-hypercube draws spread the first batch more evenly than pure randomness, giving the surrogate a better initial look.
Meta-learned warm starts — begin from configurations that worked on similar datasets, the trick Auto-sklearn uses to fill its initial design.

Validity is half the job
Pipeline spaces are conditional: max_depth exists only if the model is a tree, a scaler applies only to numeric columns, and some legal-looking combinations cannot be built at all. A sampler that ignores those dependencies spends budget on configurations that crash before they train, which is why libraries either declare the space explicitly or define it by running code, as Optuna's trial.suggest_float calls do.

Why it matters
Every sample costs a full train-and-validate cycle, so the sampling rule — more than the optimiser's cleverness — decides how much of the space a fixed compute budget really covers. Sample too timidly and the search plateaus early on near-identical candidates; sample without structure and you never exploit what the first results told you.

Configuration sampling is the act of drawing candidate configurations — preprocessing steps, model choice and hyperparameter values — from an AutoML search space so they can be trained and evaluated. Samplers range from random and space-filling draws to surrogate-guided and meta-learned proposals. It matters because each sample costs a full training run, so the sampling rule, together with how well it respects conditional dependencies in the space, sets how much of that space a fixed budget explores.

Picture a huge wall of dials: which model to use, how deep it should be, how fast it should learn. Nobody can try every combination, so a tool that builds models automatically has to choose which settings to test. Configuration sampling is how it picks. Sometimes it simply guesses at random, which works surprisingly well. More often it learns from the attempts it has already made and leans towards settings that looked promising — like a cook tasting as they season, rather than trying every recipe in the book. Since each attempt costs time and computing power, choosing well is most of the skill.