Notes

Model Recommender

Ask an experienced practitioner what to try on a new tabular dataset and you get an answer in seconds, not a week of experiments. A model recommender is the component that gives an AutoML system the same reflex.

How a recommendation is produced

The recommender holds a performance history: datasets on one axis, candidate models or pipelines on the other, scores in the cells. For a new dataset it has two ways to use that. Nearest-neighbour retrieval describes the dataset with meta-features, finds the most similar stored datasets, and returns what won there. Collaborative filtering treats the table as a ratings matrix and factorises it, so a few cheap evaluations on your data are enough to estimate the rest of the row — the same mathematics behind a shop's recommendations.

What it returns

  • A ranked shortlist, not a single answer — the search still verifies the top candidates by training them.
  • Initial configurations as well as model families, so tuning starts from settings that have already worked somewhere.
  • A confidence signal — when no stored dataset resembles yours, a good recommender says so instead of guessing.

Why it matters

Recommendation is the cheapest step in the whole pipeline: an inference over a stored table, against hours of model fitting. It largely determines where the search begins, and under a tight time budget where it begins is most of where it ends up. The flip side is inherited bias — a recommender can only ever suggest what its history contains.

A model recommender suggests which models or pipelines to try on a new dataset before any of them is trained, drawing on a stored history of how candidates performed on previous datasets. It works by nearest-dataset retrieval over meta-features or by matrix factorisation over the performance table. It matters because a recommendation costs one inference while an evaluation costs a training run, so it largely decides where a time-limited AutoML search begins.

Streaming services suggest what to watch next by noticing that your taste matches other people's, then offering what those people enjoyed. You still decide, but the shortlist saves you the scrolling.

A model recommender does this for prediction methods. It keeps records of which approaches worked well on many previous datasets. Shown a new one, it finds the datasets that look most like yours and suggests the methods that succeeded there. Those suggestions still get tested properly afterwards — the point is simply to start with a few promising candidates instead of trying everything.