Automated Feature Engineering
Most of the lift in a tabular model comes from the columns you feed it rather than the algorithm you pick, and building good columns has always been the part of the job that depends on knowing the subject. Automated feature engineering tests how much of that craft is really just systematic search.
How it works
The idea rests on primitives — small reusable operations applied to raw columns and then to each other's output:
- Transform primitives act within a row: day-of-week from a timestamp, logarithm of a price, length of an address.
- Aggregation primitives collapse related rows: count of a customer's orders, mean basket value, days since last login.
- Stacked primitives go deeper, producing candidates nobody writes by hand — the standard deviation of the gap between one customer's sessions.
Generation plus selection
Featuretools implements this as Deep Feature Synthesis across a set of related tables, tsfresh extracts hundreds of statistical descriptors from time series and prunes them with hypothesis tests, and OpenFE and Featurewiz generate then filter candidates for flat tables. Generation is the cheap half. Every serious implementation pairs it with selection, because a thousand candidate columns over ten thousand rows will contain several that match the target by luck alone.
Why it matters
The analyst's job changes shape: instead of inventing features one at a time, you declare how your tables relate and the moment at which each prediction is made, then let the machine enumerate. That second declaration carries more weight than it sounds — it is what stops an aggregate from quietly including information recorded after the event being predicted.
Automated feature engineering is the machine generation, combination and selection of input features from raw data, replacing hand-crafted column design. Tools apply transformation and aggregation primitives — counts, means, time gaps, date parts — across related tables, stack them to a chosen depth, then prune candidates that earn nothing on held-out data. It matters because feature quality dominates tabular accuracy, and the search covers combinations no analyst would write by hand.
A computer learning from a spreadsheet is only as good as the columns it is given. "Date joined" is not much use on its own; "number of orders in the last 30 days" is gold. Working out which columns like that to build used to be a human job that took weeks.
Automated feature engineering hands the inventing to software. It takes the raw records — every purchase, click and login — and builds thousands of candidate summaries from them: averages, counts, gaps between events, totals per customer. Then it throws away the ones that turn out not to help with the prediction. What survives is usually a better set of columns than most people would have thought to write.