Automated Machine Learning
AutoML automates the pipeline of feature processing, model selection, and tuning to build models with less manual effort.
Automating the pipeline
Building a good model involves many choices: how to clean and encode features, which algorithm to use, and how to set its hyperparameters. Automated machine learning (AutoML) systems search over these choices automatically, aiming to produce a strong model with minimal human tuning and to make modeling accessible beyond specialists.
What AutoML searches over
- Data preparation: imputation, encoding, scaling, feature selection
- Model selection: choosing among algorithm families
- Hyperparameter optimization: tuning each candidate
- Pipeline construction: composing preprocessing and models into a full pipeline
- Ensembling: combining several strong pipelines
Search strategies
Approaches range from exhaustive search over a fixed pipeline space, to Bayesian optimization over combined pipeline and hyperparameter choices, to evolutionary search that mutates and recombines pipelines, to meta-learning that warm-starts from what worked on similar past datasets. Many systems finish by building an ensemble of the best pipelines found, which usually beats any single one.
Benefits and limits
AutoML reliably matches or beats a competent practitioner on well-defined tabular problems and frees experts from repetitive tuning. But it does not automate the hard parts: framing the problem, sourcing and validating data, guarding against leakage, choosing an honest evaluation, and judging fairness and deployment fit. It can also overfit the validation signal if the search is long and the validation set small, so a truly held-out test set remains essential.
For neural networks specifically, the architecture itself becomes a search target, the subject of neural architecture search.