Automated Machine Learning under Resource Constraints

Abstract

Automated machine learning (AutoML) seeks to reduce the human and machine costs of finding machine learning models and hyperparameters with good predictive performance. AutoML is easy with unlimited resources: an exhaustive search across all possible solutions finds the best performing model. This dissertation studies resource-constrained AutoML, in which only limited resources (such as compute or memory) are available for model search. We present a wide variety of strategies for choosing a model under resource constraints, including meta-learning across datasets with low rank matrix and tensor decomposition and experiment design, and efficient neural architecture search (NAS) using weight sharing, reinforcement learning, and Monte Carlo sampling. We propose several AutoML frameworks that realize these ideas, and describe implementations and experimental results.