Exploiting Composite Functions in Bayesian Optimization

Abstract

Bayesian optimization is a framework for global optimization of objective functions that are expensive or time-consuming to evaluate. It has succeeded in a broad range of application domains, from hyperparameter tuning to chemical design. However, many important problems are still out of its reach. This is partly due to the generality with which classical Bayesian optimization methods treat the objective function, often ignoring available structures that can be extremely useful for optimization. Thus, there is an incentive to identify structural properties arising commonly in practice and develop methods able to leverage them to improve sampling efficiency. This dissertation focuses on objective functions with a composite structure, i.e., objective functions evaluated via two or more functions, some of which take as input the output of others. Composite objective functions are pervasive in real-world applications. They arise, for example, in calibration of expensive simulators, optimization of manufacturing processes, and multi-attribute optimization with preference information. This work develops a general framework to exploit composite functions within Bayesian optimization and demonstrates how it can dramatically improve sampling efficiency and even unlock new applications.