Bayesian optimization, a framework for global optimization of expensive-to-evaluate functions, has shown success in machine learning and experimental design because it is able to find global optima with a remarkably small number of potentially noisy objective function evaluations. In this dissertation, we study in detail how the concept of the knowledge gradient (KG) can be adopted to design novel Bayesian optimization algorithms. First, we propose a novel parallel Bayesian optimization algorithm by generalizing the concept of KG from the fully sequential setting to the parallel setting (qKG). By construction, this method provides a one-step Bayes-optimal batch of points to sample. We provide an efficient strategy for computing this Bayes-optimal batch of points, and we demonstrate that the parallel knowledge gradient method finds global optima significantly faster than previous batch Bayesian optimization algorithms on both synthetic test functions and when tuning hyperparameters of practical machine learning algorithms, especially when function evaluations are noisy. Second, we present a novel discretization-free strategy to calculate the set of points to be evaluated under the knowledge gradient method when used over a continuous domain. KG methods are widely studied for discrete ranking and selection problems, and provide a one-step Bayes-optimal point to sample, but all the previous efforts generalizing KG to continuous domains rely on a discretized finite approximation due to the computational challenges in calculating KG. However, the discretization introduces error and scales poorly as the dimension of the domain grows. In this chapter, we develop a fast discretization-free knowledge gradient method for Bayesian optimization, which is useful for all settings where KG is used over a continuous domain that overcomes these challenges. Third, we explore the “what, when, and why” of Bayesian optimization with derivative information. We also develop a Bayesian optimization algorithm that effectively leverages gradients. This algorithm accommodates incomplete and noisy gradient observations, can be used in both the sequential and batch settings, and can optionally reduce the computational overhead of inference by selecting the single most valuable directional derivative to retain. For this purpose, we develop a novel acquisition function, called the derivative-enabled knowledge-gradient (dKG). This generalizes the previously proposed batch knowledge gradient method to the derivative setting. We also provide a theoretical analysis of the algorithm: it is one- step Bayes-optimal by construction when derivatives are available, and we show (1) that it provides one-step value greater than in the derivative-free setting; and (2) that its estimator of the global optimum is asymptotically consistent. Fourth, we show some preliminary results on how KG can be adopted to settings where we have some low-fidelity but cheap approximations. To this end, we develop a novel Bayesian optimization algorithm, continuous-fidelity knowledge gradient (cfKG), which can adaptively choose both the fidelity and the desired point to sample by better balancing the trade-off between the information gain vs. the cost when we have some continuous parameters controlling the fidelity of the information source we can query. Some preliminary numerical results are shown.