There has recently been incredible progress in developing both theory and practice for applying machine learning (ML) to important problems in causal inference. However, despite this rapidly growing work, settings in which there is unmeasured confounding have remained relatively under-explored, with many remaining challenges and open problems. In this thesis, we consider various causal inference problems in which there is unmeasured confounding, and investigate how to solve them with by utilizing recent advances in adversarial machine learning. We also consider an application of this theory to more efficient policy learning from observational data. We start in Part I by considering the problem of instrumental variable (IV) regression, which is a common framework for causal inference when confounders are not fully observed, but we instead have access to instrumental variables that can only affect the outcome via the treatment. Chapter 2 proposes a novel approach for solving IV regression using smooth game optimization with neural networks, which is motivated by a novel formulation of the optimally-weighted generalized method of moments as a min-max optimization problem. Then, Chapter 3 extends this approach to a more general class of conditional moment problems, as well as to more general ML classes such as kernel spaces, and it analyzes the theoretical properties of such estimators in detail. Next, in Part II we consider various settings of policy evaluation with unobserved confounding. Chapters 4 and 5 both consider approaches to this problem by extending existing work on optimal balancing to such settings, solving for weighted combinations of the observed outcomes that identify the policy value, by minimizing an adversarial formulation of the corresponding risk. Whereas Chapter 4 does this in single treatment settings, Chapter 5 extends this to infinite-horizon RL settings. Then, Chapter 6 considers an even more general partially observed Markov decision process (POMDP) setting, and proposes a policy evaluation method based on a novel sequential extension of existing proximal causal learning methods. In particular, the methods in Chapters 5 and 6 require solving conditional moment problems, for which we propose to use our methodology from Part I. Finally, in Part III we consider an application of our theory to more efficient policy learning given observational data. In Chapter 7, we consider the popular approach of policy learning via surrogate loss reductions, and show that under the assumption that this approach is valid, the optimal policy parameters are defined by a conditional moment restriction. We show how this restriction can be efficiently solved using our methodology from Part I, and that this results in asymptotically optimal regret and superior empirical performance.