Counterfactual Evaluation And Learning From Logged User Feedback

Other Titles
Interactive systems that interact with and learn from user behavior are ubiquitous today. Machine learning algorithms are core components of such systems. In this thesis, we will study how we can re-use logged user behavior data to evaluate interactive systems and train their machine learned components in a principled way. The core message of the thesis is -- Using simple techniques from causal inference, we can improve popular machine learning algorithms so that they interact reliably. -- These improvements are effective and scalable, and complement current algorithmic and modeling advances in machine learning. -- They open further avenues for research in Counterfactual Evaluation and Learning to ensure machine learned components interact reliably with users and with each other. This thesis explores two fundamental tasks — evaluation and training of interactive systems. Solving evaluation and training tasks using logged data is an exercise in counterfactual reasoning. So we will first review concepts from causal inference for counterfactual reasoning, assignment mechanisms, statistical estimation and learning theory. The thesis then contains two parts. In the first part, we will study scenarios where unknown assignment mechanisms underlie the logged data we collect. These scenarios often arise in learning-to-rank and learning-to-recommend applications. We will view these applications through the lens of causal inference and modularize the problem of building a good ranking engine or recommender system into two components — first, infer a plausible assignment mechanism and second, reliably learn to rank or recommend assuming this mechanism was active when collecting data. The second part of the thesis focuses on scenarios where we collect logged data from past interventions. We will formalize these scenarios as batch learning from logged contextual bandit feedback. We will first develop better off-policy estimators for evaluating online user-centric metrics in information retrieval applications. In subsequent chapters, we will study the bias-variance trade-off when learning from logged interventions. This study will yield new learning principles, algorithms and insights into the design of statistical estimators for counterfactual learning. The thesis outlines a few principles, tools, datasets and software that hopefully prove to be useful to you as you build your interactive learning system.
Journal / Series
Volume & Issue
Date Issued
Statistics; Computer science; machine learning; Causality; Off-Policy
Effective Date
Expiration Date
Union Local
Number of Workers
Committee Chair
Joachims, Thorsten
Committee Co-Chair
Committee Member
Gehrke, Johannes E
Tardos, Eva
Kleinberg, Robert D
Degree Discipline
Computer Science
Degree Name
Ph. D., Computer Science
Degree Level
Doctor of Philosophy
Related Version
Related DOI
Related To
Related Part
Based on Related Item
Has Other Format(s)
Part of Related Item
Related To
Related Publication(s)
Link(s) to Related Publication(s)
Link(s) to Reference(s)
Previously Published As
Government Document
Other Identifiers
Attribution 4.0 International
dissertation or thesis
Accessibility Feature
Accessibility Hazard
Accessibility Summary
Link(s) to Catalog Record