<?xml version='1.0' encoding='UTF-8'?><?xml-stylesheet href='static/style.xsl' type='text/xsl'?><OAI-PMH xmlns="http://www.openarchives.org/OAI/2.0/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd"><responseDate>2026-11-22T10:47:40Z</responseDate><request verb="GetRecord" identifier="oai:ecommons.cornell.edu:1813/117238" metadataPrefix="dim">https://ecommons.cornell.edu/server/oai/request</request><GetRecord><record><header><identifier>oai:ecommons.cornell.edu:1813/117238</identifier><datestamp>2026-05-15T19:49:49Z</datestamp><setSpec>com_1813_35</setSpec><setSpec>col_1813_47</setSpec></header><metadata><dim:dim xmlns:dim="http://www.dspace.org/xmlns/dspace/dim" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:doc="http://www.lyncode.com/xoai" xsi:schemaLocation="http://www.dspace.org/xmlns/dspace/dim http://www.dspace.org/schema/dim.xsd">
   <dim:field mdschema="dc" element="contributor" qualifier="author">Tucker, Aaron</dim:field>
   <dim:field mdschema="dc" element="contributor" qualifier="chair" lang="en_US">Joachims, Thorsten</dim:field>
   <dim:field mdschema="dc" element="contributor" qualifier="committeeMember" lang="en_US">Sun, Wen</dim:field>
   <dim:field mdschema="dc" element="contributor" qualifier="committeeMember" lang="en_US">Weinberger, Kilian</dim:field>
   <dim:field mdschema="dc" element="date" qualifier="accessioned">2025-06-30T22:04:04Z</dim:field>
   <dim:field mdschema="dc" element="date" qualifier="available">2025-06-30T22:04:04Z</dim:field>
   <dim:field mdschema="dc" element="date" qualifier="issued">2024-12</dim:field>
   <dim:field mdschema="dc" element="identifier" qualifier="other">ProQuest Submission ID: 14777</dim:field>
   <dim:field mdschema="dc" element="identifier" qualifier="other">ProQuest Publication ID: 31641406</dim:field>
   <dim:field mdschema="dc" element="identifier" qualifier="uri">https://hdl.handle.net/1813/117238</dim:field>
   <dim:field mdschema="dc" element="identifier" qualifier="doi">http://doi.org/10.7298/pvzv-wh28</dim:field>
   <dim:field mdschema="dc" element="identifier" qualifier="bibid">16922020</dim:field>
   <dim:field mdschema="dc" element="description" lang="en_US">199 pages</dim:field>
   <dim:field mdschema="dc" element="description" qualifier="abstract" lang="en_US">AI systems have become increasingly powerful by using increasing amounts of compute in order to implement more and more complex tasks. For example, writing a rules-based search engine that explicitly computes what links a specific user would be interested in for a given query is prohibitively difficult, but machine learning can predict what links a user is most likely to click on. Reinforcement learning from human feedback takes ML further by learning a reward model from preference comparisons to improve an LLM's responses, rather than trying to specify good behavior directly. In both of these settings, human feedback is critical to aligning ML systems for personalization and to perform difficult-to-directly-specify tasks. While compute gets cheaper and cheaper, human attention stays expensive. How can human feedback be more efficient? How can it be more available? This thesis presents several projects which improve data efficiency and data availability for aligning AI systems using human feedback. First, it analyzes the exploration and exploitation tradeoff induced by introducing an explicit cost for observing rewards in a bandit setting. Second, it provides methods for allocating a fixed interaction budget to improve the data efficiency of off-policy learning and evaluation in realistic settings appropriate for search and recommendation systems. Third, it shows how to improve LLM-based assistants using implicit feedback gathered from user interactions. Finally, it outlines preliminary results and promising directions for future work for using human feedback.</dim:field>
   <dim:field mdschema="dc" element="language" qualifier="iso">en</dim:field>
   <dim:field mdschema="dc" element="rights" lang="*">Attribution 4.0 International</dim:field>
   <dim:field mdschema="dc" element="rights" qualifier="uri" lang="*">https://creativecommons.org/licenses/by/4.0/</dim:field>
   <dim:field mdschema="dc" element="subject" lang="en_US">Active Learning</dim:field>
   <dim:field mdschema="dc" element="subject" lang="en_US">Alignment</dim:field>
   <dim:field mdschema="dc" element="subject" lang="en_US">Bandits</dim:field>
   <dim:field mdschema="dc" element="subject" lang="en_US">Off-policy Learning</dim:field>
   <dim:field mdschema="dc" element="subject" lang="en_US">RLHF</dim:field>
   <dim:field mdschema="dc" element="title" lang="en_US">Improving Data Efficiency and Availability for Alignment</dim:field>
   <dim:field mdschema="dc" element="type" lang="en_US">dissertation or thesis</dim:field>
   <dim:field mdschema="dc" element="relation" qualifier="localuri">https://newcatalog.library.cornell.edu/catalog/16922020</dim:field>
   <dim:field mdschema="dc" element="format" qualifier="mimetype">application/pdf</dim:field>
   <dim:field mdschema="thesis" element="degree" qualifier="discipline">Computer Science</dim:field>
   <dim:field mdschema="thesis" element="degree" qualifier="grantor">Cornell University</dim:field>
   <dim:field mdschema="thesis" element="degree" qualifier="level">Doctor of Philosophy</dim:field>
   <dim:field mdschema="thesis" element="degree" qualifier="name">Ph. D., Computer Science</dim:field>
   <dim:field mdschema="dcterms" element="license">https://hdl.handle.net/1813/59810.2</dim:field>
   <dim:field mdschema="dspace" element="entity" qualifier="type">Publication</dim:field>
   <dim:field mdschema="cris" element="virtual" qualifier="collection" authority="https://cornell-ecommons.eks.prod.4science.cloud/handle/1813/47" confidence="600">Cornell Theses and Dissertations</dim:field>
   <dim:field mdschema="cris" element="virtual" qualifier="author">Tucker, Aaron</dim:field>
   <dim:field mdschema="cris" element="virtualsource" qualifier="collection">5893a6ea-7af3-41d7-abc6-04bcd26ab5df</dim:field>
   <dim:field mdschema="others" element="access-status">open.access</dim:field>
   <dim:field mdschema="others" element="access-status">open.access</dim:field>
   <dim:field mdschema="cerif" element="openaire" authority="" confidence="-1">&lt;Publication xmlns="https://www.openaire.eu/cerif-profile/1.1/" id="988cffd6-0b48-4105-b08b-f199ccae1f58">
	&lt;Type xmlns="https://www.openaire.eu/cerif-profile/vocab/COAR_Publication_Types">http://purl.org/coar/resource_type/c_1843&lt;/Type>
	&lt;Language>en&lt;/Language>
   	&lt;Title>Improving Data Efficiency and Availability for Alignment&lt;/Title>
   	&lt;PublishedIn>
    	&lt;Publication>
      	&lt;/Publication>
   	&lt;/PublishedIn>
   	&lt;PublicationDate>2024-12&lt;/PublicationDate>
   	&lt;DOI>http://doi.org/10.7298/pvzv-wh28&lt;/DOI>
   	&lt;Authors>
      	&lt;Author>
        	&lt;DisplayName>Tucker, Aaron&lt;/DisplayName>
         	&lt;Affiliation>
         		&lt;OrgUnit>
         		&lt;/OrgUnit>
         	&lt;/Affiliation>
      	&lt;/Author>
	&lt;/Authors>
   	&lt;Editors>
	&lt;/Editors>
    &lt;Publishers>
        &lt;Publisher>
            &lt;OrgUnit />
        &lt;/Publisher>
    &lt;/Publishers>
    &lt;License>https://creativecommons.org/licenses/by/4.0/&lt;/License>
    &lt;Keyword>Active Learning&lt;/Keyword>
    &lt;Keyword>Alignment&lt;/Keyword>
    &lt;Keyword>Bandits&lt;/Keyword>
    &lt;Keyword>Off-policy Learning&lt;/Keyword>
    &lt;Keyword>RLHF&lt;/Keyword>
   	&lt;Abstract>AI systems have become increasingly powerful by using increasing amounts of compute in order to implement more and more complex tasks. For example, writing a rules-based search engine that explicitly computes what links a specific user would be interested in for a given query is prohibitively difficult, but machine learning can predict what links a user is most likely to click on. Reinforcement learning from human feedback takes ML further by learning a reward model from preference comparisons to improve an LLM&amp;apos;s responses, rather than trying to specify good behavior directly. In both of these settings, human feedback is critical to aligning ML systems for personalization and to perform difficult-to-directly-specify tasks. While compute gets cheaper and cheaper, human attention stays expensive. How can human feedback be more efficient? How can it be more available? This thesis presents several projects which improve data efficiency and data availability for aligning AI systems using human feedback. First, it analyzes the exploration and exploitation tradeoff induced by introducing an explicit cost for observing rewards in a bandit setting. Second, it provides methods for allocating a fixed interaction budget to improve the data efficiency of off-policy learning and evaluation in realistic settings appropriate for search and recommendation systems. Third, it shows how to improve LLM-based assistants using implicit feedback gathered from user interactions. Finally, it outlines preliminary results and promising directions for future work for using human feedback.&lt;/Abstract>
	&lt;Access xmlns="http://purl.org/coar/access_right" 
    >
    &lt;/Access>
&lt;/Publication>
</dim:field>
</dim:dim>
</metadata></record></GetRecord></OAI-PMH>