<?xml version='1.0' encoding='UTF-8'?><?xml-stylesheet href='static/style.xsl' type='text/xsl'?><OAI-PMH xmlns="http://www.openarchives.org/OAI/2.0/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd"><responseDate>2026-11-22T11:24:26Z</responseDate><request verb="GetRecord" identifier="oai:ecommons.cornell.edu:1813/115862" metadataPrefix="dim">https://ecommons.cornell.edu/server/oai/request</request><GetRecord><record><header><identifier>oai:ecommons.cornell.edu:1813/115862</identifier><datestamp>2026-05-15T19:40:46Z</datestamp><setSpec>com_1813_35</setSpec><setSpec>col_1813_47</setSpec></header><metadata><dim:dim xmlns:dim="http://www.dspace.org/xmlns/dspace/dim" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:doc="http://www.lyncode.com/xoai" xsi:schemaLocation="http://www.dspace.org/xmlns/dspace/dim http://www.dspace.org/schema/dim.xsd">
   <dim:field mdschema="dc" element="contributor" qualifier="author">Sharma, Yash</dim:field>
   <dim:field mdschema="dc" element="contributor" qualifier="chair" lang="en_US">Choudhury, Sanjiban</dim:field>
   <dim:field mdschema="dc" element="contributor" qualifier="committeeMember" lang="en_US">Edelman, Shimon</dim:field>
   <dim:field mdschema="dc" element="date" qualifier="accessioned">2024-11-05T19:42:55Z</dim:field>
   <dim:field mdschema="dc" element="date" qualifier="available">2024-11-05T19:42:55Z</dim:field>
   <dim:field mdschema="dc" element="date" qualifier="issued">2024-05</dim:field>
   <dim:field mdschema="dc" element="identifier" qualifier="other">ProQuest Submission ID: 12081</dim:field>
   <dim:field mdschema="dc" element="identifier" qualifier="other">ProQuest Publication ID: 31242170</dim:field>
   <dim:field mdschema="dc" element="identifier" qualifier="uri">https://hdl.handle.net/1813/115862</dim:field>
   <dim:field mdschema="dc" element="identifier" qualifier="doi">https://doi.org/10.7298/aywm-eb05</dim:field>
   <dim:field mdschema="dc" element="identifier" qualifier="bibid">16575485</dim:field>
   <dim:field mdschema="dc" element="description" lang="en_US">78 pages</dim:field>
   <dim:field mdschema="dc" element="description" qualifier="abstract" lang="en_US">Two major challenges exist with high-level robot task planning when the goal is under specified: humans have implicit assumptions and preferences they may not articulate when specifying a goal/reward, and visual demonstrations arehard to ground in a form robots can understand. This thesis addresses these challenges leveraging Large Language Models (LLMs) and Vision-Language Models (VLMs) to convert task demonstrations to robot code offline, then adapt to changes planning online. This work is divided into three connected components. Firstly, we developed DEMO2CODE, that generates robot code from demonstrations assumed to be grounded in text form. We focus here on its quality in capturing preferences from real-world demonstrations. Secondly, we delve into grounding visual input in text form with VIDEO2DEMO, focusing on open-vocabulary predicate and action recognition in kitchen tasks. Lastly, we build and deploy an AI task planner to allow collaborative cooking in our MOSAIC framework. We focus here on evaluating the task planner on its safety violations while interacting with participants in our user study. In summary, this thesis focuses on robot task planning and human-robot interaction by using demonstrations for effective code generation, grounding visual information in text, capturing and adhering to preferences of the human, all using generative language models.</dim:field>
   <dim:field mdschema="dc" element="language" qualifier="iso">en</dim:field>
   <dim:field mdschema="dc" element="rights" lang="*">Attribution-ShareAlike 4.0 International</dim:field>
   <dim:field mdschema="dc" element="rights" qualifier="uri" lang="*">https://creativecommons.org/licenses/by-sa/4.0/</dim:field>
   <dim:field mdschema="dc" element="subject" lang="en_US">ai in decision making</dim:field>
   <dim:field mdschema="dc" element="subject" lang="en_US">deep learning</dim:field>
   <dim:field mdschema="dc" element="subject" lang="en_US">large language models</dim:field>
   <dim:field mdschema="dc" element="subject" lang="en_US">machine learning</dim:field>
   <dim:field mdschema="dc" element="title" lang="en_US">LANGUAGE MODELS FOR ROBOT TASK PLANNING WITH HUMAN DEMONSTRATIONS</dim:field>
   <dim:field mdschema="dc" element="type" lang="en_US">dissertation or thesis</dim:field>
   <dim:field mdschema="dc" element="relation" qualifier="localuri">https://newcatalog.library.cornell.edu/catalog/16575485</dim:field>
   <dim:field mdschema="dc" element="format" qualifier="mimetype">application/pdf</dim:field>
   <dim:field mdschema="thesis" element="degree" qualifier="discipline">Computer Science</dim:field>
   <dim:field mdschema="thesis" element="degree" qualifier="grantor">Cornell University</dim:field>
   <dim:field mdschema="thesis" element="degree" qualifier="level">Master of Science</dim:field>
   <dim:field mdschema="thesis" element="degree" qualifier="name">M.S., Computer Science</dim:field>
   <dim:field mdschema="dcterms" element="license">https://hdl.handle.net/1813/59810.2</dim:field>
   <dim:field mdschema="dspace" element="entity" qualifier="type">Publication</dim:field>
   <dim:field mdschema="cris" element="virtual" qualifier="collection" authority="https://cornell-ecommons.eks.prod.4science.cloud/handle/1813/47" confidence="600">Cornell Theses and Dissertations</dim:field>
   <dim:field mdschema="cris" element="virtual" qualifier="author">Sharma, Yash</dim:field>
   <dim:field mdschema="cris" element="virtualsource" qualifier="collection">5893a6ea-7af3-41d7-abc6-04bcd26ab5df</dim:field>
   <dim:field mdschema="others" element="access-status">open.access</dim:field>
   <dim:field mdschema="others" element="access-status">open.access</dim:field>
   <dim:field mdschema="cerif" element="openaire" authority="" confidence="-1">&lt;Publication xmlns="https://www.openaire.eu/cerif-profile/1.1/" id="965ddf53-bec8-4f34-b7de-b3f93336e9ee">
	&lt;Type xmlns="https://www.openaire.eu/cerif-profile/vocab/COAR_Publication_Types">http://purl.org/coar/resource_type/c_1843&lt;/Type>
	&lt;Language>en&lt;/Language>
   	&lt;Title>LANGUAGE MODELS FOR ROBOT TASK PLANNING WITH HUMAN DEMONSTRATIONS&lt;/Title>
   	&lt;PublishedIn>
    	&lt;Publication>
      	&lt;/Publication>
   	&lt;/PublishedIn>
   	&lt;PublicationDate>2024-05&lt;/PublicationDate>
   	&lt;DOI>https://doi.org/10.7298/aywm-eb05&lt;/DOI>
   	&lt;Authors>
      	&lt;Author>
        	&lt;DisplayName>Sharma, Yash&lt;/DisplayName>
         	&lt;Affiliation>
         		&lt;OrgUnit>
         		&lt;/OrgUnit>
         	&lt;/Affiliation>
      	&lt;/Author>
	&lt;/Authors>
   	&lt;Editors>
	&lt;/Editors>
    &lt;Publishers>
        &lt;Publisher>
            &lt;OrgUnit />
        &lt;/Publisher>
    &lt;/Publishers>
    &lt;License>https://creativecommons.org/licenses/by-sa/4.0/&lt;/License>
    &lt;Keyword>ai in decision making&lt;/Keyword>
    &lt;Keyword>deep learning&lt;/Keyword>
    &lt;Keyword>large language models&lt;/Keyword>
    &lt;Keyword>machine learning&lt;/Keyword>
   	&lt;Abstract>Two major challenges exist with high-level robot task planning when the goal is under specified: humans have implicit assumptions and preferences they may not articulate when specifying a goal/reward, and visual demonstrations arehard to ground in a form robots can understand. This thesis addresses these challenges leveraging Large Language Models (LLMs) and Vision-Language Models (VLMs) to convert task demonstrations to robot code offline, then adapt to changes planning online. This work is divided into three connected components. Firstly, we developed DEMO2CODE, that generates robot code from demonstrations assumed to be grounded in text form. We focus here on its quality in capturing preferences from real-world demonstrations. Secondly, we delve into grounding visual input in text form with VIDEO2DEMO, focusing on open-vocabulary predicate and action recognition in kitchen tasks. Lastly, we build and deploy an AI task planner to allow collaborative cooking in our MOSAIC framework. We focus here on evaluating the task planner on its safety violations while interacting with participants in our user study. In summary, this thesis focuses on robot task planning and human-robot interaction by using demonstrations for effective code generation, grounding visual information in text, capturing and adhering to preferences of the human, all using generative language models.&lt;/Abstract>
	&lt;Access xmlns="http://purl.org/coar/access_right" 
    >
    &lt;/Access>
&lt;/Publication>
</dim:field>
</dim:dim>
</metadata></record></GetRecord></OAI-PMH>