<?xml version='1.0' encoding='UTF-8'?><?xml-stylesheet href='static/style.xsl' type='text/xsl'?><OAI-PMH xmlns="http://www.openarchives.org/OAI/2.0/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd"><responseDate>2026-11-22T10:38:29Z</responseDate><request verb="GetRecord" identifier="oai:ecommons.cornell.edu:1813/116633" metadataPrefix="dim">https://ecommons.cornell.edu/server/oai/request</request><GetRecord><record><header><identifier>oai:ecommons.cornell.edu:1813/116633</identifier><datestamp>2026-05-15T19:45:10Z</datestamp><setSpec>com_1813_35</setSpec><setSpec>col_1813_47</setSpec></header><metadata><dim:dim xmlns:dim="http://www.dspace.org/xmlns/dspace/dim" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:doc="http://www.lyncode.com/xoai" xsi:schemaLocation="http://www.dspace.org/xmlns/dspace/dim http://www.dspace.org/schema/dim.xsd">
   <dim:field mdschema="dc" element="contributor" qualifier="author">Yauney, Gregory</dim:field>
   <dim:field mdschema="dc" element="contributor" qualifier="chair" lang="en_US">Mimno, David</dim:field>
   <dim:field mdschema="dc" element="contributor" qualifier="committeeMember" lang="en_US">Benson, Austin</dim:field>
   <dim:field mdschema="dc" element="contributor" qualifier="committeeMember" lang="en_US">Chattopadhyay, Eshan</dim:field>
   <dim:field mdschema="dc" element="date" qualifier="accessioned">2025-01-14T20:01:20Z</dim:field>
   <dim:field mdschema="dc" element="date" qualifier="available">2025-01-14T20:01:20Z</dim:field>
   <dim:field mdschema="dc" element="date" qualifier="issued">2024-08</dim:field>
   <dim:field mdschema="dc" element="identifier" qualifier="other">ProQuest Submission ID: 14600</dim:field>
   <dim:field mdschema="dc" element="identifier" qualifier="other">ProQuest Publication ID: 31489088</dim:field>
   <dim:field mdschema="dc" element="identifier" qualifier="uri">https://hdl.handle.net/1813/116633</dim:field>
   <dim:field mdschema="dc" element="identifier" qualifier="doi">https://doi.org/10.7298/n297-1511</dim:field>
   <dim:field mdschema="dc" element="identifier" qualifier="bibid">16612027</dim:field>
   <dim:field mdschema="dc" element="description" lang="en_US">178 pages</dim:field>
   <dim:field mdschema="dc" element="description" qualifier="abstract" lang="en_US">Machine learning models can now achieve high performance on many natural language classification tasks. But we currently don't know how well a contemporary large language model will perform on a new task without directly trying it out. What makes a task difficult for machine learning models? We focus on the role of data---both a model's training data and that of downstream tasks---to go beyond evaluation performance in characterizing the difficulty of natural language tasks. We intervene throughout the language modeling pipeline, examining the interaction between a task's dataset and a) pretrained representations, b) evaluation, and c) pretraining data. We use random labelings to contextualize the degree of alignment between a task's data and a task's labels under different text representations. We use classifiers that guess uniformly at random, independently across examples, to contextualize a language model's performance on the small datasets typically used to evaluate in-context learning capabilities. We also examine the extent of evidence for the hypothesis that a downstream dataset’s similarity to a model’s pretraining dataset determines the model’s performance. Finally, we turn to case studies across image-text grounding, literary history, and architectural history where we are specifically interested in a model's performance on a given challenging dataset. Understanding the interaction between data and model will make our models ever more reliable on datasets that we care about, ultimately meeting text datasets where they are.</dim:field>
   <dim:field mdschema="dc" element="language" qualifier="iso">en</dim:field>
   <dim:field mdschema="dc" element="title" lang="en_US">Characterizing the Difficulty of Natural Language Datasets for Machine Learning</dim:field>
   <dim:field mdschema="dc" element="type" lang="en_US">dissertation or thesis</dim:field>
   <dim:field mdschema="dc" element="relation" qualifier="localuri">https://newcatalog.library.cornell.edu/catalog/16612027</dim:field>
   <dim:field mdschema="dc" element="format" qualifier="mimetype">application/pdf</dim:field>
   <dim:field mdschema="thesis" element="degree" qualifier="discipline">Computer Science</dim:field>
   <dim:field mdschema="thesis" element="degree" qualifier="grantor">Cornell University</dim:field>
   <dim:field mdschema="thesis" element="degree" qualifier="level">Doctor of Philosophy</dim:field>
   <dim:field mdschema="thesis" element="degree" qualifier="name">Ph. D., Computer Science</dim:field>
   <dim:field mdschema="dcterms" element="license">https://hdl.handle.net/1813/59810.2</dim:field>
   <dim:field mdschema="dspace" element="entity" qualifier="type">Publication</dim:field>
   <dim:field mdschema="cris" element="virtual" qualifier="collection" authority="https://cornell-ecommons.eks.prod.4science.cloud/handle/1813/47" confidence="600">Cornell Theses and Dissertations</dim:field>
   <dim:field mdschema="cris" element="virtual" qualifier="author">Yauney, Gregory</dim:field>
   <dim:field mdschema="cris" element="virtualsource" qualifier="collection">5893a6ea-7af3-41d7-abc6-04bcd26ab5df</dim:field>
   <dim:field mdschema="others" element="access-status">open.access</dim:field>
   <dim:field mdschema="others" element="access-status">open.access</dim:field>
   <dim:field mdschema="cerif" element="openaire" authority="" confidence="-1">&lt;Publication xmlns="https://www.openaire.eu/cerif-profile/1.1/" id="7b12ef19-f9bc-4389-8734-00ac17430c8d">
	&lt;Type xmlns="https://www.openaire.eu/cerif-profile/vocab/COAR_Publication_Types">http://purl.org/coar/resource_type/c_1843&lt;/Type>
	&lt;Language>en&lt;/Language>
   	&lt;Title>Characterizing the Difficulty of Natural Language Datasets for Machine Learning&lt;/Title>
   	&lt;PublishedIn>
    	&lt;Publication>
      	&lt;/Publication>
   	&lt;/PublishedIn>
   	&lt;PublicationDate>2024-08&lt;/PublicationDate>
   	&lt;DOI>https://doi.org/10.7298/n297-1511&lt;/DOI>
   	&lt;Authors>
      	&lt;Author>
        	&lt;DisplayName>Yauney, Gregory&lt;/DisplayName>
         	&lt;Affiliation>
         		&lt;OrgUnit>
         		&lt;/OrgUnit>
         	&lt;/Affiliation>
      	&lt;/Author>
	&lt;/Authors>
   	&lt;Editors>
	&lt;/Editors>
    &lt;Publishers>
        &lt;Publisher>
            &lt;OrgUnit />
        &lt;/Publisher>
    &lt;/Publishers>
   	&lt;Abstract>Machine learning models can now achieve high performance on many natural language classification tasks. But we currently don&amp;apos;t know how well a contemporary large language model will perform on a new task without directly trying it out. What makes a task difficult for machine learning models? We focus on the role of data---both a model&amp;apos;s training data and that of downstream tasks---to go beyond evaluation performance in characterizing the difficulty of natural language tasks. We intervene throughout the language modeling pipeline, examining the interaction between a task&amp;apos;s dataset and a) pretrained representations, b) evaluation, and c) pretraining data. We use random labelings to contextualize the degree of alignment between a task&amp;apos;s data and a task&amp;apos;s labels under different text representations. We use classifiers that guess uniformly at random, independently across examples, to contextualize a language model&amp;apos;s performance on the small datasets typically used to evaluate in-context learning capabilities. We also examine the extent of evidence for the hypothesis that a downstream dataset’s similarity to a model’s pretraining dataset determines the model’s performance. Finally, we turn to case studies across image-text grounding, literary history, and architectural history where we are specifically interested in a model&amp;apos;s performance on a given challenging dataset. Understanding the interaction between data and model will make our models ever more reliable on datasets that we care about, ultimately meeting text datasets where they are.&lt;/Abstract>
	&lt;Access xmlns="http://purl.org/coar/access_right" 
    >
    &lt;/Access>
&lt;/Publication>
</dim:field>
</dim:dim>
</metadata></record></GetRecord></OAI-PMH>