<?xml version='1.0' encoding='UTF-8'?><?xml-stylesheet href='static/style.xsl' type='text/xsl'?><OAI-PMH xmlns="http://www.openarchives.org/OAI/2.0/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd"><responseDate>2026-11-22T10:02:01Z</responseDate><request verb="GetRecord" identifier="oai:ecommons.cornell.edu:1813/59273" metadataPrefix="dim">https://ecommons.cornell.edu/server/oai/request</request><GetRecord><record><header><identifier>oai:ecommons.cornell.edu:1813/59273</identifier><datestamp>2026-05-15T19:49:01Z</datestamp><setSpec>com_1813_35</setSpec><setSpec>col_1813_47</setSpec></header><metadata><dim:dim xmlns:dim="http://www.dspace.org/xmlns/dspace/dim" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:doc="http://www.lyncode.com/xoai" xsi:schemaLocation="http://www.dspace.org/xmlns/dspace/dim http://www.dspace.org/schema/dim.xsd">
   <dim:field mdschema="dc" element="contributor" qualifier="author">Yuan, Yang</dim:field>
   <dim:field mdschema="dc" element="contributor" qualifier="chair">Kleinberg, Robert David</dim:field>
   <dim:field mdschema="dc" element="contributor" qualifier="committeeMember">Kozen, Dexter Campbell</dim:field>
   <dim:field mdschema="dc" element="contributor" qualifier="committeeMember">Joachims, Thorsten</dim:field>
   <dim:field mdschema="dc" element="date" qualifier="accessioned">2018-10-23T13:20:30Z</dim:field>
   <dim:field mdschema="dc" element="date" qualifier="available">2018-10-23T13:20:30Z</dim:field>
   <dim:field mdschema="dc" element="date" qualifier="issued">2018-05-30</dim:field>
   <dim:field mdschema="dc" element="identifier" qualifier="other">ProQuest Submission ID: 10809</dim:field>
   <dim:field mdschema="dc" element="identifier" qualifier="other">ProQuest Publication ID: 10814646</dim:field>
   <dim:field mdschema="dc" element="identifier" qualifier="uri">https://hdl.handle.net/1813/59273</dim:field>
   <dim:field mdschema="dc" element="identifier" qualifier="doi">https://doi.org/10.7298/X40R9MMS</dim:field>
   <dim:field mdschema="dc" element="identifier" qualifier="bibid">10489357</dim:field>
   <dim:field mdschema="dc" element="description" qualifier="abstract">Machine learning has become one of the most exciting research areas in the world, with various applications. However, there exists a noticeable gap between theory and practice. On one hand, a simple algorithm like stochastic gradient descent (SGD) works very well in practice, without satisfactory theoretical explanations. On the other hand, the algorithms analyzed in the theoretical machine learning literature, although with solid guarantees, tend to be less efficient compared with the techniques widely used in practice, which are usually hand tuned or ad hoc based on intuition.  This dissertation is about bridging the gap between theory and practice from two directions. The first direction is "practice to theory", i.e., to explain and analyze the existing algorithms and empirical observations in machine learning. Along this direction, we provide  sufficient conditions for SGD to escape saddle points and local minima, as well as SGD dynamics analysis for the two-layer neural network with ReLU activation.  The other direction is "theory to practice", i.e., using theoretical tools to obtain new, better and practical algorithms. Along this direction, we introduce a new algorithm Harmonica that uses Fourier analysis and compressed sensing for tuning hyperparameters. Harmonica supports parallel sampling and works well for tuning neural networks with more than 30 hyperparameters.</dim:field>
   <dim:field mdschema="dc" element="language" qualifier="iso">en_US</dim:field>
   <dim:field mdschema="dc" element="rights" lang="*">Attribution 4.0 International</dim:field>
   <dim:field mdschema="dc" element="rights" qualifier="uri" lang="*">https://creativecommons.org/licenses/by/4.0/</dim:field>
   <dim:field mdschema="dc" element="subject">Artificial intelligence</dim:field>
   <dim:field mdschema="dc" element="subject">Computer science</dim:field>
   <dim:field mdschema="dc" element="subject">Hyperparameter tuning</dim:field>
   <dim:field mdschema="dc" element="subject">Local minima</dim:field>
   <dim:field mdschema="dc" element="subject">Non-convex optimization</dim:field>
   <dim:field mdschema="dc" element="subject">Saddle points</dim:field>
   <dim:field mdschema="dc" element="subject">Stochastic Gradient Descent</dim:field>
   <dim:field mdschema="dc" element="subject">machine learning</dim:field>
   <dim:field mdschema="dc" element="title">PROVABLE AND PRACTICAL ALGORITHMS FOR NON-CONVEX PROBLEMS IN MACHINE LEARNING</dim:field>
   <dim:field mdschema="dc" element="type">dissertation or thesis</dim:field>
   <dim:field mdschema="dc" element="format" qualifier="mimetype">application/pdf</dim:field>
   <dim:field mdschema="thesis" element="degree" qualifier="discipline">Computer Science</dim:field>
   <dim:field mdschema="thesis" element="degree" qualifier="grantor">Cornell University</dim:field>
   <dim:field mdschema="thesis" element="degree" qualifier="level">Doctor of Philosophy</dim:field>
   <dim:field mdschema="thesis" element="degree" qualifier="name">Ph. D., Computer Science</dim:field>
   <dim:field mdschema="dcterms" element="license">https://hdl.handle.net/1813/59810</dim:field>
   <dim:field mdschema="dspace" element="entity" qualifier="type">Publication</dim:field>
   <dim:field mdschema="cris" element="virtual" qualifier="collection" authority="https://cornell-ecommons.eks.prod.4science.cloud/handle/1813/47" confidence="600">Cornell Theses and Dissertations</dim:field>
   <dim:field mdschema="cris" element="virtual" qualifier="author">Yuan, Yang</dim:field>
   <dim:field mdschema="cris" element="virtualsource" qualifier="collection">5893a6ea-7af3-41d7-abc6-04bcd26ab5df</dim:field>
   <dim:field mdschema="others" element="access-status">open.access</dim:field>
   <dim:field mdschema="others" element="access-status">open.access</dim:field>
   <dim:field mdschema="cerif" element="openaire" authority="" confidence="-1">&lt;Publication xmlns="https://www.openaire.eu/cerif-profile/1.1/" id="969dcd28-28ec-4b01-be4b-d1aaf3320ff8">
	&lt;Type xmlns="https://www.openaire.eu/cerif-profile/vocab/COAR_Publication_Types">http://purl.org/coar/resource_type/c_1843&lt;/Type>
	&lt;Language>en_US&lt;/Language>
   	&lt;Title>PROVABLE AND PRACTICAL ALGORITHMS FOR NON-CONVEX PROBLEMS IN MACHINE LEARNING&lt;/Title>
   	&lt;PublishedIn>
    	&lt;Publication>
      	&lt;/Publication>
   	&lt;/PublishedIn>
   	&lt;PublicationDate>2018-05-30&lt;/PublicationDate>
   	&lt;DOI>https://doi.org/10.7298/X40R9MMS&lt;/DOI>
   	&lt;Authors>
      	&lt;Author>
        	&lt;DisplayName>Yuan, Yang&lt;/DisplayName>
         	&lt;Affiliation>
         		&lt;OrgUnit>
         		&lt;/OrgUnit>
         	&lt;/Affiliation>
      	&lt;/Author>
	&lt;/Authors>
   	&lt;Editors>
	&lt;/Editors>
    &lt;Publishers>
        &lt;Publisher>
            &lt;OrgUnit />
        &lt;/Publisher>
    &lt;/Publishers>
    &lt;License>https://creativecommons.org/licenses/by/4.0/&lt;/License>
    &lt;Keyword>Artificial intelligence&lt;/Keyword>
    &lt;Keyword>Computer science&lt;/Keyword>
    &lt;Keyword>Hyperparameter tuning&lt;/Keyword>
    &lt;Keyword>Local minima&lt;/Keyword>
    &lt;Keyword>Non-convex optimization&lt;/Keyword>
    &lt;Keyword>Saddle points&lt;/Keyword>
    &lt;Keyword>Stochastic Gradient Descent&lt;/Keyword>
    &lt;Keyword>machine learning&lt;/Keyword>
   	&lt;Abstract>Machine learning has become one of the most exciting research areas in the world, with various applications. However, there exists a noticeable gap between theory and practice. On one hand, a simple algorithm like stochastic gradient descent (SGD) works very well in practice, without satisfactory theoretical explanations. On the other hand, the algorithms analyzed in the theoretical machine learning literature, although with solid guarantees, tend to be less efficient compared with the techniques widely used in practice, which are usually hand tuned or ad hoc based on intuition.  This dissertation is about bridging the gap between theory and practice from two directions. The first direction is &amp;quot;practice to theory&amp;quot;, i.e., to explain and analyze the existing algorithms and empirical observations in machine learning. Along this direction, we provide  sufficient conditions for SGD to escape saddle points and local minima, as well as SGD dynamics analysis for the two-layer neural network with ReLU activation.  The other direction is &amp;quot;theory to practice&amp;quot;, i.e., using theoretical tools to obtain new, better and practical algorithms. Along this direction, we introduce a new algorithm Harmonica that uses Fourier analysis and compressed sensing for tuning hyperparameters. Harmonica supports parallel sampling and works well for tuning neural networks with more than 30 hyperparameters.&lt;/Abstract>
	&lt;Access xmlns="http://purl.org/coar/access_right" 
    >
    &lt;/Access>
&lt;/Publication>
</dim:field>
</dim:dim>
</metadata></record></GetRecord></OAI-PMH>