JavaScript is disabled for your browser. Some features of this site may not work without it.
Deep Sequential and Structural Neural Models of Compositionality

Author
Irsoy, Ozan
Abstract
Recent advances in deep learning have provided fruitful applications for natural language processing (NLP) tasks. One key advance was the invention of word vectors, representing every word in a dense, low-dimensional vector space. Even though word vectors provide very strong results for word level NLP tasks, producing appropriate representation for phrases and sentences is still an open research problem.
In this dissertation, we focus on compositional approaches to representation learning. In particular, we employ the notions of compositionality in which the sequence or structure information is utilized, via recurrent or recursive neural networks. We investigate the effectiveness of such approaches for specific natural language understanding tasks including opinion mining and sentiment analysis, and extend some of the approaches to provide better representation hierarchies. In particular, we propose two novel variants: bidirectional recursive neural networks, which are capable of producing context-dependent structural representations and deep recursive neural networks, which provide representation hierarchies in the structural setting. Additionally, we qualitatively investigate such models, and describe how they relate to alternative compositional approaches. Finally, we discuss challenges in interpretation and understanding of compositional neural models, propose simple tools for visualization, and perform exploratory analyses over features learned by such a model.
Date Issued
2017-01-30Subject
neural networks; Computer science; machine learning; natural language processing; Deep Learning
Committee Chair
Cardie, Claire T
Committee Member
Woodard, Dawn B.; Kleinberg, Robert David
Degree Discipline
Computer Science
Degree Name
Ph. D., Computer Science
Degree Level
Doctor of Philosophy
Rights
Attribution 4.0 International
Rights URI
Type
dissertation or thesis
Except where otherwise noted, this item's license is described as Attribution 4.0 International