JavaScript is disabled for your browser. Some features of this site may not work without it.
Robust and Scalable Spectral Topic Modeling for Large Vocabularies

Author
Cho, Sungjun
Abstract
Across many data domains, co-occurrence statistics about the joint appearance of objects are powerfully informative. In topic modeling, spectral methods can provably learn low-dimensional latent topics from easily-collected word co-occurrence statistics unlike likelihood-based methods that require exhaustive reiterations through the corpus. However, spectral methods suffer from two major drawbacks: the quality of learned topics deteriorates drastically when the empirical data does not follow the generative model, and the co-occurrence statistics itself grows to an intractable size when working with large vocabularies. This thesis is an attempt to overcome these drawbacks by developing a scalable and robust spectral topic inference framework based on Joint Stochastic Matrix Factorization. First, we provide theoretical foundations of spectral topic inference as well as step-wise algorithmic implementations of our anchor-based approach that can learn quality topics despite model-data mismatch. We then scale towards larger vocabularies by operating solely on compressed low-rank representations of co-occurrence statistics, keeping the overall cost linear with respect to the vocabulary size. Quantitative and qualitative experiments on various datasets not only demonstrate our framework's consistency and efficiency in inferring high-quality topics, but also introduce improvements in interpretability of the individual topics.
Description
56 pages
Date Issued
2020-05Subject
Natural Language Processing; Nonlinear Dimensionality Reduction; Spectral Methods; Unsupervised Learning
Committee Chair
Bindel, David
Committee Member
Mimno, David
Degree Discipline
Computer Science
Degree Name
M.S., Computer Science
Degree Level
Master of Science
Type
dissertation or thesis