Learning Deep Representations for Low-Resource Cross-Lingual Natural Language Processing

Abstract

Large-scale annotated datasets are an indispensable ingredient of modern Natural Language Processing (NLP) systems. Unfortunately, most labeled data is only available in a handful of languages; for the vast majority of human languages, few or no annotations exist to empower automated NLP technology. Cross-lingual transfer learning enables the training of NLP models using labeled data from other languages, which has become a viable technique for building NLP systems for a wider spectrum of world languages without the prohibitive need for data annotation. Existing methods for cross-lingual transfer learning, however, require cross-lingual resources (e.g. machine translation systems) to transfer models across languages. These methods are hence futile for many low-resource languages without such resources. This dissertation proposes a deep representation learning approach for low-resource cross-lingual transfer learning, and presents several models that (i) progressively remove the need for cross-lingual supervision, and (ii) go beyond the standard bilingual transfer case into the more realistic multilingual setting. By addressing key challenges in two important sub-problems, namely multilingual lexical representation and model transfer, the proposed models in this dissertation are able to transfer NLP models across multiple languages with no cross-lingual resources.