Combining Deep Learning with Reasoning: from Mapping Species to Solving Games and Crystal Structures

Other Titles



Artificial Intelligence (AI) aims to develop intelligent systems, inspired in part by human intelligence. AI systems are now performing at human and even superhuman levels on a range of tasks such as image identification, face, and speech recognition. These major recent AI achievements have been driven largely by advances in supervised deep learning, which requires large labeled datasets to supervise model training. In contrast, humans often solve complex tasks using far fewer data by amplifying intuitive pattern recognition with meticulous reasoning that uses prior knowledge, a hybrid strategy that is challenging for machines to emulate. In this thesis, we focus on integrating prior knowledge reasoning into deep learning via an interpretable latent space. When the prior knowledge is sufficiently rich, as is common in many scientific applications, we can supersede traditional example-based supervised learning and compensate for a dearth of labeled data by exploiting prior knowledge and magnifying it with logical and constraint reasoning seamlessly integrated into neural network optimization. We first illustrate this idea in the context of supervised learning on multi-label classification --- in particular, on joint species distribution modeling, where we propose the Deep Multivariate Probit Model (DMVP) to uncover species interactions and habitat associations via the interpretable latent space for the entire North American avifauna and accelerate the learning by an order of magnitude using prior knowledge of the low-rank structure of species interaction. Next, we demonstrate the capability of this approach on unsupervised tasks with rich prior knowledge via a novel framework called Deep Reasoning Networks (DRNets). For variants of visual Sudoku games, DRNets outperforms supervised state-of-the-art methods in an unsupervised manner. In materials science, DRNets surpasses previous approaches and the capability of experts on crystal-structure phase mapping, unraveling the Bi-Cu-V oxide phase diagram, and enabling the discovery of solar-fuels materials. In the future, we plan to further develop, adapt, and customize Deep Reasoning Networks to effectively solve a variety of tasks that require a combination of pattern recognition, reasoning, and learning, which are pervasive in science and other application domains.

Journal / Series

Volume & Issue


199 pages


Date Issued




Deep Learning; Prior Knowledge; Reasoning


Effective Date

Expiration Date




Union Local


Number of Workers

Committee Chair

Gomes, Carla P.

Committee Co-Chair

Committee Member

Joachims, Thorsten
Kuleshov, Volodymyr
Selman, Bart

Degree Discipline

Computer Science

Degree Name

Ph. D., Computer Science

Degree Level

Doctor of Philosophy

Related Version

Related DOI

Related To

Related Part

Based on Related Item

Has Other Format(s)

Part of Related Item

Related To

Related Publication(s)

Link(s) to Related Publication(s)


Link(s) to Reference(s)

Previously Published As

Government Document




Other Identifiers


Attribution-NonCommercial-ShareAlike 4.0 International


dissertation or thesis

Accessibility Feature

Accessibility Hazard

Accessibility Summary

Link(s) to Catalog Record