Artificial Intelligence (AI) aims to develop intelligent systems, inspired in part by human intelligence. AI systems are now performing at human and even superhuman levels on a range of tasks such as image identification, face, and speech recognition. These major recent AI achievements have been driven largely by advances in supervised deep learning, which requires large labeled datasets to supervise model training. In contrast, humans often solve complex tasks using far fewer data by amplifying intuitive pattern recognition with meticulous reasoning that uses prior knowledge, a hybrid strategy that is challenging for machines to emulate. In this thesis, we focus on integrating prior knowledge reasoning into deep learning via an interpretable latent space. When the prior knowledge is sufficiently rich, as is common in many scientific applications, we can supersede traditional example-based supervised learning and compensate for a dearth of labeled data by exploiting prior knowledge and magnifying it with logical and constraint reasoning seamlessly integrated into neural network optimization. We first illustrate this idea in the context of supervised learning on multi-label classification --- in particular, on joint species distribution modeling, where we propose the Deep Multivariate Probit Model (DMVP) to uncover species interactions and habitat associations via the interpretable latent space for the entire North American avifauna and accelerate the learning by an order of magnitude using prior knowledge of the low-rank structure of species interaction. Next, we demonstrate the capability of this approach on unsupervised tasks with rich prior knowledge via a novel framework called Deep Reasoning Networks (DRNets). For variants of visual Sudoku games, DRNets outperforms supervised state-of-the-art methods in an unsupervised manner. In materials science, DRNets surpasses previous approaches and the capability of experts on crystal-structure phase mapping, unraveling the Bi-Cu-V oxide phase diagram, and enabling the discovery of solar-fuels materials. In the future, we plan to further develop, adapt, and customize Deep Reasoning Networks to effectively solve a variety of tasks that require a combination of pattern recognition, reasoning, and learning, which are pervasive in science and other application domains.