This thesis presents a compendium of three studies that introduce novel models and estimation strategies aimed at achieving high predictive performance while retaining the economic interpretability necessary for inference in discrete choice modeling. The first chapter introduces a Hierarchical Nearest Neighbor Gaussian Process (HNNGP) model for discrete choice analysis that accounts for spatially correlated unobserved effects across individuals--addressing a key limitation of standard models that assume independence. By embedding Gaussian Processes within the utility specification, the model retains structural interpretability while flexibly capturing spatial dependencies. Applied to commuting mode choices in New York City, the HNNGP outperforms standard logit and spatial probit models in both binary and multinomial settings, yielding more accurate predictions and credible estimates of the value of travel time savings. The second chapter presents a novel Graph Convolutional Neural Network (GCNN) architecture that incorporates peer and network effects into discrete choice modeling. Unlike most traditional approaches that ignore social influence, or deep learning models that lack interpretability, this GCNN framework balances flexibility, predictive performance, and economic interpretability. Applied to commuting mode choice data in New York City and U.S. election data, the model outperforms both standard discrete choice models and general-purpose deep learning models, while enabling meaningful economic inference. The third chapter develops a deep learning architecture designed to integrate with approximate Bayesian inference methods such as Stochastic Gradient Langevin Dynamics (SGLD). The model balances predictive performance with interpretability and uncertainty quantification, addressing key limitations of standard deep learning in economic applications. It adapts to data availability by collapsing to expert-informed hypotheses when data is sparse and capturing complex nonlinearities when sufficient data is available. The proposed training framework guides the model toward regions in the parameter space where the informed component performs well, updating the more flexible component only when necessary to improve predictive accuracy. The approach is validated through simulation and empirical studies on commuting mode choices in New York City and the popular Swiss train choice dataset. Overall, this thesis offers not only novel model architectures and staged estimation procedures but also guiding principles for the design of deep learning and Gaussian process models that combine strong predictive performance with behaviorally sound inference. Additionally, as part of this work, I constructed a novel mode choice dataset for New York City, populated with revealed preference data and augmented with trip costs and travel times obtained from the Google Maps API.