Probabilistic inference and reasoning is applied to two major application areas: HVAC controls in buildings and autonomous vehicle perception. Although the physical domains differ vastly, across both applications the presented novel contributions share real-time inference of stochastic systems for improved control capability and performance. Besides performing simple state estimation, Kalman Filters in both applications are extended for model inference-estimating thermal model parameters and disturbances in buildings and dynamic object classification for perception in autonomous vehicles. Part one of this study proposes a general, scalable method to learn controloriented thermal models of buildings that could enable wide-scale deployment of cost-effective predictive controls. An Unscented Kalman Filter augmented for parameter and disturbance estimation is shown to accurately learn and predict a building's thermal response. By leveraging building topology and measurement data, the filter quickly learns parameters of a thermal network during periods of known or constrained loads and then characterizes unknown loads in order to provide accurate 24+ hour energy predictions. Performance was validated with EnergyPlus simulation data across a year-long study of a passive building. The method is extended to multi-zone actively controlled buildings by using the controller to excite unknown portions of the building's dynamics. A simulation study demonstrates self-excitation improves model estimation. Formalization of parameterization, disturbance estimation, and self-excitation routines is shown with an observability analysis. Comparing against a baseline thermostat controller, a Model Predictive Control (MPC) framework, which anticipates weather uncertainty and time-varying temperature set-points, is shown to improve energy savings and occupant comfort. Part two of this study presents a novel probabilistic perception algorithm as a real-time joint solution to data association, object tracking, and object classification for an autonomous ground vehicle (AGV) in all-weather conditions. The presented algorithm extends a Rao-Blackwellized Particle Filter originally built for Cornell's AGV for the DARPA Urban Challenge (DUC) to include multiple model tracking for classification. Additionally a state-of-the-art vision detection algorithm that includes heading information for AGV applications was implemented. Cornell's AGV from the DUC was upgraded and used to experimentally examine if and how state-of-the-art vision algorithms can complement or replace lidar and radar sensors. Sensor and algorithm performance in adverse weather and lighting conditions is tested. Experimental evaluation demonstrates that sensor diversity with a joint probabilistic perception algorithm provides robust all-weather data association, tracking, and classification.