Self-driving technology is a rapidly advancing field with the potential to revolutionize transportation, enhance road safety, and improve urban mobility. The key value of autonomous vehicles lies in their ability to optimize traffic flow and enhance planning accuracy. Current research primarily focuses on areas such as path planning, trajectory prediction, and vehicle perception, with significant progress of deep learning algorithms. However, numerous challenges continue to hinder the widespread adoption of autonomous vehicles. This dissertation addresses three critical problems in this domain: vehicle perception, path planning, and trajectory prediction. The first focus of this dissertation is improving vehicle perception for downstream tasks in autonomous driving. Two key contributions are made in this area: • Amodal Road Segmentation-Amodal perception refers to the ability to perceive the full structure of objects even when they are partially occluded, drawing from both visible and inferred information. This capability is essential for self-driving cars, as it helps detect occluded objects like vehicles, pedestrians, and road boundaries in complex driving conditions, such as poor weather or traffic jams. These insights enhance downstream tasks like path planning, ultimately reducing collision risks. Despite its potential, amodal segmentation remains underexplored due to a lack of labeled data. To tackle this, we propose a novel algorithm called Mixture Pooling as Inpainting (MFI), which builds on previous work (SFI) to improve the accuracy of amodal road segmentation. • Coarsely Aligned Image Translation under Adverse Weather Conditions- Autonomous vehicles face significant challenges when operating in adverse weather, such as snow. This work focuses on transforming sensor inputs (e.g., images) captured in harsh conditions to resemble those captured in favorable conditions, improving downstream tasks like semantic segmentation. Most existing approaches treat this as an unpaired image- to-image translation problem due to the lack of perfectly aligned datasets. However, leveraging coarsely aligned images—captured by vehicles following the same routes daily—provides valuable context. We propose a novel training objective using these coarsely aligned image pairs, resulting in improved translation quality and enhanced downstream tasks, including semantic segmentation, monocular depth estimation, and visual localization. Our method outperforms state-of-the-art unpaired image translation approaches in both image quality and downstream task performance. The second focus of this dissertation is path planning in occluded, unknown spaces. Traditional methods rely on graph- and sampling-based approaches but often plan only within known spaces, limiting speed and navigability, or assume unknown spaces are free, increasing collision risk. Real-world driving frequently involves occluding objects such as vehicles, buildings, trees, and fences, especially in cluttered environments. To address this, we propose an inpainting model that completes sparse, occluded semantic lidar point clouds, allowing for dynamic path planning through occluded areas. Using a car’s lidar data with real-time occlusions, we demonstrate that our approach enables the planning of longer, more flexible paths with additional maneuver options compared to methods without inpainting. The final focus of this dissertation is vision-based trajectory prediction. Predicting the trajectories of agents (e.g., vehicles, cyclists, pedestrians) in complex environments is crucial for safe autonomous navigation. Agents’ decisions are often influenced by factors such as interactions with other agents, weather, and traffic rules. State-of-the-art methods rely heavily on high-definition (HD) or semantic maps, but these approaches fail to account for unpredictable factors like adverse weather and are limited by the difficulty of obtaining detailed maps. We propose a flexible graph-based trajectory prediction model that uses only image-based environmental cues, eliminating the need for expensive map information. Our experimental results demonstrate robust performance, surpassing state-of-the-art methods in complex environments where map-based approaches struggle, such as those affected by diverse weather conditions.