Enhancing 3D Perception with Unlabeled Repeated Historical Data for Autonomous Vehicles

Other Titles



The evolution of autonomous vehicles is advancing rapidly, promising a radical shift in our future mobility. The cornerstone of building a reliable autonomous vehicle hinges on developing a robust perception system capable of accurately perceiving the vehicle's surroundings, primarily through the detection of other road participants. The current predominant approach relies heavily on supervised learning, requiring a substantial quantity of costly labeled data, and employs expensive 3D sensors such as LiDARs. This approach, while comprehensive, has intrinsic limitations: disambiguating small or distant objects becomes challenging due to the sparse nature of LiDAR point clouds. Furthermore, the supervised perception system is sensitive to domain differences when deployed in diverse environments, and cost-effective camera-only alternatives fail to achieve the accuracy of their LiDAR-based counterparts. These issues necessitate a critical reevaluation of the conventional methodology for 3D perception in autonomous vehicles. In this thesis, we investigated a novel yet highly practical strategy that could significantly mitigate these concerns — leveraging past visit data from the same location. Previous approaches tend to treat each scene as a new, unknown entity, ignoring the potential wealth of information from earlier traversals of identical routes. In reality, our daily commutes often involve repetitive journeys: to workplaces, schools, stores, and social gatherings. Even when navigating an unfamiliar route, we are typically retracing the paths of preceding drivers. We demonstrated that this simple shift in data perspective can yield substantial benefits: 1) Hindsight: a significant, consistent improvement in the performance of LiDAR-based 3D object detection models, 2) Rote-DA: a pronounced reduction in performance degradation of 3D perception models due to domain differences, and 3) AsyncDepth: a consistent improvement in camera-based 3D object detection models when supplemented with past LiDAR data. These promising results signify that historical data, a rich trove of information, can significantly contribute to improving 3D perception for safer autonomous driving. We also discussed potential future research directions at the conclusion of this thesis.

Journal / Series

Volume & Issue


136 pages


Date Issued




Autonomous Vehicle; Computer Vision; Historical Data; Machine Learning; Perception


Effective Date

Expiration Date




Union Local


Number of Workers

Committee Chair

Weinberger, Kilian

Committee Co-Chair

Committee Member

Hariharan, Bharath
Sun, Wen
Bindel, David

Degree Discipline

Computer Science

Degree Name

Ph. D., Computer Science

Degree Level

Doctor of Philosophy

Related Version

Related DOI

Related To

Related Part

Based on Related Item

Has Other Format(s)

Part of Related Item

Related To

Related Publication(s)

Link(s) to Related Publication(s)


Link(s) to Reference(s)

Previously Published As

Government Document




Other Identifiers


Attribution 4.0 International


dissertation or thesis

Accessibility Feature

Accessibility Hazard

Accessibility Summary

Link(s) to Catalog Record