Fine-grained wireless perception for human tracking
In the network and mobile system community, wireless perception is an active field of research that uses wireless signals for motion tracking, such as ultrasound, mmWave, and WiFi. It enables applications like gesture recognition, breath rate and heart rate detection, intrusion detection, mixed reality (MR) interaction, etc.This dissertation focus on wireless perception for human tracking, addressing key challenges in accuracy, practicality, and scalability towards fine-grained tracking using deep learning. First, we develop an acoustic sensing system, Beyond-Voice, that enables commodity devices to track 3D hand poses with millimeter precision using only built-in microphones and speakers. Next, we aim to dive deep into the individual building blocks in such wireless perception systems and develop methods that can be reused. To elaborate, we tackle the interference problem between acoustic sensing and concurrent audio playback with CoPlay, a deep learning–based optimization method that preserves both sensing performance and media quality. To reduce costly data collection and improve model generalization, we propose WixUp, a audio-agnostic data augmentation framework applicable to both acoustic and mmWave sensing, also enabling unsupervised domain adaptation. Finally, I extend the sensing range to egocentric full-body tracking in mixed reality headset with BodyWave, a mmWave radar–based system that overcomes camera occlusion limitations while maintaining competitive accuracy. Together, these contributions bring wireless perception closer to practical, robust, and privacy-preserving human tracking in real-world environments.