An Investigation of Underwater Sensor Fusion and Human-Robot Interaction Approaches
Imagine a world where scuba divers are never alone in the vast blue expanse of the ocean—where an intelligent robotic companion swims alongside them, understanding their movements, anticipating their needs, and responding to their gestures. This thesis lays the groundwork for such a "scuba buddy" through two complementary research components. First, we address the fundamental challenge of accurately tracking divers in underwater environments, where conventional sensing technologies face significant limitations. We present a novel sensor fusion framework that intelligently combines data from stereo cameras and motion capture systems to reconstruct precise diver trajectories despite the challenges of light refraction, turbidity, and temporal desynchronization. Our approach introduces a robust algorithm that successfully compensates for temporal and spatial offsets in motion capture data, achieving trajectory reconstruction with significantly reduced error margins compared to existing methods. These reconstructed trajectories serve an important purpose, that is providing critical data for solving unknown hydrodynamic coefficients in our diver hydrodynamic model—essentially teaching our robotic system how humans move underwater. Building upon this spatial awareness foundation, the second component of our research focuses on underwater human-robot interaction through visual communication. We have developed a machine learning computer vision model that can recognize diver hand gestures through stereo video inputs, overcoming the visual distortion and variable lighting conditions inherent to underwater environments. This customizable gesture recognition system allows divers to communicate intuitively with the robotic companion using a personalized gesture dictionary tailored to specific mission requirements. The integration of these two systems—precise spatial tracking and gesture-based communication—represents a significant advancement toward autonomous underwater vehicles capable of meaningful collaboration with human divers. By enabling robots to both follow divers through complex underwater environments and understand their communicative intent, this research contributes essential components to the realization of an intelligent scuba buddy that can assist with navigation, tool retrieval, photography, sample collection, and potentially life-saving interventions in the challenging domain of underwater exploration.