Cornell University
Library
Cornell UniversityLibrary

eCommons

Help
Log In(current)
  1. Home
  2. Cornell University Graduate School
  3. Cornell Theses and Dissertations
  4. An Investigation of Underwater Sensor Fusion and Human-Robot Interaction Approaches

An Investigation of Underwater Sensor Fusion and Human-Robot Interaction Approaches

File(s)
He_cornell_0058O_12409.pdf (13.54 MB)
g7_clip3DLC_Resnet50_backwardApr16shuffle1_snapshot_040_p60_labeled.mp4 (41.44 MB)
g9_clip3DLC_Resnet50_backwardApr16shuffle1_snapshot_040_p60_labeled.mp4 (56.09 MB)
Permanent Link(s)
https://doi.org/10.7298/2mw7-nk92
https://hdl.handle.net/1813/121246
Collections
Cornell Theses and Dissertations
Author
He, Zilong
Abstract

Imagine a world where scuba divers are never alone in the vast blue expanse of the ocean—where an intelligent robotic companion swims alongside them, understanding their movements, anticipating their needs, and responding to their gestures. This thesis lays the groundwork for such a "scuba buddy" through two complementary research components. First, we address the fundamental challenge of accurately tracking divers in underwater environments, where conventional sensing technologies face significant limitations. We present a novel sensor fusion framework that intelligently combines data from stereo cameras and motion capture systems to reconstruct precise diver trajectories despite the challenges of light refraction, turbidity, and temporal desynchronization. Our approach introduces a robust algorithm that successfully compensates for temporal and spatial offsets in motion capture data, achieving trajectory reconstruction with significantly reduced error margins compared to existing methods. These reconstructed trajectories serve an important purpose, that is providing critical data for solving unknown hydrodynamic coefficients in our diver hydrodynamic model—essentially teaching our robotic system how humans move underwater. Building upon this spatial awareness foundation, the second component of our research focuses on underwater human-robot interaction through visual communication. We have developed a machine learning computer vision model that can recognize diver hand gestures through stereo video inputs, overcoming the visual distortion and variable lighting conditions inherent to underwater environments. This customizable gesture recognition system allows divers to communicate intuitively with the robotic companion using a personalized gesture dictionary tailored to specific mission requirements. The integration of these two systems—precise spatial tracking and gesture-based communication—represents a significant advancement toward autonomous underwater vehicles capable of meaningful collaboration with human divers. By enabling robots to both follow divers through complex underwater environments and understand their communicative intent, this research contributes essential components to the realization of an intelligent scuba buddy that can assist with navigation, tool retrieval, photography, sample collection, and potentially life-saving interventions in the challenging domain of underwater exploration.

Description
71 pages
Supplemental file(s) description: Validation Clip#2, Validation Clip#1.
Date Issued
2025-08
Keywords
Deep Learning
•
Human Robot Interaction
•
Sensor Fusion
Committee Chair
Ferrari, Silvia
Committee Member
MacMartin, Douglas
Degree Discipline
Mechanical Engineering
Degree Name
M.S., Mechanical Engineering
Degree Level
Master of Science
Rights
Attribution-ShareAlike 4.0 International
Rights URI
https://creativecommons.org/licenses/by-sa/4.0/
Type
dissertation or thesis

Site Statistics | Help

About eCommons | Policies | Terms of use | Contact Us

copyright © 2002-2026 Cornell University Library | Privacy | Web Accessibility Assistance