Cornell University
Library
Cornell UniversityLibrary

eCommons

Help
Log In(current)
  1. Home
  2. Cornell Computing and Information Science
  3. Computing and Information Science
  4. Computing and Information Science Research
  5. Data from: WatchHand: Enabling Continuous Hand Pose Tracking On Off-the-Shelf Smartwatches

Data from: WatchHand: Enabling Continuous Hand Pose Tracking On Off-the-Shelf Smartwatches

File(s)
Lee_CIS_README_2026.md (23.48 KB)
Study1_sub05-08.zip (11.62 GB)
Study4_sub05-08.zip (10.82 GB)
Study1_sub09-12.zip (10.96 GB)
Study1_sub21-24.zip (10.3 GB)
  View More
Permanent Link(s)
https://doi.org/10.7298/qf1v-j805
https://hdl.handle.net/1813/121443
Collections
Computing and Information Science Research
Author
Kim, Jiwan
Lee, Chi-Jung
Jung, Hohurn
Yu, Tianhong
Zhang, Ruidong
Oakley, Ian
Zhang, Cheng
Abstract

These files contain data supporting all results reported in Kim et. al. WatchHand: Enabling Continuous Hand Pose Tracking On Off-the-Shelf Smartwatches. Tracking hand poses on wrist-wearables enables rich, expressive interactions, yet remains unavailable on commercial smartwatches, as prior implementations rely on external sensors or custom hardware, limiting their real-world applicability. To address this, we present WatchHand, the first continuous 3D hand pose tracking system implemented on off-the-shelf smartwatches using only their built-in speaker and microphone. WatchHand emits inaudible frequency-modulated continuous waves and captures their reflections from the hand. These acoustic signals are processed by a deep-learning model that estimates 3D hand poses for 20 finger joints. We evaluate WatchHand across diverse real-world conditions -- multiple smartwatch models, wearing-hands, body postures, noise conditions, pose-variation protocols -- and achieve a mean per-joint position error of 7.87 mm in cross-session tests with device remounting. Although performance drops for unseen users or gestures, the model adapts effectively with lightweight fine-tuning on small amounts of data. Overall, WatchHand lowers the barrier to smartwatch-based hand tracking by eliminating additional hardware while enabling robust, always-available interactions on millions of existing devices.

Description
Please cite as:
Jiwan Kim, Chi-Jung Lee, Hohurn Jung, Tianhong Yu, Ruidong Zhang, Ian Oakley, Cheng Zhang. (2026) Data from: WatchHand: Enabling Continuous Hand Pose Tracking On Off-the-Shelf Smartwatches [Data set] Cornell University Library eCommons Repository. https://doi.org/10.7298/qf1v-j805
Sponsorship
This project was supported by the National Science Foundation under Grant No.2239569 (NSF CAREER Award) and the IITP (Institute of Information & Communications Technology Planning & Evaluation) - ITRC (Information Technology Research Center) grant funded by the Korea government (Ministry of Science and ICT) (IITP-2026-RS2024-00436398).
Date Issued
2026
Keywords
active acoustic sensing
•
wearable
•
UbiComp
•
HCI
Related Publication(s)
Kim, J., Lee, C.-J., Jung, H., Catherine Yu, T., Zhang, R., Oakley, I., & Zhang, C. (2026). WatchHand: Enabling Continuous Hand Pose Tracking On Off-the-Shelf Smartwatches. Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems, 1–21. https://doi.org/10.1145/3772318.3790932
‌
Link(s) to Related Publication(s)
https://doi.org/10.1145/3772318.3790932
Rights
Attribution 4.0 International
Rights URI
https://creativecommons.org/licenses/by/4.0/
Type
dataset

Site Statistics | Help

About eCommons | Policies | Terms of use | Contact Us

copyright © 2002-2026 Cornell University Library | Privacy | Web Accessibility Assistance