With the rise of augmented and virtual reality (AR/VR) systems and deployments in the real world, such as the Meta Ray-Bans, Apple Vision Pro, and Oculus amongst others, there is an increased need to not only display information to users using a head-mounted screen, but to also understand user actions to enable both seamless input for short-term application usage and seamless logging of activities for "past memory". As a result, computer vision and machine perception have extended beyond internet images and towards the world of "egocentric" (first-person) sensing, where cameras and other sensors are located directly on the human body rather than on external parts of the environment and are facing outwards. However, the state-of-the-art in egocentric perception and understanding focuses primarily on detecting and recognizing short clips of human actions from egocentric videos. This is both compute-intensive and unable to capture the long-term trajectory of objects throughout a single video, making long-term behavioral understanding hard to capture. This thesis presents both ActSonic, a novel alternative sensing modality using ultrasonic sensors on eyeglasses for efficient and privacy-sensitive egocentric activity recognition via human motion, and an evaluation of how existing trackers and segmentation systems trained on third-person views fare against the challenging task of tracking objects from a first-person view.