Light and Motion: Modeling and Visualizing How Scenes Change Over Time

Abstract

The visible world changes in many ways as time passes. Image and video data capturing these changes is plentiful, but analyzing such data presents many challenges. This thesis presents a set of techniques for modeling, analyzing, and visualizing two key aspects of visual change over time: illumination and motion. The first part discusses the challenging problem of illumination modeling in unstructured photo collections, proposing an approach for shadow detection and sun direction estimation that works on untagged photo collections downloaded from the internet. The second part of the thesis begins by presenting a state-of-the-art method for using motion cues to segment moving foreground objects from static backgrounds. Then, we pose the new problem of how to visualize motions at multiple timescales---from seconds to minutes to hours---in fixed-viewpoint timelapse videos. Two approaches to solving this problem are presented, one based on optimization-driven motion normalization and the other building on the motion segmentation algorithm to segment different timescale videos into foreground layers. The thesis concludes with a discussion of future work, including how the study of illumination and motion might come together to enable further progress in both computer vision and visualization applications.