We consider the problem of tracking non-rigid objects moving in a complex scene. We describe a model-based tracking method, in which two-dimensional geometric models are used to localize an object in each frame of an image sequence. The basic idea is to decompose the image of a solid object moving in space into two components: a two-dimensional motion and a two-dimensional shape change. The motion component is factored out, and the shape change is represented by explicitly storing a sequence of two-dimensional models, one corresponding to each image frame. The major assumption underlying the method is that the two-dimensional shape of an object will change slowly from one frame to the next. There is no assumption, however, that the two-dimensional image motion in successive frames will be small. Thus, the method can track objects that move arbitrarily far in the image from one frame to the next.