Tracking People by Learning Their Appearance
- 30 November 2006
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 29 (1), 65-81
- https://doi.org/10.1109/tpami.2007.250600
Abstract
An open vision problem is to automatically track the articulations of people from a video sequence. This problem is difficult because one needs to determine both the number of people in each frame and estimate their configurations. But, finding people and localizing their limbs is hard because people can move fast and unpredictably, can appear in a variety of poses and clothes, and are often surrounded by limb-like clutter. We develop a completely automatic system that works in two stages; it first builds a model of appearance of each person in a video and then it tracks by detecting those models in each frame ("tracking by model-building and detection"). We develop two algorithms that build models; one bottom-up approach groups together candidate body parts found throughout a sequence. We also describe a top-down approach that automatically builds people-models by detecting convenient key poses within a sequence. We finally show that building a discriminative model of appearance is quite helpful since it exploits structure in a background (without background-subtraction). We demonstrate the resulting tracker on hundreds of thousands of frames of unscripted indoor and outdoor activity, a feature-length film ("Run Lola Run"), and legacy sports footage (from the 2002 World Series and 1998 Winter Olympics). Experiments suggest that our system 1) can count distinct individuals, 2) can identify and track them, 3) can recover when it loses track, for example, if individuals are occluded or briefly leave the view, 4) can identify body configuration accurately, and 5) is not dependent on particular models of human motionKeywords
This publication has 30 references indexed in Scilit:
- Strike a Pose: Tracking People by Finding Stylized PosesPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2005
- Pictorial Structures for Object RecognitionInternational Journal of Computer Vision, 2005
- Nonparametric belief propagationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2003
- Finding and tracking people from the bottom upPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2003
- PAMPAS: real-valued graphical models for computer visionPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2003
- Articulated body motion capture by annealed particle filteringPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- Probabilistic Tracking with Exemplars in a Metric SpaceInternational Journal of Computer Vision, 2002
- Probabilistic Methods for Finding PeopleInternational Journal of Computer Vision, 2001
- CONDENSATION—Conditional Density Propagation for Visual TrackingInternational Journal of Computer Vision, 1998
- The Representation and Matching of Pictorial StructuresIEEE Transactions on Computers, 1973