Tracking loose-limbed people

Abstract

We pose the problem of 3D human tracking as one of in- ference in a graphical model. Unlike traditional kinematic tree representations, our model of the body is a collection of loosely-connected limbs. Conditional probabilities relating the 3D pose of connected limbs are learned from motion- captured training data. Similarly, we learn probabilistic models for the temporal evolution of each limb (forward and backward in time). Human pose and motion estimation is then solved with non-parametric belief propagation us- ing a variation of particle filtering that can be applied over a general loopy graph. The loose-limbed model and decen- tralized graph structure facilitate the use of low-level vi- sual cues. We adopt simple limb and head detectors to pro- vide "bottom-up" information that is incorporated into the inference process at every time-step; these detectors per- mit automatic initialization and aid recovery from transient tracking failures. We illustrate the method by automatically tracking a walking person in video imagery using four cal- ibrated cameras. Our experimental apparatus includes a marker-based motion capture system aligned with the coor- dinate frame of the calibrated cameras with which we quan- titatively evaluate the accuracy of our 3D person tracker.