3-D Head Tracking via Invariant Keypoint Learning
- 9 March 2012
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 22 (8), 1113-1126
- https://doi.org/10.1109/tcsvt.2012.2190474
Abstract
Keypoint matching is a standard tool to solve the correspondence problem in vision applications. However, in 3-D face tracking, this approach is often deficient because the human face complexities, together with its rich viewpoint, nonrigid expression, and lighting variations in typical applications, can cause many variations impossible to handle by existing keypoint detectors and descriptors. In this paper, we propose a new approach to tailor keypoint matching to track the 3-D pose of the user head in a video stream. The core idea is to learn keypoints that are explicitly invariant to these challenging transformations. First, we select keypoints that are stable under randomly drawn small viewpoints, nonrigid deformations, and illumination changes. Then, we treat keypoint descriptor learning at different large angles as an incremental scheme to learn discriminative descriptors. At matching time, to reduce the ratio of outlier correspondences, we use second-order color information to prune keypoints unlikely to lie on the face. Moreover, we integrate optical flow correspondences in an adaptive way to remove motion jitter efficiently. Extensive experiments show that the proposed approach can lead to fast, robust, and accurate 3-D head tracking results even under very challenging scenarios.Keywords
This publication has 39 references indexed in Scilit:
- Monocular head pose estimation using generalized adaptive view-based appearance modelImage and Vision Computing, 2010
- Fast Keypoint Recognition Using Random FernsIeee Transactions On Pattern Analysis and Machine Intelligence, 2009
- HyHOPE: Hybrid Head Orientation and Position Estimation for vision-based driver head trackingPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2008
- Re-thinking non-rigid structure from motionPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2008
- Accurate Non-Iterative O(n) Solution to the PnP ProblemPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2007
- Scale & Affine Invariant Interest Point DetectorsInternational Journal of Computer Vision, 2004
- Head pose estimation by nonlinear manifold learningPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2004
- Example-based head trackingPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- Facial feature extraction and pose determinationPattern Recognition, 2000
- Recognition of human head orientation based on artificial neural networksIEEE Transactions on Neural Networks, 1998