3D shape regression for real-time facial animation
Top Cited Papers
- 21 July 2013
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Graphics
- Vol. 32 (4), 1-10
- https://doi.org/10.1145/2461912.2462012
Abstract
We present a real-time performance-driven facial animation system based on 3D shape regression. In this system, the 3D positions of facial landmark points are inferred by a regressor from 2D video frames of an ordinary web camera. From these 3D points, the pose and expressions of the face are recovered by fitting a user-specific blendshape model to them. The main technical contribution of this work is the 3D regression algorithm that learns an accurate, user-specific face alignment model from an easily acquired set of training data, generated from images of the user performing a sequence of predefined facial poses and expressions. Experiments show that our system can accurately recover 3D face shapes even for fast motions, non-frontal faces, and exaggerated expressions. In addition, some capacity to handle partial occlusions and changing lighting conditions is demonstrated.Keywords
Funding Information
- National Natural Science Foundation of China (61003145, 61272305)
- Ministry of Science and Technology of the People's Republic of China (2009CB320801)
This publication has 30 references indexed in Scilit:
- Realtime performance-based facial animationACM Transactions on Graphics, 2011
- High-quality passive facial performance capture using anchor framesACM Transactions on Graphics, 2011
- Leveraging motion capture and 3D scanning for high-fidelity facial performance acquisitionACM Transactions on Graphics, 2011
- Expression flow for 3D-aware face component transferACM Transactions on Graphics, 2011
- High resolution passive facial performance captureACM Transactions on Graphics, 2010
- High-quality single-shot capture of facial geometryACM Transactions on Graphics, 2010
- Example-based facial riggingACM Transactions on Graphics, 2010
- Face transfer with multilinear modelsACM Transactions on Graphics, 2005
- Spacetime facesACM Transactions on Graphics, 2004
- A flexible new technique for camera calibrationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2000