End-to-End 3D Face Reconstruction with Deep Neural Networks
- 1 July 2017
- conference paper
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 10636919,p. 1503-1512
- https://doi.org/10.1109/cvpr.2017.164
Abstract
Monocular 3D facial shape reconstruction from a single 2D facial image has been an active research area due to its wide applications. Inspired by the success of deep neural networks (DNN), we propose a DNN-based approach for End-to-End 3D FAce Reconstruction (UH-E2FAR) from a single 2D image. Different from recent works that reconstruct and refine the 3D face in an iterative manner using both an RGB image and an initial 3D facial shape rendering, our DNN model is end-to-end, and thus the complicated 3D rendering process can be avoided. Moreover, we integrate in the DNN architecture two components, namely a multi-task loss function and a fusion convolutional neural network (CNN) to improve facial expression reconstruction. With the multi-task loss function, 3D face reconstruction is divided into neutral 3D facial shape reconstruction and expressive 3D facial shape reconstruction. The neutral 3D facial shape is class-specific. Therefore, higher layer features are useful. In comparison, the expressive 3D facial shape favors lower or intermediate layer features. With the fusion-CNN, features from different intermediate layers are fused and transformed for predicting the 3D expressive facial shape. Through extensive experiments, we demonstrate the superiority of our end-to-end framework in improving the accuracy of 3D face reconstruction.Keywords
This publication has 27 references indexed in Scilit:
- Pose-robust face signature for multi-view face recognitionPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2015
- Effective face frontalization in unconstrained imagesPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2015
- Coupled Dictionary Learning for the Detail-Enhanced Synthesis of 3-D Facial ExpressionsIEEE Transactions on Cybernetics, 2015
- Multi-Task Pose-Invariant Face RecognitionIEEE Transactions on Image Processing, 2015
- Deep Face RecognitionPublished by British Machine Vision Association and Society for Pattern Recognition ,2015
- FaceWarehouse: A 3D Facial Expression Database for Visual ComputingIEEE Transactions on Visualization and Computer Graphics, 2013
- A statistical method for robust 3D surface reconstruction from sparse dataPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2004
- A morphable model for the synthesis of 3D facesPublished by Association for Computing Machinery (ACM) ,1999
- A method for registration of 3-D shapesIEEE Transactions on Pattern Analysis and Machine Intelligence, 1992
- Illumination for computer generated picturesCommunications of the ACM, 1975