Multi-Task Convolutional Neural Network for Pose-Invariant Face Recognition
- 23 October 2017
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 27 (2), 964-975
- https://doi.org/10.1109/tip.2017.2765830
Abstract
This paper explores multi-task learning (MTL) for face recognition. First, we propose a multi-task convolutional neural network (CNN) for face recognition, where identity classification is the main task and pose, illumination, and expression (PIE) estimations are the side tasks. Second, we develop a dynamic-weighting scheme to automatically assign the loss weights to each side task, which solves the crucial problem of balancing between different tasks in MTL. Third, we propose a pose-directed multi-task CNN by grouping different poses to learn pose-specific identity features, simultaneously across all poses in a joint framework. Last but not least, we propose an energy-based weight analysis method to explore how CNN-based MTL works. We observe that the side tasks serve as regularizations to disentangle the PIE variations from the learnt identity features. Extensive experiments on the entire multi-PIE dataset demonstrate the effectiveness of the proposed approach. To the best of our knowledge, this is the first work using all data in multi-PIE for face recognition. Our approach is also applicable to in-the-wild data sets for pose-invariant face recognition and achieves comparable or better performance than state of the art on LFW, CFP, and IJB-A datasets.Other Versions
This publication has 48 references indexed in Scilit:
- Learning Deep Representation for Face Alignment with Auxiliary AttributesIEEE Transactions on Pattern Analysis and Machine Intelligence, 2015
- Robust H ∞ control for singular systems with state delay and parameter uncertaintyAdvances in Difference Equations, 2015
- Multi-Task Pose-Invariant Face RecognitionIEEE Transactions on Image Processing, 2015
- A Scalable Formulation of Probabilistic Linear Discriminant Analysis: Applied to Face RecognitionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2013
- Pose-robust face recognition via sparse representationPattern Recognition, 2012
- Structured feature selection and task relationship inference for multi-task learningKnowledge and Information Systems, 2012
- Separability Oriented Preprocessing for Illumination-Insensitive Face RecognitionLecture Notes in Computer Science, 2012
- Multi-PIEImage and Vision Computing, 2010
- Lighting Aware Preprocessing for Face Recognition across Varying IlluminationLecture Notes in Computer Science, 2010
- Convex multi-task feature learningMachine Learning, 2008