Trunk-Branch Ensemble Convolutional Neural Networks for Video-Based Face Recognition
Top Cited Papers
- 2 May 2017
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 40 (4), 1002-1014
- https://doi.org/10.1109/tpami.2017.2700390
Abstract
Human faces in surveillance videos often suffer from severe image blur, dramatic pose variations, and occlusion. In this paper, we propose a comprehensive framework based on Convolutional Neural Networks (CNN) to overcome challenges in video-based face recognition (VFR). First, to learn blur-robust face representations, we artificially blur training data composed of clear still images to account for a shortfall in real-world video training data. Using training data composed of both still images and artificially blurred data, CNN is encouraged to learn blur-insensitive features automatically. Second, to enhance robustness of CNN features to pose variations and occlusion, we propose a Trunk-Branch Ensemble CNN model (TBE-CNN), which extracts complementary information from holistic face images and patches cropped around facial components. TBE-CNN is an end-to-end model that extracts features efficiently by sharing the low- and middle-level convolutional layers between the trunk and branch networks. Third, to further promote the discriminative power of the representations learnt by TBE-CNN, we propose an improved triplet loss function. Systematic experiments justify the effectiveness of the proposed techniques. Most impressively, TBE-CNN achieves state-of-the-art performance on three popular video face databases: PaSC, COX Face, and YouTube Faces. With the proposed techniques, we also obtain the first place in the BTAS 2016 Video Person Recognition Evaluation.Keywords
Other Versions
Funding Information
- Australian Research Council (FT-130101457, DP-140102164)
This publication has 48 references indexed in Scilit:
- A comparative study of video-based object recognition from an egocentric viewpointNeurocomputing, 2016
- Face recognition on large-scale video in the wild with hybrid Euclidean-and-Riemannian metric learningPattern Recognition, 2015
- Multi-Directional Multi-Level Dual-Cross Patterns for Robust Face RecognitionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2015
- Report on the FG 2015 Video Person Recognition EvaluationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2015
- Image Set-Based Collaborative Representation for Face RecognitionIEEE Transactions on Information Forensics and Security, 2014
- Toward Large-Population Face Identification in Unconstrained VideosIEEE Transactions on Circuits and Systems for Video Technology, 2014
- Health care professionals' perspectives of living and dying with primary malignant glioma: Implications for a unique cancer trajectoryPalliative & Supportive Care, 2013
- FACE RECOGNITION FROM VIDEO: A REVIEWInternational Journal of Pattern Recognition and Artificial Intelligence, 2012
- Dictionary-Based Face Recognition from VideoLecture Notes in Computer Science, 2012
- Combining appearance and motion for face and gender recognition from videosPattern Recognition, 2009