Learning Expressionlets on Spatio-temporal Manifold for Dynamic Facial Expression Recognition
Top Cited Papers
- 1 June 2014
- conference paper
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1749-1756
- https://doi.org/10.1109/cvpr.2014.226
Abstract
Facial expression is temporally dynamic event which can be decomposed into a set of muscle motions occurring in different facial regions over various time intervals. For dynamic expression recognition, two key issues, temporal alignment and semantics-aware dynamic representation, must be taken into account. In this paper, we attempt to solve both problems via manifold modeling of videos based on a novel mid-level representation, i.e. expressionlet. Specifically, our method contains three key components: 1) each expression video clip is modeled as a spatio-temporal manifold (STM) formed by dense low-level features, 2) a Universal Manifold Model (UMM) is learned over all low-level features and represented as a set of local ST modes to statistically unify all the STMs. 3) the local modes on each STM can be instantiated by fitting to UMM, and the corresponding expressionlet is constructed by modeling the variations in each local ST mode. With above strategy, expression videos are naturally aligned both spatially and temporally. To enhance the discriminative power, the expressionlet-based STM representation is further processed with discriminant embedding. Our method is evaluated on four public expression databases, CK+, MMI, Oulu-CASIA, and AFEW. In all cases, our method reports results better than the known state-of-the-art.Keywords
This publication has 24 references indexed in Scilit:
- Probabilistic Elastic Matching for Pose Variant Face VerificationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2013
- Manifold based Sparse Representation for robust expression recognition without neutral subtractionPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2011
- Facial expression recognition with temporal modeling of shapesPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2011
- Facial expression recognition from near-infrared videosImage and Vision Computing, 2011
- Learning hierarchical invariant spatio-temporal features for action recognition with independent subspace analysisPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2011
- The computer expression recognition toolbox (CERT)Published by Institute of Electrical and Electronics Engineers (IEEE) ,2011
- The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expressionPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2010
- A Spatio-Temporal Descriptor Based on 3D-GradientsPublished by British Machine Vision Association and Society for Pattern Recognition ,2008
- Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial ExpressionsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2007
- Comprehensive database for facial expression analysisPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002