Facial Expression Recognition via Joint Deep Learning of RGB-Depth Map Latent Representations
Open Access
- 1 October 2017
- conference paper
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE) in 2017 IEEE International Conference on Computer Vision Workshops (ICCVW)
- p. 3161-3168
- https://doi.org/10.1109/iccvw.2017.374
Abstract
Humans use facial expressions successfully for conveying their emotional states. However, replicating such success in the human-computer interaction domain is an active research problem. In this paper, we propose deep convolutional neural network (DCNN) for joint learning of robust facial expression features from fused RGB and depth map latent representations. We posit that learning jointly from both modalities result in a more robust classifier for facial expression recognition (FER) as opposed to learning from either of the modalities independently. Particularly, we construct a learning pipeline that allows us to learn several hierarchical levels of feature representations and then perform the fusion of RGB and depth map latent representations for joint learning of facial expressions. Our experimental results on the BU-3DFE dataset validate the proposed fusion approach, as a model learned from the joint modalities outperforms models learned from either of the modalities.Keywords
This publication has 27 references indexed in Scilit:
- Convolutional Neural Network Models for Facial Expression Recognition Using BU-3DFE DatabaseLecture Notes in Electrical Engineering, 2016
- Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet ClassificationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2015
- Image based Static Facial Expression Recognition with Multiple Deep Network LearningPublished by Association for Computing Machinery (ACM) ,2015
- An efficient multimodal 2D + 3D feature-based approach to automatic facial expression recognitionComputer Vision and Image Understanding, 2015
- Expression-invariant face recognition using depth and intensity dual-tree complex wavelet transform featuresJournal of Electronic Imaging, 2015
- Automatic Analysis of Facial Affect: A Survey of Registration, Representation, and RecognitionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2014
- A neural-AdaBoost based facial expression recognition systemExpert Systems with Applications, 2014
- Featural processing in recognition of emotional facial expressionsCognition and Emotion, 2013
- Facial expression recognition based on fusion feature of PCA and LBP with SVMOptik, 2013
- Automatic 3D Facial Expression Recognition Based on a Bayesian Belief Net and a Statistical Facial Feature ModelPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2010