Unsupervised Domain Adaptation with Regularized Optimal Transport for Multimodal 2D+3D Facial Expression Recognition
- 1 May 2018
- conference paper
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE) in 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018)
Abstract
Since human expressions have strong flexibility and personality, subject-independent facial expression recognition is a typical data bias problem. To address this problem, we propose a novel approach, namely unsupervised domain adaptation with regularized optimal transport for multimodal 2D+3D Facial Expression Recognition (FER). In particular, Wasserstein distance is employed to measure the distribution inconsistency between the training samples (i.e. source domain) and test samples (i.e. target domain). Minimization of this Wasserstein distance is equivalent to finding an optimal transport mapping from training to test samples. Once we find this mapping, original training samples can be transformed into a new space in which the distributions of the mapped training samples and the test samples can be well-aligned. In this case, classifier learned from the transformed training samples can be well generalized to the test samples for expression prediction. In practice, approximate optimal transport can be effectively solved by adding entropy regularization. To fully explore the class label information of training samples, group sparsity regularizer is also used to enforce that the training samples from the same expression class can be mapped to the same group. Experimental results evaluated on the BU-3DFE and Bosphorus databases demonstrate that the proposed approach can achieve superior performance compared with the state-of-the-art methods.Keywords
This publication has 27 references indexed in Scilit:
- An Automatic Framework for Textured 3D Video-Based Facial Expression RecognitionIEEE Transactions on Affective Computing, 2014
- Learning person-specific models for facial expression and action unit recognitionPattern Recognition Letters, 2013
- Static and dynamic 3D facial expression recognition: A comprehensive surveyImage and Vision Computing, 2012
- 3D/4D facial expression analysis: An advanced annotated face model approachImage and Vision Computing, 2012
- Shape analysis of local facial patches for 3D facial expression recognitionPattern Recognition, 2011
- 3D Facial Expression Recognition Based on Histograms of Surface Differential QuantitiesPublished by Springer Science and Business Media LLC ,2011
- Real-time 2D+3D facial action and expression recognitionPattern Recognition, 2010
- Bosphorus Database for 3D Face AnalysisLecture Notes in Computer Science, 2008
- Facial Expression Recognition Using 3D Facial Feature DistancesLecture Notes in Computer Science, 2007
- Automatic analysis of facial expressions: the state of the artIEEE Transactions on Pattern Analysis and Machine Intelligence, 2000