Feature Acquisition and Analysis for Facial Expression Recognition Using Convolutional Neural Networks
- 1 September 2017
- journal article
- Published by Japanese Society for Artificial Intelligence in Transactions of the Japanese Society for Artificial Intelligence
- Vol. 32 (5), F-H34_1-H34_1
- https://doi.org/10.1527/tjsai.f-h34
Abstract
Facial expressions play an important role in communication as much as words. In facial expression recognition by human, it is difficult to uniquely judge, because facial expression has the sway of recognition by individual difference and subjective recognition. Therefore, it is difficult to evaluate the reliability of the result from recognition accuracy alone, and the analysis for explaining the result and feature learned by Convolutional Neural Networks (CNN) will be considered important. In this study, we carried out the facial expression recognition from facial expression images using CNN. In addition, we analysed CNN for understanding learned features and prediction results. Emotions we focused on are “happiness”, “sadness”, “surprise”, “anger”, “disgust”, “fear” and “neutral”. As a result, using 32286 facial expression images, have obtained an emotion recognition score of about 57%; for two emotions (Happiness, Surprise) the recognition score exceeded 70%, but Anger and Fear was less than 50%. In the analysis of CNN, we focused on the learning process, input and intermediate layer. Analysis of the learning progress confirmed that increased data can be recognised in the following order “happiness”, “surprise”, “neutral”, “anger”, “disgust”, “sadness” and “fear”. From the analysis result of the input and intermediate layer, we confirmed that the feature of the eyes and mouth strongly influence the facial expression recognition, and intermediate layer neurons had active patterns corresponding to facial expressions, and also these activate patterns do not respond to partial features of facial expressions. From these results, we concluded that CNN has learned the partial features of eyes and mouth from input, and recognise the facial expression using hidden layer units having the area corresponding to each facial expression.Keywords
This publication has 11 references indexed in Scilit:
- Densely Connected Convolutional NetworksPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2017
- A Neural Algorithm of Artistic StyleJournal of Vision, 2016
- "Why Should I Trust You?"Published by Association for Computing Machinery (ACM) ,2016
- Visualizing and Understanding Convolutional NetworksLecture Notes in Computer Science, 2014
- Challenges in Representation Learning: A Report on Three Machine Learning ContestsPublished by Springer Science and Business Media LLC ,2013
- Static facial expression analysis in tough conditions: Data, evaluation protocol and benchmarkPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2011
- Facial Expression Recognition using Feature Extraction based on Estimation of Useful Facial FeaturesJournal of Japan Society for Fuzzy Theory and Intelligent Informatics, 2011
- Extracting and composing robust features with denoising autoencodersPublished by Association for Computing Machinery (ACM) ,2008
- Cross-Cultural Emotion Recognition among Canadian Ethnic GroupsJournal of Cross-Cultural Psychology, 2005
- Backpropagation Applied to Handwritten Zip Code RecognitionNeural Computation, 1989