Improving Human–Robot Interaction by Enhancing NAO Robot Awareness of Human Facial Expression
Open Access
- 27 September 2021
- Vol. 21 (19), 6438
- https://doi.org/10.3390/s21196438
Abstract
An intriguing challenge in the human–robot interaction field is the prospect of endowing robots with emotional intelligence to make the interaction more genuine, intuitive, and natural. A crucial aspect in achieving this goal is the robot’s capability to infer and interpret human emotions. Thanks to its design and open programming platform, the NAO humanoid robot is one of the most widely used agents for human interaction. As with person-to-person communication, facial expressions are the privileged channel for recognizing the interlocutor’s emotional expressions. Although NAO is equipped with a facial expression recognition module, specific use cases may require additional features and affective computing capabilities that are not currently available. This study proposes a highly accurate convolutional-neural-network-based facial expression recognition model that is able to further enhance the NAO robot’ awareness of human facial expressions and provide the robot with an interlocutor’s arousal level detection capability. Indeed, the model tested during human–robot interactions was 91% and 90% accurate in recognizing happy and sad facial expressions, respectively; 75% accurate in recognizing surprised and scared expressions; and less accurate in recognizing neutral and angry expressions. Finally, the model was successfully integrated into the NAO SDK, thus allowing for high-performing facial expression classification with an inference time of 0.34 ± 0.04 s.Keywords
This publication has 27 references indexed in Scilit:
- Adaptive facial point detection and emotion recognition for a humanoid robotComputer Vision and Image Understanding, 2015
- Social robots in advanced dementiaFrontiers in Aging Neuroscience, 2015
- Automated facial coding: Validation of basic emotions and FACS AUs in FaceReader.Journal of Neuroscience, Psychology, and Economics, 2014
- DeepFace: Closing the Gap to Human-Level Performance in Face VerificationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2014
- Rich Feature Hierarchies for Accurate Object Detection and Semantic SegmentationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2014
- A Socially Assistive Robot Exercise Coach for the ElderlyJournal of Human-Robot Interaction, 2013
- Coherence between Emotion and Facial Expression: Evidence from Laboratory ExperimentsEmotion Review, 2013
- Robots for Use in Autism ResearchAnnual Review of Biomedical Engineering, 2012
- Effects of robot-assisted therapy on stroke rehabilitation in upper limbs: Systematic review and meta-analysis of the literatureJournal of Rehabilitation Research and Development, 2012
- Are there basic emotions?Psychological Review, 1992