Adaptive Body Gesture Representation for Automatic Emotion Recognition
- 9 March 2016
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Interactive Intelligent Systems
- Vol. 6 (1), 1-31
- https://doi.org/10.1145/2818740
Abstract
We present a computational model and a system for the automated recognition of emotions starting from full-body movement. Three-dimensional motion data of full-body movements are obtained either from professional optical motion-capture systems (Qualisys) or from low-cost RGB-D sensors (Kinect and Kinect2). A number of features are then automatically extracted at different levels, from kinematics of a single joint to more global expressive features inspired by psychology and humanistic theories (e.g., contraction index, fluidity, and impulsiveness). An abstraction layer based on dictionary learning further processes these movement features to increase the model generality and to deal with intraclass variability, noise, and incomplete information characterizing emotion expression in human movement. The resulting feature vector is the input for a classifier performing real-time automatic emotion recognition based on linear support vector machines. The recognition performance of the proposed model is presented and discussed, including the tradeoff between precision of the tracking measures (we compare the Kinect RGB-D sensor and the Qualisys motion-capture system) versus dimension of the training dataset. The resulting model and system have been successfully applied in the development of serious games for helping autistic children learn to recognize and express emotions by means of their full-body movement.Keywords
Funding Information
- European Commission (289021)
This publication has 45 references indexed in Scilit:
- Standing up for the body. Recent progress in uncovering the networks involved in the perception of bodies and bodily expressionsNeuroscience & Biobehavioral Reviews, 2010
- Why bodies? Twelve reasons for including bodily expressions in affective neurosciencePhilosophical Transactions B, 2009
- Communicating expressiveness and affect in multimodal interactive systemsIEEE MultiMedia, 2005
- Periodicity transformsIEEE Transactions on Signal Processing, 1999
- Perception of Emotion from Dynamic Point-Light Displays Represented in DancePerception, 1996
- Impulsivity, Sensation Seeking, and Behavioral and Emotional Responses to AlcoholAlcohol: Clinical and Experimental Research, 1991
- The contribution of general features of body movement to the attribution of emotionsJournal of Nonverbal Behavior, 1989
- Detecting deception from the body or face.Journal of Personality and Social Psychology, 1974
- Visual perception of biological motion and a model for its analysisPerception & Psychophysics, 1973
- Differential communication of affect by head and body cues.Journal of Personality and Social Psychology, 1965