Bimodal Emotion Recognition using Machine Learning
- 30 April 2021
- journal article
- Published by Blue Eyes Intelligence Engineering and Sciences Engineering and Sciences Publication - BEIESP in International Journal of Engineering and Advanced Technology
- Vol. 10 (4), 189-194
- https://doi.org/10.35940/ijeat.d2451.0410421
Abstract
The predominant communication channel to convey relevant and high impact information is the emotions that is embedded on our communications. Researchers have tried to exploit these emotions in recent years for human robot interactions (HRI) and human computer interactions (HCI). Emotion recognition through speech or through facial expression is termed as single mode emotion recognition. The rate of accuracy of these single mode emotion recognitions are improved using the proposed bimodal method by combining the modalities of speech and facing and recognition of emotions using a Convolutional Neural Network (CNN) model. In this paper, the proposed bimodal emotion recognition system, contains three major parts such as processing of audio, processing of video and fusion of data for detecting the emotion of a person. The fusion of visual information and audio data obtained from two different channels enhances the emotion recognition rate by providing the complementary data. The proposed method aims to classify 7 basic emotions (anger, disgust, fear, happy, neutral, sad, surprise) from an input video. We take audio and image frame from the video input to predict the final emotion of a person. The dataset used is an audio-visual dataset uniquely suited for the study of multi-modal emotion expression and perception. Dataset used here is RAVDESS dataset which contains audio-visual dataset, visual dataset and audio dataset. For bimodal emotion detection the audio-visual dataset is used.Keywords
This publication has 16 references indexed in Scilit:
- Neural network method through facial expression recognitionPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2017
- User Specific Weights Based on Score Distance and EER for Weighted Sum Rule FusionPublished by Springer Science and Business Media LLC ,2016
- Study on CNN in the recognition of emotion in audio and imagesPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2016
- Analysis of k-Fold Cross-Validation over Hold-Out Validation on Colossal Datasets for Quality ClassificationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2016
- HERS:Human emotion recognition systemPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2016
- Random decision forests for automatic brain tumor segmentation on multi-modal MRI imagesPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2015
- Speech Emotion Recognition Using Fourier ParametersIEEE Transactions on Affective Computing, 2015
- Recognition of emotions from video using acoustic and facial featuresSignal, Image and Video Processing, 2013
- Integrating Iris and Signature Traits for Personal Authentication Using User-SpecificWeightingSensors, 2012
- Speech emotion recognition using hidden Markov modelsSpeech Communication, 2003