Toward detecting emotions in spoken dialogs
- 22 February 2005
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Speech and Audio Processing
- Vol. 13 (2), 293-303
- https://doi.org/10.1109/tsa.2004.838534
Abstract
The importance of automatically recognizing emotions from human speech has grown with the increasing role of spoken language interfaces in human-computer interaction applications. This paper explores the detection of domain-specific emotions using language and discourse information in conjunction with acoustic correlates of emotion in speech signals. The specific focus is on a case study of detecting negative and non-negative emotions using spoken language data obtained from a call center application. Most previous studies in emotion recognition have used only the acoustic information contained in speech. In this paper, a combination of three sources of information-acoustic, lexical, and discourse-is used for emotion recognition. To capture emotion information at the language level, an information-theoretic notion of emotional salience is introduced. Optimization of the acoustic correlates of emotion with respect to classification error was accomplished by investigating different feature sets obtained from feature selection, followed by principal component analysis. Experimental results on our call center data show that the best results are obtained when acoustic and language information are combined. Results show that combining all the information, rather than using only acoustic information, improves emotion classification by 40.7% for males and 36.4% for females (linear discriminant classifier used for acoustic information).Keywords
This publication has 15 references indexed in Scilit:
- Recognition of negative emotions from the speech signalPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2005
- Recognizing emotion in speechPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- Automatic spoken affect classification and analysisPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- Decision templates for multiple classifier fusion: an experimental comparisonPattern Recognition, 2001
- Emotion recognition in human-computer interactionIEEE Signal Processing Magazine, 2001
- Combining multiple classifiers by averaging or by multiplying?Pattern Recognition, 2000
- On automated language acquisitionThe Journal of the Acoustical Society of America, 1995
- Toward the simulation of emotion in synthetic speech: A review of the literature on human vocal emotionThe Journal of the Acoustical Society of America, 1993
- The Cognitive Structure of EmotionsPublished by Cambridge University Press (CUP) ,1988
- Optimal Data Fusion in Multiple Sensor Detection SystemsIEEE Transactions on Aerospace and Electronic Systems, 1986