Multimodal Biometric Human Recognition for Perceptual Human–Computer Interaction
- 21 June 2010
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews)
- Vol. 40 (6), 676-681
- https://doi.org/10.1109/tsmcc.2010.2050476
Abstract
In this paper, a novel video-based multimodal biometric verification scheme using the subspace-based low-level feature fusion of face and speech is developed for specific speaker recognition for perceptual human-computer interaction (HCI). In the proposed scheme, human face is tracked and face pose is estimated to weight the detected facelike regions in successive frames, where ill-posed faces and false-positive detections are assigned with lower credit to enhance the accuracy. In the audio modality, mel-frequency cepstral coefficients are extracted for voice-based biometric verification. In the fusion step, features from both modalities are projected into nonlinear Laplacian Eigenmap subspace for multimodal speaker recognition and combined at low level. The proposed approach is tested on the video database of ten human subjects, and the results show that the proposed scheme can attain better accuracy in comparison with the conventional multimodal fusion using latent semantic analysis as well as the single-modality verifications. The experiment on MATLAB shows the potential of the proposed scheme to attain the real-time performance for perceptual HCI applications.Keywords
This publication has 26 references indexed in Scilit:
- Score bi-Gaussian equalisation for multimodal person verificationIET Signal Processing, 2009
- Audio–Visual Affective Expression Recognition Through Multistream Fused HMMIEEE Transactions on Multimedia, 2008
- Interrelation Between Speech and Facial Gestures in Emotional Utterances: A Single Subject StudyIEEE Transactions on Audio, Speech, and Language Processing, 2007
- Robust Biometric Person Identification Using Automatic Classifier Fusion of Speech, Mouth, and Face ExpertsIEEE Transactions on Multimedia, 2007
- Audiovisual Probabilistic Tracking of Multiple Speakers in MeetingsIEEE Transactions on Audio, Speech, and Language Processing, 2007
- Audio-Visual BiometricsProceedings of the IEEE, 2006
- Multimodal biometric databases: an overviewIEEE Aerospace and Electronic Systems Magazine, 2006
- Face recognition using LaplacianfacesIeee Transactions On Pattern Analysis and Machine Intelligence, 2005
- Kernel machine based learning for multi-view face detection and pose estimationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- Multimodal decision-level fusion for person authenticationIEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans, 1999