A survey of speech-hand gesture recognition for the development of multimodal interfaces in computer games

Abstract

Hand gestures and speech constitute one of the most important modalities in human-to-human communication. The investigation of this phenomenon has been the focus of research about multimodal interfaces for speech and hand gesture recognition in human computer interaction(HCI). Computer games, as one of the most popular application areas of HCI, are likely to make use of this technology. Integration of speech and gesture in computer games make game play more enjoyable. This article does not intend to cover exhaustively all the issues related to multimodal interface creation in computer games. We primarily discuss some of the key issues related to multimodal interfaces for speech and hand gesture recognition and present a survey of current approaches to overcome the problems faced in this area.