Sign Language Recognition with the Kinect Sensor Based on Conditional Random Fields
Open Access
- 24 December 2014
- Vol. 15 (1), 135-147
- https://doi.org/10.3390/s150100135
Abstract
Sign language is a visual language used by deaf people. One difficulty of sign language recognition is that sign instances of vary in both motion and shape in three-dimensional (3D) space. In this research, we use 3D depth information from hand motions, generated from Microsoft’s Kinect sensor and apply a hierarchical conditional random field (CRF) that recognizes hand signs from the hand motions. The proposed method uses a hierarchical CRF to detect candidate segments of signs using hand motions, and then a BoostMap embedding method to verify the hand shapes of the segmented signs. Experiments demonstrated that the proposed method could recognize signs from signed sentence data at a rate of 90.4%.Keywords
This publication has 10 references indexed in Scilit:
- Feature extraction in Brazilian Sign Language Recognition based on phonological structure and using RGB-D sensorsExpert Systems with Applications, 2014
- Robust sign language recognition by combining manual and non-manual features based on conditional random field and support vector machinePattern Recognition Letters, 2013
- Human-Computer Interaction Based on Hand Gestures Using RGB-D SensorsSensors, 2013
- Robust Part-Based Hand Gesture Recognition Using Kinect SensorIEEE Transactions on Multimedia, 2013
- Towards Real-Time and Rotation-Invariant American Sign Language Alphabet Recognition Using a Range CameraSensors, 2012
- A Kinect-based system for physical rehabilitation: A pilot study for young adults with motor disabilitiesResearch in Developmental Disabilities, 2011
- Simultaneous spotting of signs and fingerspellings based on hierarchical conditional random fields and boostmap embeddingsPattern Recognition, 2010
- Sign Language Spotting with a Threshold Model Based on Conditional Random FieldsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2008
- BoostMap: An Embedding Method for Efficient Nearest Neighbor RetrievalIEEE Transactions on Pattern Analysis and Machine Intelligence, 2007
- Random sample consensusCommunications of the ACM, 1981