Robust Part-Based Hand Gesture Recognition Using Kinect Sensor
Top Cited Papers
- 25 February 2013
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 15 (5), 1110-1120
- https://doi.org/10.1109/tmm.2013.2246148
Abstract
The recently developed depth sensors, e.g., the Kinect sensor, have provided new opportunities for human-computer interaction (HCI). Although great progress has been made by leveraging the Kinect sensor, e.g., in human body tracking, face recognition and human action recognition, robust hand gesture recognition remains an open problem. Compared to the entire human body, the hand is a smaller object with more complex articulations and more easily affected by segmentation errors. It is thus a very challenging problem to recognize hand gestures. This paper focuses on building a robust part-based hand gesture recognition system using Kinect sensor. To handle the noisy hand shapes obtained from the Kinect sensor, we propose a novel distance metric, Finger-Earth Mover's Distance (FEMD), to measure the dissimilarity between hand shapes. As it only matches the finger parts while not the whole hand, it can better distinguish the hand gestures of slight differences. The extensive experiments demonstrate that our hand gesture recognition system is accurate (a 93.2% mean accuracy on a challenging 10-gesture dataset), efficient (average 0.0750 s per frame), robust to hand articulations, distortions and orientation or scale changes, and can work in uncontrolled environments (cluttered backgrounds and lighting conditions). The superiority of our system is further demonstrated in two real-life HCI applications.Keywords
This publication has 33 references indexed in Scilit:
- Vision-based hand-gesture applicationsCommunications of the ACM, 2011
- Image retrievalACM Computing Surveys, 2008
- Vision-based hand pose estimation: A reviewComputer Vision and Image Understanding, 2007
- Real-Time Particle FiltersProceedings of the IEEE, 2004
- Extraction of 2D motion trajectories and its application to hand gesture recognitionIeee Transactions On Pattern Analysis and Machine Intelligence, 2002
- Shape matching and object recognition using shape contextsIeee Transactions On Pattern Analysis and Machine Intelligence, 2002
- A fuzzy rule-based approach to spatio-temporal hand gesture recognitionIEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews), 2000
- An HMM-based threshold model approach for gesture recognitionIeee Transactions On Pattern Analysis and Machine Intelligence, 1999
- Parametric hidden Markov models for gesture recognitionIeee Transactions On Pattern Analysis and Machine Intelligence, 1999
- Real-time American sign language recognition using desk and wearable computer based videoIeee Transactions On Pattern Analysis and Machine Intelligence, 1998