Gesture Recognition of Somatosensory Interactive Acupoint Massage Based on Image Feature Deep Learning Model
- 30 June 2021
- journal article
- research article
- Published by International Information and Engineering Technology Association in Traitement du Signal
- Vol. 38 (3), 565-572
- https://doi.org/10.18280/ts.380304
Abstract
During rehabilitation, many postoperative patients need to perform autonomous massage on time and on demand. Thus, this paper develops an individualized, intelligent, and independent rehabilitation training system for based on image feature deep learning model acupoint massage that excludes human factors. The system, which innovatively integrates massage gesture recognition with human pose recognition. It relies on the binocular depth camera Kinect DK and Google MediaPipe Holistic pipeline to collect the real-time image feature data on joints and gestures of the patient in autonomous massage. Then the system calculates the coordinates of each finger joint, and computes the human poses with VGG-16, a convolutional neural network (CNN); the calculated results are translated, and presented in a virtual reality (VR) model based on Unity 3D, aiming to guide the patient actions in autonomous massage. This is because the image feature of the gesture recognition and pose recognition is hindered, when the hand or the human is occluded by the body or other things, owing to the limited recognition range of the hardware. The experimental results show that, the proposed system could correctly recognize up to 84% of non-occluded gestures, and up to 93% of non-occluded poses; the system also exhibited a good real-time performance, a high operability, and a low cost. Facing the lack of medical staff, our system can effectively improve the life quality of patients.Keywords
Funding Information
- Zhejiang Provincial Basic Public Welfare Research Project (LGF20H290002)
- Traditional Chinese medicine scientific research fund project of zhejiang province (2019ZA005)
This publication has 18 references indexed in Scilit:
- Real-Time Hand Gesture Recognition Using Surface Electromyography and Machine Learning: A Systematic Literature ReviewSensors, 2020
- Multimodal hand gesture recognition combining temporal and pose information based on CNN descriptors and histogram of cumulative magnitudesJournal of Visual Communication and Image Representation, 2020
- Research on key issues of gesture recognition for artificial intelligenceSoft Computing, 2019
- Convolutional Neural Network Based American Sign Language Static Hand Gesture RecognitionInternational Journal of Ambient Computing and Intelligence, 2019
- CNN‐RNN based method for license plate recognitionCAAI Transactions on Intelligence Technology, 2018
- Traffic Command Gesture Recognition for Virtual Urban Scenes Based on a Spatiotemporal Convolution Neural NetworkISPRS International Journal of Geo-Information, 2018
- Multi-task convolutional neural network system for license plate recognitionInternational Journal of Control, Automation and Systems, 2017
- Leap Motion-based virtual reality training for improving motor functional recovery of upper limbs and neural reorganization in subacute stroke patientsNeural Regeneration Research, 2017
- Real-time sensory substitution to enable players who are blind to play video games using whole body gesturesEntertainment Computing, 2014
- Vision based hand gesture recognition for human computer interaction: a surveyArtificial Intelligence Review, 2012