Multimodal Classification of Parkinson’s Disease in Home Environments with Resiliency to Missing Modalities
Open Access
- 16 June 2021
- Vol. 21 (12), 4133
- https://doi.org/10.3390/s21124133
Abstract
Parkinson’s disease (PD) is a chronic neurodegenerative condition that affects a patient’s everyday life. Authors have proposed that a machine learning and sensor-based approach that continuously monitors patients in naturalistic settings can provide constant evaluation of PD and objectively analyse its progression. In this paper, we make progress toward such PD evaluation by presenting a multimodal deep learning approach for discriminating between people with PD and without PD. Specifically, our proposed architecture, named MCPD-Net, uses two data modalities, acquired from vision and accelerometer sensors in a home environment to train variational autoencoder (VAE) models. These are modality-specific VAEs that predict effective representations of human movements to be fused and given to a classification module. During our end-to-end training, we minimise the difference between the latent spaces corresponding to the two data modalities. This makes our method capable of dealing with missing modalities during inference. We show that our proposed multimodal method outperforms unimodal and other multimodal approaches by an average increase in -score of 0.25 and 0.09, respectively, on a data set with real patients. We also show that our method still outperforms other approaches by an average increase in -score of 0.17 when a modality is missing during inference, demonstrating the benefit of training on multiple modalities.
Keywords
Funding Information
- UK Engineering and Physical Sciences Research Council (EP/R005273/1, 204813/Z/16/Z)
This publication has 44 references indexed in Scilit:
- Navigating social and ethical challenges of biobanking for human microbiome researchBMC Medical Ethics, 2017
- Subtle motor disturbances in PREDICT-PD participantsJournal of Neurology, Neurosurgery & Psychiatry, 2016
- SPHERE: A Sensor Platform for Healthcare in a Residential EnvironmentPublished by Springer Science and Business Media LLC ,2016
- Unsupervised home monitoring of Parkinson's disease motor symptoms using body-worn accelerometersParkinsonism & Related Disorders, 2016
- Parkinson’s disease hand tremor detection system for mobile applicationJournal of Medical Engineering & Technology, 2016
- Bridging e-Health and the Internet of Things: The SPHERE ProjectIEEE Intelligent Systems, 2015
- Spatiotemporal Gait Patterns During Overt and Covert Evaluation in Patients With Parkinson’s Disease and Healthy Subjects: Is There a Hawthorne Effect?Journal of Applied Biomechanics, 2015
- Detecting and monitoring the symptoms of Parkinson's disease using smartphones: A pilot studyParkinsonism & Related Disorders, 2015
- Medical Technology in Smart Homes: Exploring the User's Perspective on Privacy, Intimacy and TrustPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2011
- Parkinson's disease: clinical features and diagnosisJournal of Neurology, Neurosurgery & Psychiatry, 2008