Concurrent validity of machine learning-classified functional upper extremity use from accelerometry in chronic stroke

Abstract

Objective: Investigate the validity of machine learning derived amount of real-world functional upper extremity (UE) use in individuals with stroke. We hypothesized that machine learning classification of wrist-worn accelerometry will be as accurate as frame-by-frame video labeling (ground truth). A second objective was to validate the machine learning classification against measures of impairment, function, dexterity, and self-reported UE use. Design: Cross-sectional, convenience sampling. Setting: Outpatient rehabilitation. Participants: Individuals (>18-yrs) with neuroimaging-confirmed ischemic or hemorrhagic stroke >6-months prior (N=31) with persistent impairment of the hemiparetic arm and Upper-Extremity Fugl-Meyer (UEFM) score=12-57. Methods: Participants wore an accelerometer on each arm and were video recorded while completing an “activity script” comprised of activities and instrumental activities of daily living in a simulated apartment in outpatient rehabilitation. The video was annotated to determine the ground truth amount of functional UE use. Main outcome measures: Amount of real-world UE use was estimated using a Random Forest classifier trained on the accelerometry data. UE motor function was measured with the Action Research Arm Test (ARAT), UEFM, Nine-hole-peg test (9HPT). Amount of real-world UE use was measured using the Motor Activity Log (MAL). Results: Machine learning estimated use ratio was significantly correlated with use ratio derived from video annotation, ARAT, UEFM, 9HPT, and to a lesser extent with MAL. Bland-Altman plots showed excellent agreement between use ratios calculated from video-annotated and machine-learning classification. Factor analysis showed machine learning use ratios capture the same construct as ARAT, UEFM, 9HPT, and MAL and explain 83% of the variance in UE motor performance. Conclusions: Our machine learning approach provides a valid measure of functional UE use. The accuracy, validity, and small footprint of this machine learning approach makes it feasible for measurement of UE recovery in stroke rehabilitation trials.