Derivation and Validation of an Ensemble Model for the Prediction of Agitation in Mechanically Ventilated Patients Maintained Under Light Sedation

Abstract

Objectives: Light sedation is recommended over deep sedation for invasive mechanical ventilation to improve clinical outcome but may increase the risk of agitation. This study aimed to develop and prospectively validate an ensemble machine learning model for the prediction of agitation on a daily basis. Design: Variables collected in the early morning were used to develop an ensemble model by aggregating four machine learning algorithms including support vector machines, C5.0, adaptive boosting with classification trees, and extreme gradient boosting with classification trees, to predict the occurrence of agitation in the subsequent 24 hours. Setting: The training dataset was prospectively collected in 95 ICUs from 80 Chinese hospitals on May 11, 2016, and the validation dataset was collected in 20 out of these 95 ICUs on December 16, 2019. Patients: Invasive mechanical ventilation patients who were maintained under light sedation for 24 hours prior to the study day and who were to be maintained at the same sedation level for the next 24 hours. Interventions: None. Measurements and Main Results: A total of 578 invasive mechanical ventilation patients from 95 ICUs in 80 Chinese hospitals, including 459 in the training dataset and 119 in the validation dataset, were enrolled. Agitation was observed in 36% (270/578) of the invasive mechanical ventilation patients. The stepwise regression model showed that higher body temperature (odds ratio for 1°C increase: 5.29; 95% CI, 3.70–7.84; p < 0.001), greater minute ventilation (odds ratio for 1 L/min increase: 1.15; 95% CI, 1.02–1.30; p = 0.019), higher Richmond Agitation-Sedation Scale (odds ratio for 1-point increase: 2.43; 95% CI, 1.92–3.16; p < 0.001), and days on invasive mechanical ventilation (odds ratio for 1-d increase: 0.95; 95% CI, 0.93–0.98; p = 0.001) were independently associated with agitation in the subsequent 24 hours. In the validation dataset, the ensemble model showed good discrimination (area under the receiver operating characteristic curve, 0.918; 95% CI, 0.866–0.969) and calibration (Hosmer-Lemeshow test p = 0.459) in predicting the occurrence of agitation within 24 hours. Conclusions: This study developed an ensemble model for the prediction of agitation in invasive mechanical ventilation patients under light sedation. The model showed good calibration and discrimination in an independent dataset.