A comparison of machine learning models versus clinical evaluation for mortality prediction in patients with sepsis
Open Access
- 19 January 2021
- journal article
- research article
- Published by Public Library of Science (PLoS) in PLOS ONE
- Vol. 16 (1), e0245157
- https://doi.org/10.1371/journal.pone.0245157
Abstract
Patients with sepsis who present to an emergency department (ED) have highly variable underlying disease severity, and can be categorized from low to high risk. Development of a risk stratification tool for these patients is important for appropriate triage and early treatment. The aim of this study was to develop machine learning models predicting 31-day mortality in patients presenting to the ED with sepsis and to compare these to internal medicine physicians and clinical risk scores. A single-center, retrospective cohort study was conducted amongst 1,344 emergency department patients fulfilling sepsis criteria. Laboratory and clinical data that was available in the first two hours of presentation from these patients were randomly partitioned into a development (n = 1,244) and validation dataset (n = 100). Machine learning models were trained and evaluated on the development dataset and compared to internal medicine physicians and risk scores in the independent validation dataset. The primary outcome was 31-day mortality. A number of 1,344 patients were included of whom 174 (13.0%) died. Machine learning models trained with laboratory or a combination of laboratory + clinical data achieved an area-under-the ROC curve of 0.82 (95% CI: 0.80–0.84) and 0.84 (95% CI: 0.81–0.87) for predicting 31-day mortality, respectively. In the validation set, models outperformed internal medicine physicians and clinical risk scores in sensitivity (92% vs. 72% vs. 78%;p0.02). The model had higher diagnostic accuracy with an area-under-the-ROC curve of 0.85 (95%CI: 0.78–0.92) compared to abbMEDS (0.63,0.54–0.73), mREMS (0.63,0.54–0.72) and internal medicine physicians (0.74,0.65–0.82). Machine learning models outperformed internal medicine physicians and clinical risk scores in predicting 31-day mortality. These models are a promising tool to aid in risk stratification of patients presenting to the ED with sepsis.Keywords
Funding Information
- Noyons stipendium
This publication has 43 references indexed in Scilit:
- Explaining prediction models and individual predictions with feature contributionsKnowledge and Information Systems, 2013
- Frequent Users of Emergency Departments: The Myths, the Data, and the Policy ImplicationsAnnals of Emergency Medicine, 2010
- Comparison of severity of illness scoring systems in the prediction of hospital mortality in severe sepsis and septic shockJournal of Emergencies, Trauma, and Shock, 2010
- Prediction of mortality in adult emergency department patients with sepsisEmergency Medicine Journal, 2009
- Evaluation of SOFA-based models for predicting mortality in the ICU: A systematic reviewCritical Care, 2009
- Rapid Emergency Medicine Score Can Predict Long‐term Mortality in Nonsurgical Emergency Department PatientsAcademic Emergency Medicine, 2004
- Analysis of regression in game theory approachApplied Stochastic Models in Business and Industry, 2001
- The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failureIntensive Care Medicine, 1996
- VERIFICATION OF FORECASTS EXPRESSED IN TERMS OF PROBABILITYMonthly Weather Review, 1950