Trends in the conduct and reporting of clinical prediction model development and validation: a systematic review
Open Access
- 19 January 2022
- journal article
- review article
- Published by Oxford University Press (OUP) in Journal of the American Medical Informatics Association
- Vol. 29 (5), 983-989
- https://doi.org/10.1093/jamia/ocac002
Abstract
Objectives This systematic review aims to provide further insights into the conduct and reporting of clinical prediction model development and validation over time. We focus on assessing the reporting of information necessary to enable external validation by other investigators. Materials and Methods We searched Embase, Medline, Web-of-Science, Cochrane Library, and Google Scholar to identify studies that developed 1 or more multivariable prognostic prediction models using electronic health record (EHR) data published in the period 2009-2019. Results We identified 422 studies that developed a total of 579 clinical prediction models using EHR data. We observed a steep increase over the years in the number of developed models. The percentage of models externally validated in the same paper remained at around 10%. Throughout 2009-2019, for both the target population and the outcome definitions, code lists were provided for less than 20% of the models. For about half of the models that were developed using regression analysis, the final model was not completely presented. Discussion Overall, we observed limited improvement over time in the conduct and reporting of clinical prediction model development and validation. In particular, the prediction problem definition was often not clearly reported, and the final model was often not completely presented. Conclusion Improvement in the reporting of information necessary to enable external validation by other investigators is still urgently needed to increase clinical adoption of developed models.Funding Information
- European Health Data & Evidence Network
- Innovative Medicines Initiative 2 Joint Undertaking (JU (806968)
- European Union’s Horizon 2020 research and innovation program and EFPIA
This publication has 20 references indexed in Scilit:
- Design and implementation of a standardized framework to generate and evaluate patient-level prediction models using observational healthcare dataJournal of the American Medical Informatics Association, 2018
- Opportunities and challenges in developing risk prediction models with electronic health records data: a systematic reviewJournal of the American Medical Informatics Association, 2016
- Sample size considerations for the external validation of a multivariable prognostic model: a resampling studyStatistics in Medicine, 2015
- Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies: The CHARMS ChecklistPLoS Medicine, 2014
- External validation of multivariable prediction models: a systematic review of methodological conduct and reportingBMC Medical Research Methodology, 2014
- Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statementBMJ, 2014
- Reporting and Methods in Clinical Prediction Research: A Systematic ReviewPLoS Medicine, 2012
- Assessing the Performance of Prediction ModelsEpidemiology, 2010
- The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaborationJournal of Clinical Epidemiology, 2009
- The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and ElaborationPLoS Medicine, 2009