Assessing the Accuracy of a Deep Learning Method to Risk Stratify Indeterminate Pulmonary Nodules
Open Access
- 15 July 2020
- journal article
- research article
- Published by American Thoracic Society in American Journal of Respiratory and Critical Care Medicine
- Vol. 202 (2), 241-249
- https://doi.org/10.1164/rccm.201903-0505oc
Abstract
Rationale: The management indeterminate pulmonary nodules (IPNs) remains challenging, resulting in invasive procedures and delays in diagnosis and treatment. Strategies to decrease the rate of unnecessary invasive procedures and to optimize surveillance regimens are needed. Objectives: Develop and validate a deep learning method to improve the management of IPNs. Methods: A Lung Cancer Prediction Convolutional Neural Network (LCP-CNN) model was trained using CT images of IPNs from the National Lung Screening Trial (NLST), internally validated, and externally tested on cohorts from two academic institutions. Measurements and Main Results: The area under the receiver operating characteristic curve in the external validation cohorts were 83.5% (95%CI:75.4-90.7%) and 91.9% (95%CI:88.7-94.7%) compared with 78.1% (95%CI:68.7-86.4%) and 81.9 (95%CI:76.1-87.1%) respectively for a commonly used clinical risk model for incidental nodules. Using 5% and 65% malignancy thresholds defining low and high-risk categories, the overall net reclassification in the validation cohorts for cancers and benign nodules compared to the Mayo model was 0.34 (Vanderbilt) and 0.30 (Oxford) as a rule-in test, and 0.33 (Vanderbilt) and 0.58 (Oxford) as a rule-out test. The LCP-CNN compared to traditional risk prediction models was associated with an improved accuracy in the predicted likelihood of disease at each threshold of management and in our external validation cohorts. Conclusions: This study demonstrates that this deep learning algorithm can correctly reclassify IPNs into low or high-risk categories in over a third of cancers and benign nodules when compared to conventional risk models, potentially reducing the number of unnecessary invasive procedures and delays in diagnosis.This publication has 47 references indexed in Scilit:
- Probability of Cancer in Pulmonary Nodules Detected on First Screening CTThe New England Journal of Medicine, 2013
- Measurement Methods and Algorithms for the Management of Solid NodulesJournal of Thoracic Imaging, 2012
- Radiomics: Extracting more information from medical images using advanced feature analysisEuropean Journal of Cancer, 2012
- Reduced Lung-Cancer Mortality with Low-Dose Computed Tomographic ScreeningThe New England Journal of Medicine, 2011
- 5-Year Lung Cancer Screening Experience: Growth Curves of 18 Lung Cancers Compared to Histologic Type, CT Attenuation, Stage, Survival, and SizeSocial psychiatry. Sozialpsychiatrie. Psychiatrie sociale, 2009
- Management of Lung Nodules Detected by Volume CT ScanningThe New England Journal of Medicine, 2009
- Smooth or Attached Solid Indeterminate Nodules Detected at Baseline CT Screening in the NELSON Study: Cancer Risk during 1 Year of Follow-upRadiology, 2009
- Pivotal Evaluation of the Accuracy of a Biomarker Used for Classification or Prediction: Standards for Study DesignJNCI Journal of the National Cancer Institute, 2008
- A Clinical Model To Estimate the Pretest Probability of Lung Cancer in Patients With Solitary Pulmonary NodulesSocial psychiatry. Sozialpsychiatrie. Psychiatrie sociale, 2007
- Estimating misclassification error with small samples via bootstrap cross-validationBioinformatics, 2005