Man against machine: diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists
Top Cited Papers
Open Access
- 28 May 2018
- journal article
- research article
- Published by Elsevier BV in Annals of Oncology
- Vol. 29 (8), 1836-1842
- https://doi.org/10.1093/annonc/mdy166
Abstract
Deep learning convolutional neural networks (CNN) may facilitate melanoma detection, but data comparing a CNN’s diagnostic performance to larger groups of dermatologists are lacking. Google’s Inception v4 CNN architecture was trained and validated using dermoscopic images and corresponding diagnoses. In a comparative cross-sectional reader study a 100-image test-set was used (level-I: dermoscopy only; level-II: dermoscopy plus clinical information and images). Main outcome measures were sensitivity, specificity and area under the curve (AUC) of receiver operating characteristics (ROC) for diagnostic classification (dichotomous) of lesions by the CNN versus an international group of 58 dermatologists during level-I or -II of the reader study. Secondary end points included the dermatologists’ diagnostic performance in their management decisions and differences in the diagnostic performance of dermatologists during level-I and -II of the reader study. Additionally, the CNN’s performance was compared with the top-five algorithms of the 2016 International Symposium on Biomedical Imaging (ISBI) challenge. In level-I dermatologists achieved a mean (±standard deviation) sensitivity and specificity for lesion classification of 86.6% (±9.3%) and 71.3% (±11.2%), respectively. More clinical information (level-II) improved the sensitivity to 88.9% (±9.6%, P = 0.19) and specificity to 75.7% (±11.7%, P < 0.05). The CNN ROC curve revealed a higher specificity of 82.5% when compared with dermatologists in level-I (71.3%, P < 0.01) and level-II (75.7%, P < 0.01) at their sensitivities of 86.6% and 88.9%, respectively. The CNN ROC AUC was greater than the mean ROC area of dermatologists (0.86 versus 0.79, P < 0.01). The CNN scored results close to the top three algorithms of the ISBI 2016 challenge. For the first time we compared a CNN’s diagnostic performance with a large international group of 58 dermatologists, including 30 experts. Most dermatologists were outperformed by the CNN. Irrespective of any physicians’ experience, they may benefit from assistance by a CNN’s image classification. This study was registered at the German Clinical Trial Register (DRKS-Study-ID: DRKS00013570; https://www.drks.de/drks_web/).Keywords
This publication has 19 references indexed in Scilit:
- Emerging trends in the epidemiology of melanomaBritish Journal of Dermatology, 2014
- Segmentation of Skin Lesions From Digital Images Using Joint Statistical Texture DistinctivenessIEEE Transactions on Biomedical Engineering, 2014
- Variables Predicting Change in Benign Melanocytic Nevi Undergoing Short-term Dermoscopic ImagingArchives of Dermatology, 2011
- Impact of dermoscopy and short-term sequential digital dermoscopy imaging for the management of pigmented lesions in primary care: a sequential intervention trialBritish Journal of Dermatology, 2009
- Dermoscopy compared with naked eye examination for the diagnosis of primary melanoma: a meta-analysis of studies performed in a clinical settingBritish Journal of Dermatology, 2008
- Assessment of the Optimal Interval for and Sensitivity of Short-term Sequential Digital Dermoscopy Monitoring for the Diagnosis of MelanomaArchives of Dermatology, 2008
- Melanoma ScreeningArchives of Dermatology, 2007
- Comparative Performance of 4 Dermoscopic Algorithms by Nonexperts for the Diagnosis of Melanocytic LesionsArchives of Dermatology, 2005
- Is Dermoscopy (Epiluminescence Microscopy) Useful for the Diagnosis of Melanoma?Archives of Dermatology, 2001
- In vivo epiluminescence microscopy of pigmented skin lesions. I. Pattern analysis of pigmented skin lesionsJournal of the American Academy of Dermatology, 1987