Technical and imaging factors influencing performance of deep learning systems for diabetic retinopathy
Open Access
- 23 March 2020
- journal article
- research article
- Published by Springer Science and Business Media LLC in npj Digital Medicine
- Vol. 3 (1), 1-12
- https://doi.org/10.1038/s41746-020-0247-1
Abstract
Deep learning (DL) has been shown to be effective in developing diabetic retinopathy (DR) algorithms, possibly tackling financial and manpower challenges hindering implementation of DR screening. However, our systematic review of the literature reveals few studies studied the impact of different factors on these DL algorithms, that are important for clinical deployment in real-world settings. Using 455,491 retinal images, we evaluated two technical and three image-related factors in detection of referable DR. For technical factors, the performances of four DL models (VGGNet, ResNet, DenseNet, Ensemble) and two computational frameworks (Caffe, TensorFlow) were evaluated while for image-related factors, we evaluated image compression levels (reducing image size, 350, 300, 250, 200, 150 KB), number of fields (7-field, 2-field, 1-field) and media clarity (pseudophakic vs phakic). In detection of referable DR, four DL models showed comparable diagnostic performance (AUC 0.936-0.944). To develop the VGGNet model, two computational frameworks had similar AUC (0.936). The DL performance dropped when image size decreased below 250 KB (AUC 0.936, 0.900, p < 0.001). The DL performance performed better when there were increased number of fields (dataset 1: 2-field vs 1-field-AUC 0.936 vs 0.908, p < 0.001; dataset 2: 7-field vs 2-field vs 1-field, AUC 0.949 vs 0.911 vs 0.895). DL performed better in the pseudophakic than phakic eyes (AUC 0.918 vs 0.833, p < 0.001). Various image-related factors play more significant roles than technical factors in determining the diagnostic performance, suggesting the importance of having robust training and testing datasets for DL training and deployment in the real-world settings.This publication has 56 references indexed in Scilit:
- GRADING DIABETIC RETINOPATHY SEVERITY FROM COMPRESSED DIGITAL RETINAL IMAGES COMPARED WITH UNCOMPRESSED IMAGES AND FILMRetina, 2010
- Diabetic retinopathyThe Lancet, 2010
- Prevalence and Risk Factors for Diabetic Retinopathy: The Singapore Malay Eye StudyOphthalmology, 2008
- Single-field fundus photography for diabetic retinopathy screening: A report by the American Academy of OphthalmologyOphthalmology, 2004
- Effect of digital image compression on screening for diabetic retinopathyBritish Journal of Ophthalmology, 2001
- Vascular lesions in diabetes are distributed non-uniformly within the retinaExperimental Eye Research, 1995
- How Effective Are Treatments for Diabetic Retinopathy?JAMA, 1993
- Grading Diabetic Retinopathy from Stereoscopic Color Fundus Photographs—An Extension of the Modified Airlie House ClassificationOphthalmology, 1991
- Neural network ensemblesIEEE Transactions on Pattern Analysis and Machine Intelligence, 1990
- Simple diabetic retinopathy. Evolution of the lesions and therapeutic considerations.British Journal of Ophthalmology, 1970