Online citations, reference lists, and bibliographies.

Development And Validation Of A Deep Learning Algorithm For Detection Of Diabetic Retinopathy In Retinal Fundus Photographs.

Varun Gulshan, L. Peng, Marc Coram, Martin C. Stumpe, D. Wu, A. Narayanaswamy, Subhashini Venugopalan, Kasumi Widner, T. Madams, J. Cuadros, R. Kim, R. Raman, P. Nelson, J. Mega, D. R. Webster
Published 2016 · Medicine

Cite This
Download PDF
Analyze on Scholarcy
Share
Importance Deep learning is a family of computational methods that allow an algorithm to program itself by learning from a large set of examples that demonstrate the desired behavior, removing the need to specify rules explicitly. Application of these methods to medical imaging requires further assessment and validation. Objective To apply deep learning to create an algorithm for automated detection of diabetic retinopathy and diabetic macular edema in retinal fundus photographs. Design and Setting A specific type of neural network optimized for image classification called a deep convolutional neural network was trained using a retrospective development data set of 128 175 retinal images, which were graded 3 to 7 times for diabetic retinopathy, diabetic macular edema, and image gradability by a panel of 54 US licensed ophthalmologists and ophthalmology senior residents between May and December 2015. The resultant algorithm was validated in January and February 2016 using 2 separate data sets, both graded by at least 7 US board-certified ophthalmologists with high intragrader consistency. Exposure Deep learning-trained algorithm. Main Outcomes and Measures The sensitivity and specificity of the algorithm for detecting referable diabetic retinopathy (RDR), defined as moderate and worse diabetic retinopathy, referable diabetic macular edema, or both, were generated based on the reference standard of the majority decision of the ophthalmologist panel. The algorithm was evaluated at 2 operating points selected from the development set, one selected for high specificity and another for high sensitivity. Results The EyePACS-1 data set consisted of 9963 images from 4997 patients (mean age, 54.4 years; 62.2% women; prevalence of RDR, 683/8878 fully gradable images [7.8%]); the Messidor-2 data set had 1748 images from 874 patients (mean age, 57.6 years; 42.6% women; prevalence of RDR, 254/1745 fully gradable images [14.6%]). For detecting RDR, the algorithm had an area under the receiver operating curve of 0.991 (95% CI, 0.988-0.993) for EyePACS-1 and 0.990 (95% CI, 0.986-0.995) for Messidor-2. Using the first operating cut point with high specificity, for EyePACS-1, the sensitivity was 90.3% (95% CI, 87.5%-92.7%) and the specificity was 98.1% (95% CI, 97.8%-98.5%). For Messidor-2, the sensitivity was 87.0% (95% CI, 81.1%-91.0%) and the specificity was 98.5% (95% CI, 97.7%-99.1%). Using a second operating point with high sensitivity in the development set, for EyePACS-1 the sensitivity was 97.5% and specificity was 93.4% and for Messidor-2 the sensitivity was 96.1% and specificity was 93.9%. Conclusions and Relevance In this evaluation of retinal fundus photographs from adults with diabetes, an algorithm based on deep machine learning had high sensitivity and specificity for detecting referable diabetic retinopathy. Further research is necessary to determine the feasibility of applying this algorithm in the clinical setting and to determine whether use of the algorithm could lead to improved care and outcomes compared with current ophthalmologic assessment.
This paper references
10.1093/biomet/26.4.404
THE USE OF CONFIDENCE OR FIDUCIAL LIMITS ILLUSTRATED IN THE CASE OF THE BINOMIAL
C. J. Clopper (1934)
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
S. Ioffe (2015)
10.1016/j.media.2011.07.004
Exudate-based diabetic macular edema detection in fundus images using publicly available datasets
L. Giancardo (2012)
10.1109/TMI.2008.920619
Optimal Wavelet Transform for the Detection of Microaneurysms in Retina Photographs
G. Quellec (2008)
10.1109/CVPR.2016.308
Rethinking the Inception Architecture for Computer Vision
Christian Szegedy (2016)
10.1061/(ASCE)GT.1943-5606.0001284
ImageNet Classification with Deep Convolutional Neural Networks
Alex Krizhevsky (2012)
10.1056/NEJM199412013312206
Variability in radiologists' interpretations of mammograms.
J. Elmore (1994)
10.1145/3065386
ImageNet classification with deep convolutional neural networks
A. Krizhevsky (2017)
10.1136/bjo.2007.119453
The efficacy of automated “disease/no disease” grading for diabetic retinopathy in a systematic screening programme
S. Philip (2007)
10.1016/S0161-6420(99)00010-X
A screening approach to the surveillance of patients with diabetes for the presence of vision-threatening retinopathy.
G. Bresnick (2000)
10.1586/eop.12.52
Diabetic retinopathy management guidelines
R. Chakrabarti (2012)
10.1007/s11263-015-0816-y
ImageNet Large Scale Visual Recognition Challenge
Olga Russakovsky (2015)
10.1016/j.media.2012.06.003
A multiple-instance learning framework for diabetic retinopathy screening
G. Quellec (2012)
EyeArt: Automated, High-throughput, Image Analysis for Diabetic Retinopathy Screening
K. Solanki (2015)
Overfitting in Neural Nets: Backpropagation, Conjugate Gradient, and Early Stopping
R. Caruana (2000)
10.1001/jama.2015.1405
Diagnostic concordance among pathologists interpreting breast biopsy specimens.
J. Elmore (2015)
10.1142/S1793351X16500045
Deep Learning
X. Hao (2016)
10.1001/jamaophthalmol.2013.1743
Automated analysis of retinal images for detection of referable diabetic retinopathy.
M. Abràmoff (2013)
10.1001/jama.2010.1111
Prevalence of diabetic retinopathy in the United States, 2005-2008.
X. Zhang (2010)
10.1016/j.compbiomed.2013.10.007
Computer-aided diagnosis of diabetic retinopathy: A review
M. Mookiah (2013)
Prevalence of diabetic retinopathy in India: Sankara Nethralaya Diabetic Retinopathy Epidemiology and Molecular Genetics Study report
R Raman (2009)
FEEDBACK ON A PUBLICLY DISTRIBUTED IMAGE DATABASE: THE MESSIDOR DATABASE
Tienne (2014)
10.1034/J.1600-0420.2003.00004.X
Screening for diabetic retinopathy by non-ophthalmologists: an effective public health tool.
L. Verma (2003)
10.1016/j.ophtha.2008.09.010
Prevalence of diabetic retinopathy in India: Sankara Nethralaya Diabetic Retinopathy Epidemiology and Molecular Genetics Study report 2.
R. Raman (2009)
Large Scale Distributed Deep Networks
J. Dean (2012)



This paper is referenced by
10.1038/s41598-019-38491-0
DeepSOFA: A Continuous Acuity Score for Critically Ill Patients using Clinically Interpretable Deep Learning
Benjamin Shickel (2019)
10.1609/aaai.v33i01.33011093
Pathological Evidence Exploration in Deep Retinal Image Diagnosis
Yuhao Niu (2019)
10.1093/jamia/ocy017
Distributed deep learning networks among institutions for medical imaging
K. Chang (2018)
10.1148/radiol.2017162740
Immediate Allergic Reactions to Gadolinium-based Contrast Agents: A Systematic Review and Meta-Analysis.
A. Behzadi (2018)
10.1161/CIRCIMAGING.117.005614
Machine Learning Approaches in Cardiovascular Imaging
Mir Henglin (2017)
10.1364/BOE.9.005353
Automated identification of cone photoreceptors in adaptive optics optical coherence tomography images using transfer learning.
Morgan Heisler (2018)
10.3389/frai.2020.00072
Automated Assessment of Peristomal Skin Discoloration and Leakage Area Using Artificial Intelligence
N. K. Andersen (2020)
10.1503/cmaj.170955
Laying the digital and analytical foundations for Canada’s future health care system
M. Mamdani (2018)
10.1080/02713683.2017.1297463
Reaching the Unreachable: Novel Approaches to Telemedicine Screening of Underserved Populations for Vitreoretinal Disease
A. Murchison (2017)
10.1111/jdi.13210
Intra‐individual association between C‐reactive protein and insulin administration in postoperative lumbar spinal canal stenosis patients: A retrospective cohort study
Ken Kurisu (2020)
10.1073/pnas.1717139115
Predicting cancer outcomes from histology and genomics using convolutional networks
Pooya Mobadersany (2018)
10.23919/MVA.2019.8757991
EyeWeS: Weakly Supervised Pre-Trained Convolutional Neural Networks for Diabetic Retinopathy Detection
P. Costa (2019)
10.1109/ICHI.2018.00095
Interpretable Machine Learning in Healthcare
M. Ahmad (2018)
10.1016/j.crad.2017.11.015
Artificial intelligence in fracture detection: transfer learning from deep convolutional neural networks.
D. H. Kim (2018)
10.1038/s41551-020-0577-y
Dense anatomical annotation of slit-lamp images improves the performance of deep learning for the diagnosis of ophthalmic disorders
Wangting Li (2020)
Antibody Incubation Dye Bleaching Nuclear Staining Imaging n Channels Registration & Stitching Image Segmentation Feature Extraction Facetto Analysis
R. Krueger (2019)
10.1016/j.surg.2018.06.022
Big data: More than big data sets.
Adrienne N. Cobb (2018)
10.1016/j.compbiomed.2017.09.005
Automated arteriole and venule classification using deep learning for retinal images from the UK Biobank cohort
R. A. Welikala (2017)
10.1109/cvpr42600.2020.00970
Attribution in Scale and Space
Shawn Z. Xu (2020)
3D-RADNet: Extracting labels from DICOM metadata for training general medical domain deep 3D convolution neural networks
R. Du (2020)
10.1016/j.ajo.2019.11.006
Human versus Machine: Comparing a Deep Learning Algorithm to Human Gradings for Detecting Glaucoma on Fundus Photographs.
A. Jammal (2019)
10.1364/boe.379551
Noise reduction in optical coherence tomography images using a deep neural network with perceptually-sensitive loss function.
Bin Qiu (2020)
X-ToM: Explaining with Theory-of-Mind for Gaining Justified Human Trust
Arjun Reddy Akula (2019)
10.3390/sym12050721
AResU-Net: Attention Residual U-Net for Brain Tumor Segmentation
Jianxin Zhang (2020)
10.1016/j.preteyeres.2019.04.003
Deep learning in ophthalmology: The technical and clinical considerations
D. Ting (2019)
Training Artificial Neural Networks by Generalized Likelihood Ratio Method: Exploring Brain-like Learning to Improve Adversarial Defensiveness
Li Xiao (2019)
10.1038/s41598-018-37925-5
Comparison of different smartphone cameras to evaluate conjunctival hyperaemia in normal subjects
Carles Otero (2019)
10.3390/APP8112178
Adipocyte Size Evaluation Based on Photoacoustic Spectral Analysis Combined with Deep Learning Method
X. Ma (2018)
10.3390/jcm9061839
Deep Learning for Automated Detection of Cyst and Tumors of the Jaw in Panoramic Radiographs
Hyunwoo Yang (2020)
10.1055/S-0043-115900
Rahmenbedingungen zur Sammlung von "Real-Life"-Daten am Beispiel der Augenklinik der Universität München
Karsten Kortuem (2017)
10.1016/j.jacr.2017.12.037
Artificial Intelligence and Radiology: Collaboration Is Key.
P. Yi (2018)
10.1007/978-3-319-96136-1_25
Long Short-Term Memory Recurrent Neural Network for Stroke Prediction
Pattanapong Chantamit-o-pas (2018)
See more
Semantic Scholar Logo Some data provided by SemanticScholar