TY - JOUR
T1 - Deep Learning System Outperforms Clinicians in Identifying Optic Disc Abnormalities
AU - BONSAI (Brain and Optic Nerve Study With Artificial Intelligence) Group
AU - Vasseneix, Caroline
AU - Nusinovici, Simon
AU - Xu, Xinxing
AU - Hwang, Jeong Min
AU - Hamann, Steffen
AU - Chen, John J.
AU - Loo, Jing Liang
AU - Milea, Leonard
AU - Tan, Kenneth B.K.
AU - Ting, Daniel S.W.
AU - Liu, Yong
AU - Newman, Nancy J.
AU - Biousse, Valerie
AU - Wong, Tien Ying
AU - Milea, Dan
AU - Najjar, Raymond P.
AU - Gohier, Philippe
AU - Miller, Neil
AU - Vanikieti, Kavin
AU - La Morgia, Chiara
AU - Rougier, Marie Bénédicte
AU - Ambika, Selvakumar
AU - Fonseca, Pedro
AU - Lagrèze, Wolf Alexander
AU - Sanda, Nicolae
AU - Chiquet, Christophe
AU - Yang, Hui
AU - Chan, Carmen K.M.
AU - Cheung, Carol Y.
AU - Chau, Tran Thi Ha
AU - Jurkute, Neringa
AU - Yu-Wai-Man, Patrick
AU - Kho, Richard
AU - Jonas, Jost B.
AU - Vignal-Clermont, Catherine
AU - Kim, Dong Hyun
AU - Yang, Hee Kyung
AU - Aung, Tin
AU - Singhal, Shweta
AU - Tow, Sharon
AU - Nongpiur, Monisha Esther
AU - Perera, Shamira
AU - Narayanaswamy, Arun
AU - Thirugnanam, Umapathi N.
AU - Fraser, Clare L.
AU - Mejico, Luis J.
AU - Fard, Masoud Aghsaei
N1 - Publisher Copyright:
© 2023 by North American Neuro-Ophthalmology Society.
PY - 2023/6/1
Y1 - 2023/6/1
N2 - Background: The examination of the optic nerve head (optic disc) is mandatory in patients with headache, hypertension, or any neurological symptoms, yet it is rarely or poorly performed in general clinics. We recently developed a brain and optic nerve study with artificial intelligence-deep learning system (BONSAI-DLS) capable of accurately detecting optic disc abnormalities including papilledema (swelling due to elevated intracranial pressure) on digital fundus photographs with a comparable classification performance to expert neuro-ophthalmologists, but its performance compared to first-line clinicians remains unknown. Methods: In this international, cross-sectional multicenter study, the DLS, trained on 14,341 fundus photographs, was tested on a retrospectively collected convenience sample of 800 photographs (400 normal optic discs, 201 papilledema and 199 other abnormalities) from 454 patients with a robust ground truth diagnosis provided by the referring expert neuro-ophthalmologists. The areas under the receiver-operating-characteristic curves were calculated for the BONSAI-DLS. Error rates, accuracy, sensitivity, and specificity of the algorithm were compared with those of 30 clinicians with or without ophthalmic training (6 general ophthalmologists, 6 optometrists, 6 neurologists, 6 internists, 6 emergency department [ED] physicians) who graded the same testing set of images. Results: With an error rate of 15.3%, the DLS outperformed all clinicians (average error rates 24.4%, 24.8%, 38.2%, 44.8%, 47.9% for general ophthalmologists, optometrists, neurologists, internists and ED physicians, respectively) in the overall classification of optic disc appearance. The DLS displayed significantly higher accuracies than 100%, 86.7% and 93.3% of clinicians (n = 30) for the classification of papilledema, normal, and other disc abnormalities, respectively. Conclusions: The performance of the BONSAI-DLS to classify optic discs on fundus photographs was superior to that of clinicians with or without ophthalmic training. A trained DLS may offer valuable diagnostic aid to clinicians from various clinical settings for the screening of optic disc abnormalities harboring potentially sight- or life-threatening neurological conditions.
AB - Background: The examination of the optic nerve head (optic disc) is mandatory in patients with headache, hypertension, or any neurological symptoms, yet it is rarely or poorly performed in general clinics. We recently developed a brain and optic nerve study with artificial intelligence-deep learning system (BONSAI-DLS) capable of accurately detecting optic disc abnormalities including papilledema (swelling due to elevated intracranial pressure) on digital fundus photographs with a comparable classification performance to expert neuro-ophthalmologists, but its performance compared to first-line clinicians remains unknown. Methods: In this international, cross-sectional multicenter study, the DLS, trained on 14,341 fundus photographs, was tested on a retrospectively collected convenience sample of 800 photographs (400 normal optic discs, 201 papilledema and 199 other abnormalities) from 454 patients with a robust ground truth diagnosis provided by the referring expert neuro-ophthalmologists. The areas under the receiver-operating-characteristic curves were calculated for the BONSAI-DLS. Error rates, accuracy, sensitivity, and specificity of the algorithm were compared with those of 30 clinicians with or without ophthalmic training (6 general ophthalmologists, 6 optometrists, 6 neurologists, 6 internists, 6 emergency department [ED] physicians) who graded the same testing set of images. Results: With an error rate of 15.3%, the DLS outperformed all clinicians (average error rates 24.4%, 24.8%, 38.2%, 44.8%, 47.9% for general ophthalmologists, optometrists, neurologists, internists and ED physicians, respectively) in the overall classification of optic disc appearance. The DLS displayed significantly higher accuracies than 100%, 86.7% and 93.3% of clinicians (n = 30) for the classification of papilledema, normal, and other disc abnormalities, respectively. Conclusions: The performance of the BONSAI-DLS to classify optic discs on fundus photographs was superior to that of clinicians with or without ophthalmic training. A trained DLS may offer valuable diagnostic aid to clinicians from various clinical settings for the screening of optic disc abnormalities harboring potentially sight- or life-threatening neurological conditions.
UR - http://www.scopus.com/inward/record.url?scp=85159737109&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85159737109&partnerID=8YFLogxK
U2 - 10.1097/WNO.0000000000001800
DO - 10.1097/WNO.0000000000001800
M3 - Article
C2 - 36719740
AN - SCOPUS:85159737109
SN - 1070-8022
VL - 43
SP - 159
EP - 167
JO - Journal of Neuro-Ophthalmology
JF - Journal of Neuro-Ophthalmology
IS - 2
ER -