TY - GEN
T1 - Learning Representations of Endoscopic Videos to Detect Tool Presence Without Supervision
AU - Li, David Z.
AU - Ishii, Masaru
AU - Taylor, Russell H.
AU - Hager, Gregory D.
AU - Sinha, Ayushi
N1 - Funding Information:
This work received the support of CAMed-Clinical additive manufacturing for medical applications (COMET K-Project 871132), which is funded by the Austrian Federal Ministry of Transport, Innovation and Technology (BMVIT), and the Austrian Federal Ministry for Digital and Economic Affairs (BMDW), and the Styrian Business Promotion Agency (SFG). Further, this work received funding from the Austrian Science Fund (FWF) KLI 678-B31 (enFaced-Virtual and Augmented Reality Training and Navigation Module for 3D-Printed Facial Defect Reconstructions) and the TU Graz Lead Project (Mechanics, Modeling and Simulation of Aortic Dissection). Moreover, we want to point out to our medical online framework Studierfenster (www.studierfenster.at) [17], where an automatic cranial implant design system has been incorporated. Finally, we thank the creators of the CQ500 dataset (http:// headctstudy.qure.ai/dataset).
Funding Information:
Acknowledgements. This work was supported by the Johns Hopkins University Provost’s Postdoctoral fellowship, NVIDIA GPU grant, and other Johns Hopkins University internal funds. We would also like to thank Daniel Malinsky and Robert DiPi-etro for their invaluable feedback. We would also like to acknowledge the JHU Department of Computer Science providing a research GPU cluster.
Publisher Copyright:
© 2020, Springer Nature Switzerland AG.
PY - 2020
Y1 - 2020
N2 - In this work, we explore whether it is possible to learn representations of endoscopic video frames to perform tasks such as identifying surgical tool presence without supervision. We use a maximum mean discrepancy (MMD) variational autoencoder (VAE) to learn low-dimensional latent representations of endoscopic videos and manipulate these representations to distinguish frames containing tools from those without tools. We use three different methods to manipulate these latent representations in order to predict tool presence in each frame. Our fully unsupervised methods can identify whether endoscopic video frames contain tools with average precision of 71.56, 73.93, and 76.18, respectively, comparable to supervised methods. Our code is available at https://github.com/zdavidli/tool-presence/.
AB - In this work, we explore whether it is possible to learn representations of endoscopic video frames to perform tasks such as identifying surgical tool presence without supervision. We use a maximum mean discrepancy (MMD) variational autoencoder (VAE) to learn low-dimensional latent representations of endoscopic videos and manipulate these representations to distinguish frames containing tools from those without tools. We use three different methods to manipulate these latent representations in order to predict tool presence in each frame. Our fully unsupervised methods can identify whether endoscopic video frames contain tools with average precision of 71.56, 73.93, and 76.18, respectively, comparable to supervised methods. Our code is available at https://github.com/zdavidli/tool-presence/.
KW - Endoscopic video
KW - Maximum mean discrepancy
KW - Representation learning
KW - Tool presence
KW - Variational autoencoder
UR - http://www.scopus.com/inward/record.url?scp=85092616710&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85092616710&partnerID=8YFLogxK
U2 - 10.1007/978-3-030-60946-7_6
DO - 10.1007/978-3-030-60946-7_6
M3 - Conference contribution
AN - SCOPUS:85092616710
SN - 9783030609450
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 54
EP - 63
BT - Multimodal Learning for Clinical Decision Support and Clinical Image-Based Procedures - 10th International Workshop, ML-CDS 2020, and 9th International Workshop, CLIP 2020, Held in Conjunction with MICCAI 2020, Proceedings
A2 - Syeda-Mahmood, Tanveer
A2 - Drechsler, Klaus
A2 - Greenspan, Hayit
A2 - Madabhushi, Anant
A2 - Karargyris, Alexandros
A2 - Oyarzun Laura, Cristina
A2 - Wesarg, Stefan
A2 - Linguraru, Marius George
A2 - Shekhar, Raj
A2 - Erdt, Marius
A2 - González Ballester, Miguel Ángel
PB - Springer Science and Business Media Deutschland GmbH
T2 - 10th International Workshop on Multimodal Learning for Clinical Decision Support, ML-CDS 2020, and the 9th International Workshop on Clinical Image-Based Procedures, CLIP 2020, held in conjunction with the 23rd International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2020
Y2 - 4 October 2020 through 8 October 2020
ER -