TY - GEN
T1 - Learning Representations of Endoscopic Videos to Detect Tool Presence Without Supervision
AU - Li, David Z.
AU - Ishii, Masaru
AU - Taylor, Russell H.
AU - Hager, Gregory D.
AU - Sinha, Ayushi
N1 - Publisher Copyright:
© 2020, Springer Nature Switzerland AG.
PY - 2020
Y1 - 2020
N2 - In this work, we explore whether it is possible to learn representations of endoscopic video frames to perform tasks such as identifying surgical tool presence without supervision. We use a maximum mean discrepancy (MMD) variational autoencoder (VAE) to learn low-dimensional latent representations of endoscopic videos and manipulate these representations to distinguish frames containing tools from those without tools. We use three different methods to manipulate these latent representations in order to predict tool presence in each frame. Our fully unsupervised methods can identify whether endoscopic video frames contain tools with average precision of 71.56, 73.93, and 76.18, respectively, comparable to supervised methods. Our code is available at https://github.com/zdavidli/tool-presence/.
AB - In this work, we explore whether it is possible to learn representations of endoscopic video frames to perform tasks such as identifying surgical tool presence without supervision. We use a maximum mean discrepancy (MMD) variational autoencoder (VAE) to learn low-dimensional latent representations of endoscopic videos and manipulate these representations to distinguish frames containing tools from those without tools. We use three different methods to manipulate these latent representations in order to predict tool presence in each frame. Our fully unsupervised methods can identify whether endoscopic video frames contain tools with average precision of 71.56, 73.93, and 76.18, respectively, comparable to supervised methods. Our code is available at https://github.com/zdavidli/tool-presence/.
KW - Endoscopic video
KW - Maximum mean discrepancy
KW - Representation learning
KW - Tool presence
KW - Variational autoencoder
UR - http://www.scopus.com/inward/record.url?scp=85092616710&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85092616710&partnerID=8YFLogxK
U2 - 10.1007/978-3-030-60946-7_6
DO - 10.1007/978-3-030-60946-7_6
M3 - Conference contribution
AN - SCOPUS:85092616710
SN - 9783030609450
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 54
EP - 63
BT - Multimodal Learning for Clinical Decision Support and Clinical Image-Based Procedures - 10th International Workshop, ML-CDS 2020, and 9th International Workshop, CLIP 2020, Held in Conjunction with MICCAI 2020, Proceedings
A2 - Syeda-Mahmood, Tanveer
A2 - Drechsler, Klaus
A2 - Greenspan, Hayit
A2 - Madabhushi, Anant
A2 - Karargyris, Alexandros
A2 - Oyarzun Laura, Cristina
A2 - Wesarg, Stefan
A2 - Linguraru, Marius George
A2 - Shekhar, Raj
A2 - Erdt, Marius
A2 - González Ballester, Miguel Ángel
PB - Springer Science and Business Media Deutschland GmbH
T2 - 10th International Workshop on Multimodal Learning for Clinical Decision Support, ML-CDS 2020, and the 9th International Workshop on Clinical Image-Based Procedures, CLIP 2020, held in conjunction with the 23rd International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2020
Y2 - 4 October 2020 through 8 October 2020
ER -