Recognizing surgical activities with recurrent neural networks

Robert Dipietro, Colin Lea, Anand Malpani, Narges Ahmidi, S. Swaroop Vedula, Gyusung I. Lee, Mija R. Lee, Gregory D. Hager

Research output: Chapter in Book/Report/Conference proceedingConference contribution

60 Scopus citations

Abstract

We apply recurrent neural networks to the task of recognizing surgical activities from robot kinematics. Prior work in this area focuses on recognizing short,low-level activities,or gestures,and has been based on variants of hidden Markov models and conditional random fields. In contrast,we work on recognizing both gestures and longer,higher-level activites,or maneuvers,and we model the mapping from kinematics to gestures/maneuvers with recurrent neural networks. To our knowledge,we are the first to apply recurrent neural networks to this task. Using a single model and a single set of hyperparameters,we match state-of-the-art performance for gesture recognition and advance state-of-the-art performance for maneuver recognition,in terms of both accuracy and edit distance. Code is available at https://github.com/ rdipietro/miccai-2016-surgical-activity-rec.

Original languageEnglish (US)
Title of host publicationMedical Image Computing and Computer-Assisted Intervention - MICCAI 2016 - 19th International Conference, Proceedings
EditorsSebastian Ourselin, Leo Joskowicz, Mert R. Sabuncu, William Wells, Gozde Unal
PublisherSpringer Verlag
Pages551-558
Number of pages8
ISBN (Print)9783319467191
DOIs
StatePublished - 2016
Event1st International Workshop on Simulation and Synthesis in Medical Imaging, SASHIMI 2016 held in conjunction with 19th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2016 - Athens, Greece
Duration: Oct 21 2016Oct 21 2016

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume9900 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other1st International Workshop on Simulation and Synthesis in Medical Imaging, SASHIMI 2016 held in conjunction with 19th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2016
Country/TerritoryGreece
CityAthens
Period10/21/1610/21/16

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint

Dive into the research topics of 'Recognizing surgical activities with recurrent neural networks'. Together they form a unique fingerprint.

Cite this