Surgical gesture segmentation and recognition

Lingling Tao, Luca Zappella, Gregory D. Hager, René Vidal

Research output: Chapter in Book/Report/Conference proceedingConference contribution

64 Scopus citations


Automatic surgical gesture segmentation and recognition can provide useful feedback for surgical training in robotic surgery. Most prior work in this field relies on the robot's kinematic data. Although recent work [1,2] shows that the robot's video data can be equally effective for surgical gesture recognition, the segmentation of the video into gestures is assumed to be known. In this paper, we propose a framework for joint segmentation and recognition of surgical gestures from kinematic and video data. Unlike prior work that relies on either frame-level kinematic cues, or segment-level kinematic or video cues, our approach exploits both cues by using a combined Markov/semi-Markov conditional random field (MsM-CRF) model. Our experiments show that the proposed model improves over a Markov or semi-Markov CRF when using video data alone, gives results that are comparable to state-of-the-art methods on kinematic data alone, and improves over state-of-the-art methods when combining kinematic and video data.

Original languageEnglish (US)
Title of host publicationMedical Image Computing and Computer-Assisted Intervention, MICCAI 2013 - 16th International Conference, Proceedings
Number of pages8
EditionPART 3
StatePublished - 2013
Event16th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2013 - Nagoya, Japan
Duration: Sep 22 2013Sep 26 2013

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
NumberPART 3
Volume8151 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Other16th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2013


  • conditional random fields
  • structured output learning
  • surgical gesture recognition
  • surgical gesture segmentation
  • time series analysis

ASJC Scopus subject areas

  • Theoretical Computer Science
  • General Computer Science


Dive into the research topics of 'Surgical gesture segmentation and recognition'. Together they form a unique fingerprint.

Cite this