Inside-out tracking and projection mapping for robot-assisted transcranial magnetic stimulation

Yihao Liu; Shuya Joshua Liu; Shahriar Sefati; Tian Jing; Amir Kheradmand; Mehran Armand

doi:10.1117/12.2614245

Inside-out tracking and projection mapping for robot-assisted transcranial magnetic stimulation

Yihao Liu, Shuya Joshua Liu, Shahriar Sefati, Tian Jing, Amir Kheradmand, Mehran Armand

School of Medicine

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Transcranial Magnetic Stimulation (TMS) is a neurostimulation technique based on the principle of electromagnetic induction of an electric field in the brain with both research and clinical applications. To produce an optimal neuro-modulatory effect, TMS coil must be placed on the head and oriented accurately with respect to the region of interest within the brain. A robotic method can enhance the accuracy and facilitate the procedure for TMS coil placement. This work presents two system improvements for robot-assisted TMS (RA-TMS) application. Previous systems have used outside-in tracking method where a stationary external infrared (IR) tracker is used as a reference point to track the head and TMS coil positions. This method is prone to losing track of the coil or the head if the IR camera is blocked by the robotic arm during its motion. To address this issue, we implemented an inside-out tracking method by mounting a portable IR camera on the robot end-effector. This method guarantees that the line of sight of the IR camera is not obscured by the robotic arm at any time during its motion. We also integrated a portable projection mapping device (PPMD) into the RA-TMS system to provide visual guidance during TMS application. PPMD can track the head via an IR tracker, and can project a planned contact point of TMS coil on the head or overlay the underlying brain anatomy in real-time.

Original language	English (US)
Title of host publication	Optical Architectures for Displays and Sensing in Augmented, Virtual, and Mixed Reality (AR, VR, MR) III
Editors	Bernard C. Kress, Christophe Peroz
Publisher	SPIE
ISBN (Electronic)	9781510647329
DOIs	https://doi.org/10.1117/12.2614245
State	Published - 2022
Event	Optical Architectures for Displays and Sensing in Augmented, Virtual, and Mixed Reality (AR, VR, MR) III 2022 - San Francisco, United States Duration: Jan 23 2022 → Jan 25 2022

Publication series

Name	Progress in Biomedical Optics and Imaging - Proceedings of SPIE
Volume	11931
ISSN (Print)	1605-7422

Conference

Conference	Optical Architectures for Displays and Sensing in Augmented, Virtual, and Mixed Reality (AR, VR, MR) III 2022
Country/Territory	United States
City	San Francisco
Period	1/23/22 → 1/25/22

Keywords

optical tracker
projection mapping
robot-assisted TMS

ASJC Scopus subject areas

Electronic, Optical and Magnetic Materials
Atomic and Molecular Physics, and Optics
Biomaterials
Radiology Nuclear Medicine and imaging

Access to Document

10.1117/12.2614245

Cite this

Liu, Y., Liu, S. J., Sefati, S., Jing, T., Kheradmand, A., & Armand, M. (2022). Inside-out tracking and projection mapping for robot-assisted transcranial magnetic stimulation. In B. C. Kress, & C. Peroz (Eds.), Optical Architectures for Displays and Sensing in Augmented, Virtual, and Mixed Reality (AR, VR, MR) III Article 1193108 (Progress in Biomedical Optics and Imaging - Proceedings of SPIE; Vol. 11931). SPIE. https://doi.org/10.1117/12.2614245

Inside-out tracking and projection mapping for robot-assisted transcranial magnetic stimulation. / Liu, Yihao; Liu, Shuya Joshua; Sefati, Shahriar et al.
Optical Architectures for Displays and Sensing in Augmented, Virtual, and Mixed Reality (AR, VR, MR) III. ed. / Bernard C. Kress; Christophe Peroz. SPIE, 2022. 1193108 (Progress in Biomedical Optics and Imaging - Proceedings of SPIE; Vol. 11931).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Liu, Y, Liu, SJ, Sefati, S, Jing, T , Kheradmand, A & Armand, M 2022, Inside-out tracking and projection mapping for robot-assisted transcranial magnetic stimulation. in BC Kress & C Peroz (eds), Optical Architectures for Displays and Sensing in Augmented, Virtual, and Mixed Reality (AR, VR, MR) III., 1193108, Progress in Biomedical Optics and Imaging - Proceedings of SPIE, vol. 11931, SPIE, Optical Architectures for Displays and Sensing in Augmented, Virtual, and Mixed Reality (AR, VR, MR) III 2022, San Francisco, United States, 1/23/22. https://doi.org/10.1117/12.2614245

Liu Y, Liu SJ, Sefati S, Jing T , Kheradmand A, Armand M. Inside-out tracking and projection mapping for robot-assisted transcranial magnetic stimulation. In Kress BC, Peroz C, editors, Optical Architectures for Displays and Sensing in Augmented, Virtual, and Mixed Reality (AR, VR, MR) III. SPIE. 2022. 1193108. (Progress in Biomedical Optics and Imaging - Proceedings of SPIE). doi: 10.1117/12.2614245

Liu, Yihao ; Liu, Shuya Joshua ; Sefati, Shahriar et al. / Inside-out tracking and projection mapping for robot-assisted transcranial magnetic stimulation. Optical Architectures for Displays and Sensing in Augmented, Virtual, and Mixed Reality (AR, VR, MR) III. editor / Bernard C. Kress ; Christophe Peroz. SPIE, 2022. (Progress in Biomedical Optics and Imaging - Proceedings of SPIE).

@inproceedings{7d5372c02bca430196b4d2f2a7ced860,

title = "Inside-out tracking and projection mapping for robot-assisted transcranial magnetic stimulation",

abstract = "Transcranial Magnetic Stimulation (TMS) is a neurostimulation technique based on the principle of electromagnetic induction of an electric field in the brain with both research and clinical applications. To produce an optimal neuro-modulatory effect, TMS coil must be placed on the head and oriented accurately with respect to the region of interest within the brain. A robotic method can enhance the accuracy and facilitate the procedure for TMS coil placement. This work presents two system improvements for robot-assisted TMS (RA-TMS) application. Previous systems have used outside-in tracking method where a stationary external infrared (IR) tracker is used as a reference point to track the head and TMS coil positions. This method is prone to losing track of the coil or the head if the IR camera is blocked by the robotic arm during its motion. To address this issue, we implemented an inside-out tracking method by mounting a portable IR camera on the robot end-effector. This method guarantees that the line of sight of the IR camera is not obscured by the robotic arm at any time during its motion. We also integrated a portable projection mapping device (PPMD) into the RA-TMS system to provide visual guidance during TMS application. PPMD can track the head via an IR tracker, and can project a planned contact point of TMS coil on the head or overlay the underlying brain anatomy in real-time.",

keywords = "optical tracker, projection mapping, robot-assisted TMS",

author = "Yihao Liu and Liu, {Shuya Joshua} and Shahriar Sefati and Tian Jing and Amir Kheradmand and Mehran Armand",

note = "Funding Information: This work was supported by a grant from the National Institute of Deafness and Other Communication Disorders (R01DC018815). We thank Alejandro Martin-Gomez for comments and assistance with preparing a figure in the manuscript. Publisher Copyright: {\textcopyright} 2022 SPIE.; Optical Architectures for Displays and Sensing in Augmented, Virtual, and Mixed Reality (AR, VR, MR) III 2022 ; Conference date: 23-01-2022 Through 25-01-2022",

year = "2022",

doi = "10.1117/12.2614245",

language = "English (US)",

series = "Progress in Biomedical Optics and Imaging - Proceedings of SPIE",

publisher = "SPIE",

editor = "Kress, {Bernard C.} and Christophe Peroz",

booktitle = "Optical Architectures for Displays and Sensing in Augmented, Virtual, and Mixed Reality (AR, VR, MR) III",

}

TY - GEN

T1 - Inside-out tracking and projection mapping for robot-assisted transcranial magnetic stimulation

AU - Liu, Yihao

AU - Liu, Shuya Joshua

AU - Sefati, Shahriar

AU - Jing, Tian

AU - Kheradmand, Amir

AU - Armand, Mehran

N1 - Funding Information: This work was supported by a grant from the National Institute of Deafness and Other Communication Disorders (R01DC018815). We thank Alejandro Martin-Gomez for comments and assistance with preparing a figure in the manuscript. Publisher Copyright: © 2022 SPIE.

PY - 2022

Y1 - 2022

N2 - Transcranial Magnetic Stimulation (TMS) is a neurostimulation technique based on the principle of electromagnetic induction of an electric field in the brain with both research and clinical applications. To produce an optimal neuro-modulatory effect, TMS coil must be placed on the head and oriented accurately with respect to the region of interest within the brain. A robotic method can enhance the accuracy and facilitate the procedure for TMS coil placement. This work presents two system improvements for robot-assisted TMS (RA-TMS) application. Previous systems have used outside-in tracking method where a stationary external infrared (IR) tracker is used as a reference point to track the head and TMS coil positions. This method is prone to losing track of the coil or the head if the IR camera is blocked by the robotic arm during its motion. To address this issue, we implemented an inside-out tracking method by mounting a portable IR camera on the robot end-effector. This method guarantees that the line of sight of the IR camera is not obscured by the robotic arm at any time during its motion. We also integrated a portable projection mapping device (PPMD) into the RA-TMS system to provide visual guidance during TMS application. PPMD can track the head via an IR tracker, and can project a planned contact point of TMS coil on the head or overlay the underlying brain anatomy in real-time.

AB - Transcranial Magnetic Stimulation (TMS) is a neurostimulation technique based on the principle of electromagnetic induction of an electric field in the brain with both research and clinical applications. To produce an optimal neuro-modulatory effect, TMS coil must be placed on the head and oriented accurately with respect to the region of interest within the brain. A robotic method can enhance the accuracy and facilitate the procedure for TMS coil placement. This work presents two system improvements for robot-assisted TMS (RA-TMS) application. Previous systems have used outside-in tracking method where a stationary external infrared (IR) tracker is used as a reference point to track the head and TMS coil positions. This method is prone to losing track of the coil or the head if the IR camera is blocked by the robotic arm during its motion. To address this issue, we implemented an inside-out tracking method by mounting a portable IR camera on the robot end-effector. This method guarantees that the line of sight of the IR camera is not obscured by the robotic arm at any time during its motion. We also integrated a portable projection mapping device (PPMD) into the RA-TMS system to provide visual guidance during TMS application. PPMD can track the head via an IR tracker, and can project a planned contact point of TMS coil on the head or overlay the underlying brain anatomy in real-time.

KW - optical tracker

KW - projection mapping

KW - robot-assisted TMS

UR - http://www.scopus.com/inward/record.url?scp=85131225635&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85131225635&partnerID=8YFLogxK

U2 - 10.1117/12.2614245

DO - 10.1117/12.2614245

M3 - Conference contribution

AN - SCOPUS:85131225635

T3 - Progress in Biomedical Optics and Imaging - Proceedings of SPIE

BT - Optical Architectures for Displays and Sensing in Augmented, Virtual, and Mixed Reality (AR, VR, MR) III

A2 - Kress, Bernard C.

A2 - Peroz, Christophe

PB - SPIE

T2 - Optical Architectures for Displays and Sensing in Augmented, Virtual, and Mixed Reality (AR, VR, MR) III 2022

Y2 - 23 January 2022 through 25 January 2022

ER -

Inside-out tracking and projection mapping for robot-assisted transcranial magnetic stimulation

Abstract

Publication series

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this