ACMTF for fusion of multi-modal neuroimaging data and identification of biomarkers

Evrim Acar, Yuri Levin-Schwartz, Vince D. Calhoun, Tulay Adali

Research output: Chapter in Book/Report/Conference proceedingConference contribution

10 Scopus citations


Joint analysis of neuroimaging data from multiple modalities has the potential to improve our understanding of brain function since each modality provides complementary information. In this paper, we address the problem of jointly analyzing functional magnetic resonance imaging (fMRI), structural MRI (sMRI) and electroencephalography (EEG) data collected during an auditory oddball (AOD) task with the goal of capturing neural patterns that differ between patients with schizophrenia and healthy controls. Traditionally, fusion methods such as joint independent component analysis (jICA) have been used to jointly analyze such multi-modal neuroimaging data. However, previous jICA analyses typically analyze the EEG signal from a single electrode or concatenate signals from multiple electrodes, thus ignoring the potential multilinear structure of the EEG data, and models the data using a common mixing matrix for both modalities. In this paper, we arrange the multi-channel EEG signals as a third-order tensor with modes: subjects, time samples and electrodes, and jointly analyze the tensor with the fMRI and sMRI data, both in the form of subjects by voxels matrices, using a structure-revealing coupled matrix and tensor factorization (CMTF) model. Through this modeling approach, we (i) exploit the multilinear structure of multi-channel EEG data and (ii) capture weights for components indicative of the level of contribution from each modality. We compare the results of the structurerevealing CMTF model with those of jICA and demonstrate that, while both models capture significant distinguishing patterns between patients and controls, the structure-revealing CMTF model provides more robust activation.

Original languageEnglish (US)
Title of host publication25th European Signal Processing Conference, EUSIPCO 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages5
ISBN (Electronic)9780992862671
StatePublished - Oct 23 2017
Event25th European Signal Processing Conference, EUSIPCO 2017 - Kos, Greece
Duration: Aug 28 2017Sep 2 2017

Publication series

Name25th European Signal Processing Conference, EUSIPCO 2017


Other25th European Signal Processing Conference, EUSIPCO 2017

ASJC Scopus subject areas

  • Signal Processing


Dive into the research topics of 'ACMTF for fusion of multi-modal neuroimaging data and identification of biomarkers'. Together they form a unique fingerprint.

Cite this