Multimodal fusion of structural and functional brain imaging data

Jing Sui, Vince Daniel Calhoun

Research output: Chapter in Book/Report/Conference proceedingChapter

2 Scopus citations


Recent years have witnessed a rapid growth of interest in moving functional magnetic resonance imaging (fMRI) beyond simple scan-length averages and into approaches that can integrate structural MRI measures and capture rich multimodal interactions. It is becoming increasingly clear that multimodal fusion is able to provide more information for individual subjects by exploiting covariation between modalities, rather an analysis of each modality alone. Multimodal fusion is a more complicated endeavor that must be approached carefully and efficient methods should be developed to draw generalized and valid conclusions out of high dimensional data with a limited number of subjects, such as patients with brain disorders. Numerous research efforts have been reported in the field based on various statistical models, including independent component analysis (ICA), canonical correlation analysis (CCA), and partial least squares (PLS). In this chapter, we survey a number of methods previously shown in multimodal fusion reports, performed with or without prior information, and with their possible strengths and limitations addressed. To examine the function-structure associations of the brain in a more comprehensive and integrated manner, we also reviewed a number of multimodal studies that combined fMRI and structural (sMRI and/or diffusion tensor MRI) measures, which could reveal important brain alterations that may not be fully detected by employing separate analysis of individual modalities, and also enable us to identify potential brain illness biomarkers.

Original languageEnglish (US)
Title of host publicationNeuromethods
PublisherHumana Press Inc.
Number of pages17
StatePublished - Sep 1 2016
Externally publishedYes

Publication series

ISSN (Print)08932336
ISSN (Electronic)19406045


  • Canonical correlation analysis
  • Data driven
  • Diffusion MRI
  • Functional magnetic resonance imaging
  • Independent component analysis
  • Multimodal fusion methods
  • Structural MRI

ASJC Scopus subject areas

  • Neuroscience(all)
  • Biochemistry, Genetics and Molecular Biology(all)
  • Pharmacology, Toxicology and Pharmaceutics(all)
  • Psychiatry and Mental health


Dive into the research topics of 'Multimodal fusion of structural and functional brain imaging data'. Together they form a unique fingerprint.

Cite this