Multimodal integration

Research output: Chapter in Book/Report/Conference proceedingChapter

3 Citations (Scopus)

Abstract

Multimodal machine learning is a recent yet fast-growing topic in brain disorders research. The aim is to capitalize on the complementary nature of different modalities to build better prediction models for psychiatric and neurologic disorders. In this chapter, we focus on three groups of methods for multimodal integration that differ with respect to the stage in which they are implemented along the machine learning pipeline: early, intermediate, or late. We first describe data fusion as an early data integration method. This is an emerging and promising approach that involves the integration of the multiple modalities at an early stage of the pipeline (e.g., feature extraction stage). We then discuss kernel-based methods and deep learning as intermediate data integration methods. Finally, we discuss ensemble methods as examples of late data integration methods. In the last section, we present some exemplar studies that have used multimodal machine learning to investigate brain disorders.
Original languageEnglish
Title of host publicationMachine Learning
Subtitle of host publicationMethods and Applications to Brain Disorders
PublisherElsevier
Pages283-305
Number of pages23
ISBN (Electronic)9780128157398
ISBN (Print)9780128157398
DOIs
Publication statusPublished - 1 Jan 2019

Keywords

  • Alzheimer’s disease
  • Brain disorders
  • Data fusion
  • Ensemble
  • Machine learning
  • Multi-kernel learning
  • Multimodal
  • Neuroimaging
  • Psychosis
  • Sum of kernels

Cite this