A 3D Generative Model of Pathological Multi-modal MR Images and Segmentations

Virginia Fernandez*, Walter Hugo Lopez Pinaya, Pedro Borges, Mark S. Graham, Tom Vercauteren, M. Jorge Cardoso

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference paperpeer-review

Abstract

Generative modelling and synthetic data can be a surrogate for real medical imaging datasets, whose scarcity and difficulty to share can be a nuisance when delivering accurate deep learning models for healthcare applications. In recent years, there has been an increased interest in using these models for data augmentation and synthetic data sharing, using architectures such as generative adversarial networks (GANs) or diffusion models (DMs). Nonetheless, the application of synthetic data to tasks such as 3D magnetic resonance imaging (MRI) segmentation remains limited due to the lack of labels associated with the generated images. Moreover, many of the proposed generative MRI models lack the ability to generate arbitrary modalities due to the absence of explicit contrast conditioning. These limitations prevent the user from adjusting the contrast and content of the images and obtaining more generalisable data for training task-specific models. In this work, we propose brainSPADE3D, a 3D generative model for brain MRI and associated segmentations, where the user can condition on specific pathological phenotypes and contrasts. The proposed joint imaging-segmentation generative model is shown to generate high-fidelity synthetic images and associated segmentations, with the ability to combine pathologies. We demonstrate how the model can alleviate issues with segmentation model performance when unexpected pathologies are present in the data.

Original languageEnglish
Title of host publicationDeep Generative Models - Third MICCAI Workshop, DGM4MICCAI 2023, Held in Conjunction with MICCAI 2023, Proceedings
EditorsAnirban Mukhopadhyay, Ilkay Oksuz, Sandy Engelhardt, Dajiang Zhu, Yixuan Yuan
PublisherSpringer Science and Business Media Deutschland GmbH
Pages132-142
Number of pages11
ISBN (Print)9783031537660
DOIs
Publication statusPublished - 2024
Event3rd Workshop on Deep Generative Models for Medical Image Computing and Computer Assisted Intervention, DGM4MICCAI 2023 Held in Conjunction with 26th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2023 - Vancouver, Canada
Duration: 8 Oct 202312 Oct 2023

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume14533 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference3rd Workshop on Deep Generative Models for Medical Image Computing and Computer Assisted Intervention, DGM4MICCAI 2023 Held in Conjunction with 26th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2023
Country/TerritoryCanada
CityVancouver
Period8/10/202312/10/2023

Fingerprint

Dive into the research topics of 'A 3D Generative Model of Pathological Multi-modal MR Images and Segmentations'. Together they form a unique fingerprint.

Cite this