Mesh U-Nets for 3D Cardiac Deformation Modeling

Marcel Beetz; Jorge Corral Acero; Abhirup Banerjee; Ingo Eitel; Ernesto Zacur; Torben Lange; Thomas Stiermaier; Ruben Evertz; Sören J. Backhaus; Holger Thiele; Alfonso Bueno-Orovio; Pablo Lamata; Andreas Schuster; Vicente Grau

doi:10.1007/978-3-031-23443-9_23

Mesh U-Nets for 3D Cardiac Deformation Modeling

Marcel Beetz^*, Jorge Corral Acero, Abhirup Banerjee, Ingo Eitel, Ernesto Zacur, Torben Lange, Thomas Stiermaier, Ruben Evertz, Sören J. Backhaus, Holger Thiele, Alfonso Bueno-Orovio, Pablo Lamata, Andreas Schuster, Vicente Grau

^*Corresponding author for this work

Biomedical Engineering Department

Research output: Chapter in Book/Report/Conference proceeding › Conference paper › peer-review

Abstract

During a cardiac cycle, the heart anatomy undergoes a series of complex 3D deformations, which can be analyzed to diagnose various cardiovascular pathologies including myocardial infarction. While volume-based metrics such as ejection fraction are commonly used in clinical practice to assess these deformations globally, they only provide limited information about localized changes in the 3D cardiac structures. The objective of this work is to develop a novel geometric deep learning approach to capture the mechanical deformation of complete 3D ventricular shapes, offering potential to discover new image-based biomarkers for cardiac disease diagnosis. To this end, we propose the mesh U-Net, which combines mesh-based convolution and pooling operations with U-Net-inspired skip connections in a hierarchical step-wise encoder-decoder architecture, in order to enable accurate and efficient learning directly on 3D anatomical meshes. The proposed network is trained to model both cardiac contraction and relaxation, that is, to predict the 3D cardiac anatomy at the end-systolic phase of the cardiac cycle based on the corresponding anatomy at end-diastole and vice versa. We evaluate our method on a multi-center cardiac magnetic resonance imaging (MRI) dataset of 1021 patients with acute myocardial infarction. We find mean surface distances between the predicted and gold standard anatomical meshes close to the pixel resolution of the underlying images and high similarity in multiple commonly used clinical metrics for both prediction directions. In addition, we show that the mesh U-Net compares favorably to a 3D U-Net benchmark by using 66% fewer network parameters and drastically smaller data sizes, while at the same time improving predictive performance by 14%. We also observe that the mesh U-Net is able to capture subpopulation-specific differences in mechanical deformation patterns between patients with different myocardial infarction types and clinical outcomes.

Original language	English
Title of host publication	Statistical Atlases and Computational Models of the Heart. Regular and CMRxMotion Challenge Papers - 13th International Workshop, STACOM 2022, Held in Conjunction with MICCAI 2022, Revised Selected Papers
Editors	Oscar Camara, Esther Puyol-Antón, Avan Suinesiaputra, Alistair Young, Chen Qin, Maxime Sermesant, Shuo Wang
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	245-257
Number of pages	13
ISBN (Print)	9783031234422
DOIs	https://doi.org/10.1007/978-3-031-23443-9_23
Publication status	Published - 2022
Event	13th International Workshop on Statistical Atlases and Computational Models of the Heart, STACOM 2022, held in conjunction with the 25th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2022 - Singapore, Singapore Duration: 18 Sept 2022 → 18 Sept 2022

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	13593 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	13th International Workshop on Statistical Atlases and Computational Models of the Heart, STACOM 2022, held in conjunction with the 25th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2022
Country/Territory	Singapore
City	Singapore
Period	18/09/2022 → 18/09/2022

Keywords

3D heart contraction
Acute myocardial infarction
Cardiac mechanics
Cardiac MRI
Geometric deep learning
Mesh sampling
Spectral graph convolutions

Access to Document

10.1007/978-3-031-23443-9_23

Cite this

Beetz, M., Acero, J. C., Banerjee, A., Eitel, I., Zacur, E., Lange, T., Stiermaier, T., Evertz, R., Backhaus, S. J., Thiele, H., Bueno-Orovio, A., Lamata, P., Schuster, A., & Grau, V. (2022). Mesh U-Nets for 3D Cardiac Deformation Modeling. In O. Camara, E. Puyol-Antón, A. Suinesiaputra, A. Young, C. Qin, M. Sermesant, & S. Wang (Eds.), Statistical Atlases and Computational Models of the Heart. Regular and CMRxMotion Challenge Papers - 13th International Workshop, STACOM 2022, Held in Conjunction with MICCAI 2022, Revised Selected Papers (pp. 245-257). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13593 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-031-23443-9_23

Beetz, Marcel ; Acero, Jorge Corral ; Banerjee, Abhirup et al. / Mesh U-Nets for 3D Cardiac Deformation Modeling. Statistical Atlases and Computational Models of the Heart. Regular and CMRxMotion Challenge Papers - 13th International Workshop, STACOM 2022, Held in Conjunction with MICCAI 2022, Revised Selected Papers. editor / Oscar Camara ; Esther Puyol-Antón ; Avan Suinesiaputra ; Alistair Young ; Chen Qin ; Maxime Sermesant ; Shuo Wang. Springer Science and Business Media Deutschland GmbH, 2022. pp. 245-257 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inbook{04cbbcc973a54cd4a461acb0622029b4,

title = "Mesh U-Nets for 3D Cardiac Deformation Modeling",

abstract = "During a cardiac cycle, the heart anatomy undergoes a series of complex 3D deformations, which can be analyzed to diagnose various cardiovascular pathologies including myocardial infarction. While volume-based metrics such as ejection fraction are commonly used in clinical practice to assess these deformations globally, they only provide limited information about localized changes in the 3D cardiac structures. The objective of this work is to develop a novel geometric deep learning approach to capture the mechanical deformation of complete 3D ventricular shapes, offering potential to discover new image-based biomarkers for cardiac disease diagnosis. To this end, we propose the mesh U-Net, which combines mesh-based convolution and pooling operations with U-Net-inspired skip connections in a hierarchical step-wise encoder-decoder architecture, in order to enable accurate and efficient learning directly on 3D anatomical meshes. The proposed network is trained to model both cardiac contraction and relaxation, that is, to predict the 3D cardiac anatomy at the end-systolic phase of the cardiac cycle based on the corresponding anatomy at end-diastole and vice versa. We evaluate our method on a multi-center cardiac magnetic resonance imaging (MRI) dataset of 1021 patients with acute myocardial infarction. We find mean surface distances between the predicted and gold standard anatomical meshes close to the pixel resolution of the underlying images and high similarity in multiple commonly used clinical metrics for both prediction directions. In addition, we show that the mesh U-Net compares favorably to a 3D U-Net benchmark by using 66% fewer network parameters and drastically smaller data sizes, while at the same time improving predictive performance by 14%. We also observe that the mesh U-Net is able to capture subpopulation-specific differences in mechanical deformation patterns between patients with different myocardial infarction types and clinical outcomes.",

keywords = "3D heart contraction, Acute myocardial infarction, Cardiac mechanics, Cardiac MRI, Geometric deep learning, Mesh sampling, Spectral graph convolutions",

author = "Marcel Beetz and Acero, {Jorge Corral} and Abhirup Banerjee and Ingo Eitel and Ernesto Zacur and Torben Lange and Thomas Stiermaier and Ruben Evertz and Backhaus, {S{\"o}ren J.} and Holger Thiele and Alfonso Bueno-Orovio and Pablo Lamata and Andreas Schuster and Vicente Grau",

note = "Funding Information: Acknowledgments. The authors express no conflict of interest. The work of MB is supported by the Stiftung der Deutschen Wirtschaft (Foundation of German Business). AB is a Royal Society University Research Fellow and is supported by the Royal Society (Grant No. URF\R1\221314). The work of AB and VG is supported by the British Heart Foundation (BHF) Project under Grant PG/20/21/35082. The work of VG is supported by the CompBioMed 2 Centre of Excellence in Computational Biomedicine (European Commission Horizon 2020 research and innovation programme, grant agreement No. 823712). The work of JCA is supported by the EU{\textquoteright}s Horizon 2020 research and innovation program under the Marie Sklodowska-Curie (g.a. 764738) and the EPSRC Impact Acceleration Account (D4D00010 DF48.01), funded by UK Research and Innovation. ABO holds a BHF Intermediate Basic Science Research Fellowship (FS/17/22/32644). The work is also supported by the German Center for Cardiovascular Research, the British Heart Foundation (PG/16/75/32383), and the Wellcome Trust (209450/Z/17). Publisher Copyright: {\textcopyright} 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.; 13th International Workshop on Statistical Atlases and Computational Models of the Heart, STACOM 2022, held in conjunction with the 25th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2022 ; Conference date: 18-09-2022 Through 18-09-2022",

year = "2022",

doi = "10.1007/978-3-031-23443-9_23",

language = "English",

isbn = "9783031234422",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "245--257",

editor = "Oscar Camara and Esther Puyol-Ant{\'o}n and Avan Suinesiaputra and Alistair Young and Chen Qin and Maxime Sermesant and Shuo Wang",

booktitle = "Statistical Atlases and Computational Models of the Heart. Regular and CMRxMotion Challenge Papers - 13th International Workshop, STACOM 2022, Held in Conjunction with MICCAI 2022, Revised Selected Papers",

address = "Germany",

}

Beetz, M, Acero, JC, Banerjee, A, Eitel, I, Zacur, E, Lange, T, Stiermaier, T, Evertz, R, Backhaus, SJ, Thiele, H, Bueno-Orovio, A, Lamata, P, Schuster, A & Grau, V 2022, Mesh U-Nets for 3D Cardiac Deformation Modeling. in O Camara, E Puyol-Antón, A Suinesiaputra, A Young, C Qin, M Sermesant & S Wang (eds), Statistical Atlases and Computational Models of the Heart. Regular and CMRxMotion Challenge Papers - 13th International Workshop, STACOM 2022, Held in Conjunction with MICCAI 2022, Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 13593 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 245-257, 13th International Workshop on Statistical Atlases and Computational Models of the Heart, STACOM 2022, held in conjunction with the 25th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2022, Singapore, Singapore, 18/09/2022. https://doi.org/10.1007/978-3-031-23443-9_23

Mesh U-Nets for 3D Cardiac Deformation Modeling. / Beetz, Marcel; Acero, Jorge Corral; Banerjee, Abhirup et al.

Statistical Atlases and Computational Models of the Heart. Regular and CMRxMotion Challenge Papers - 13th International Workshop, STACOM 2022, Held in Conjunction with MICCAI 2022, Revised Selected Papers. ed. / Oscar Camara; Esther Puyol-Antón; Avan Suinesiaputra; Alistair Young; Chen Qin; Maxime Sermesant; Shuo Wang. Springer Science and Business Media Deutschland GmbH, 2022. p. 245-257 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13593 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference paper › peer-review

TY - CHAP

T1 - Mesh U-Nets for 3D Cardiac Deformation Modeling

AU - Beetz, Marcel

AU - Acero, Jorge Corral

AU - Banerjee, Abhirup

AU - Eitel, Ingo

AU - Zacur, Ernesto

AU - Lange, Torben

AU - Stiermaier, Thomas

AU - Evertz, Ruben

AU - Backhaus, Sören J.

AU - Thiele, Holger

AU - Bueno-Orovio, Alfonso

AU - Lamata, Pablo

AU - Schuster, Andreas

AU - Grau, Vicente

N1 - Funding Information: Acknowledgments. The authors express no conflict of interest. The work of MB is supported by the Stiftung der Deutschen Wirtschaft (Foundation of German Business). AB is a Royal Society University Research Fellow and is supported by the Royal Society (Grant No. URF\R1\221314). The work of AB and VG is supported by the British Heart Foundation (BHF) Project under Grant PG/20/21/35082. The work of VG is supported by the CompBioMed 2 Centre of Excellence in Computational Biomedicine (European Commission Horizon 2020 research and innovation programme, grant agreement No. 823712). The work of JCA is supported by the EU’s Horizon 2020 research and innovation program under the Marie Sklodowska-Curie (g.a. 764738) and the EPSRC Impact Acceleration Account (D4D00010 DF48.01), funded by UK Research and Innovation. ABO holds a BHF Intermediate Basic Science Research Fellowship (FS/17/22/32644). The work is also supported by the German Center for Cardiovascular Research, the British Heart Foundation (PG/16/75/32383), and the Wellcome Trust (209450/Z/17). Publisher Copyright: © 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.

PY - 2022

Y1 - 2022

N2 - During a cardiac cycle, the heart anatomy undergoes a series of complex 3D deformations, which can be analyzed to diagnose various cardiovascular pathologies including myocardial infarction. While volume-based metrics such as ejection fraction are commonly used in clinical practice to assess these deformations globally, they only provide limited information about localized changes in the 3D cardiac structures. The objective of this work is to develop a novel geometric deep learning approach to capture the mechanical deformation of complete 3D ventricular shapes, offering potential to discover new image-based biomarkers for cardiac disease diagnosis. To this end, we propose the mesh U-Net, which combines mesh-based convolution and pooling operations with U-Net-inspired skip connections in a hierarchical step-wise encoder-decoder architecture, in order to enable accurate and efficient learning directly on 3D anatomical meshes. The proposed network is trained to model both cardiac contraction and relaxation, that is, to predict the 3D cardiac anatomy at the end-systolic phase of the cardiac cycle based on the corresponding anatomy at end-diastole and vice versa. We evaluate our method on a multi-center cardiac magnetic resonance imaging (MRI) dataset of 1021 patients with acute myocardial infarction. We find mean surface distances between the predicted and gold standard anatomical meshes close to the pixel resolution of the underlying images and high similarity in multiple commonly used clinical metrics for both prediction directions. In addition, we show that the mesh U-Net compares favorably to a 3D U-Net benchmark by using 66% fewer network parameters and drastically smaller data sizes, while at the same time improving predictive performance by 14%. We also observe that the mesh U-Net is able to capture subpopulation-specific differences in mechanical deformation patterns between patients with different myocardial infarction types and clinical outcomes.

AB - During a cardiac cycle, the heart anatomy undergoes a series of complex 3D deformations, which can be analyzed to diagnose various cardiovascular pathologies including myocardial infarction. While volume-based metrics such as ejection fraction are commonly used in clinical practice to assess these deformations globally, they only provide limited information about localized changes in the 3D cardiac structures. The objective of this work is to develop a novel geometric deep learning approach to capture the mechanical deformation of complete 3D ventricular shapes, offering potential to discover new image-based biomarkers for cardiac disease diagnosis. To this end, we propose the mesh U-Net, which combines mesh-based convolution and pooling operations with U-Net-inspired skip connections in a hierarchical step-wise encoder-decoder architecture, in order to enable accurate and efficient learning directly on 3D anatomical meshes. The proposed network is trained to model both cardiac contraction and relaxation, that is, to predict the 3D cardiac anatomy at the end-systolic phase of the cardiac cycle based on the corresponding anatomy at end-diastole and vice versa. We evaluate our method on a multi-center cardiac magnetic resonance imaging (MRI) dataset of 1021 patients with acute myocardial infarction. We find mean surface distances between the predicted and gold standard anatomical meshes close to the pixel resolution of the underlying images and high similarity in multiple commonly used clinical metrics for both prediction directions. In addition, we show that the mesh U-Net compares favorably to a 3D U-Net benchmark by using 66% fewer network parameters and drastically smaller data sizes, while at the same time improving predictive performance by 14%. We also observe that the mesh U-Net is able to capture subpopulation-specific differences in mechanical deformation patterns between patients with different myocardial infarction types and clinical outcomes.

KW - 3D heart contraction

KW - Acute myocardial infarction

KW - Cardiac mechanics

KW - Cardiac MRI

KW - Geometric deep learning

KW - Mesh sampling

KW - Spectral graph convolutions

UR - http://www.scopus.com/inward/record.url?scp=85147996130&partnerID=8YFLogxK

U2 - 10.1007/978-3-031-23443-9_23

DO - 10.1007/978-3-031-23443-9_23

M3 - Conference paper

AN - SCOPUS:85147996130

SN - 9783031234422

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 245

EP - 257

BT - Statistical Atlases and Computational Models of the Heart. Regular and CMRxMotion Challenge Papers - 13th International Workshop, STACOM 2022, Held in Conjunction with MICCAI 2022, Revised Selected Papers

A2 - Camara, Oscar

A2 - Puyol-Antón, Esther

A2 - Suinesiaputra, Avan

A2 - Young, Alistair

A2 - Qin, Chen

A2 - Sermesant, Maxime

A2 - Wang, Shuo

PB - Springer Science and Business Media Deutschland GmbH

T2 - 13th International Workshop on Statistical Atlases and Computational Models of the Heart, STACOM 2022, held in conjunction with the 25th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2022

Y2 - 18 September 2022 through 18 September 2022

ER -

Beetz M, Acero JC, Banerjee A, Eitel I, Zacur E, Lange T et al. Mesh U-Nets for 3D Cardiac Deformation Modeling. In Camara O, Puyol-Antón E, Suinesiaputra A, Young A, Qin C, Sermesant M, Wang S, editors, Statistical Atlases and Computational Models of the Heart. Regular and CMRxMotion Challenge Papers - 13th International Workshop, STACOM 2022, Held in Conjunction with MICCAI 2022, Revised Selected Papers. Springer Science and Business Media Deutschland GmbH. 2022. p. 245-257. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-031-23443-9_23