Uncertainty-Aware Deep Learning Based Deformable Registration

Irina Grigorescu; Alena Uus; Daan Christiaens; Lucilio Cordero-Grande; Jana Hutter; Dafnis Batalle; A. David Edwards; Joseph V. Hajnal; Marc Modat; Maria Deprez

doi:10.1007/978-3-030-87735-4_6

Uncertainty-Aware Deep Learning Based Deformable Registration

Irina Grigorescu^*, Alena Uus, Daan Christiaens, Lucilio Cordero-Grande, Jana Hutter, Dafnis Batalle, A. David Edwards, Joseph V. Hajnal, Marc Modat, Maria Deprez

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference paper › peer-review

Abstract

We introduce an uncertainty-aware deep learning deformable image registration solution for magnetic resonance imaging multi-channel data. In our proposed framework, the contributions of structural and microstructural data to the displacement field are weighted with spatially varying certainty maps. We produce certainty maps by employing a conditional variational autoencoder image registration network, which enables us to generate uncertainty maps in the deformation field itself. Our approach is quantitatively evaluated on pairwise registrations of 36 neonates to a standard structural and/or microstructural template, and compared with models trained on either single modality, or both modalities together. Our results show that by incorporating uncertainty while fusing the two modalities, we achieve superior alignment in cortical gray matter and white matter regions, while also achieving a good alignment of the white matter tracts. In addition, for each of our trained models, we show examples of average uncertainty maps calculated for 10 neonates scanned at 40 weeks post-menstrual age.

Original language	English
Title of host publication	Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, and Perinatal Imaging, Placental and Preterm Image Analysis - 3rd International Workshop, UNSURE 2021, and 6th International Workshop, PIPPI 2021, Held in Conjunction with MICCAI 2021, Proceedings
Editors	Carole H. Sudre, Roxane Licandro, Christian Baumgartner, Andrew Melbourne, Adrian Dalca, Jana Hutter, Ryutaro Tanno, Esra Abaci Turk, Koen Van Leemput, Jordina Torrents Barrena, William M. Wells, Christopher Macgowan
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	54-63
Number of pages	10
ISBN (Print)	9783030877347
DOIs	https://doi.org/10.1007/978-3-030-87735-4_6
Publication status	Published - 2021
Event	3rd International Workshop on Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, UNSURE 2021, and the 6th International Workshop on Perinatal, Preterm and Paediatric Image Analysis, PIPPI 2021, held in conjunction with the 24th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2021 - Virtual, Online Duration: 1 Oct 2021 → 1 Oct 2021

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	12959 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	3rd International Workshop on Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, UNSURE 2021, and the 6th International Workshop on Perinatal, Preterm and Paediatric Image Analysis, PIPPI 2021, held in conjunction with the 24th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2021
City	Virtual, Online
Period	1/10/2021 → 1/10/2021

Keywords

Certainty maps
Multi-channel registration
Uncertainty

Access to Document

10.1007/978-3-030-87735-4_6

Cite this

Grigorescu, I., Uus, A., Christiaens, D., Cordero-Grande, L., Hutter, J., Batalle, D., Edwards, A. D., Hajnal, J. V., Modat, M., & Deprez, M. (2021). Uncertainty-Aware Deep Learning Based Deformable Registration. In C. H. Sudre, R. Licandro, C. Baumgartner, A. Melbourne, A. Dalca, J. Hutter, R. Tanno, E. Abaci Turk, K. Van Leemput, J. Torrents Barrena, W. M. Wells, & C. Macgowan (Eds.), Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, and Perinatal Imaging, Placental and Preterm Image Analysis - 3rd International Workshop, UNSURE 2021, and 6th International Workshop, PIPPI 2021, Held in Conjunction with MICCAI 2021, Proceedings (pp. 54-63). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12959 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-030-87735-4_6

Grigorescu, Irina ; Uus, Alena ; Christiaens, Daan et al. / Uncertainty-Aware Deep Learning Based Deformable Registration. Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, and Perinatal Imaging, Placental and Preterm Image Analysis - 3rd International Workshop, UNSURE 2021, and 6th International Workshop, PIPPI 2021, Held in Conjunction with MICCAI 2021, Proceedings. editor / Carole H. Sudre ; Roxane Licandro ; Christian Baumgartner ; Andrew Melbourne ; Adrian Dalca ; Jana Hutter ; Ryutaro Tanno ; Esra Abaci Turk ; Koen Van Leemput ; Jordina Torrents Barrena ; William M. Wells ; Christopher Macgowan. Springer Science and Business Media Deutschland GmbH, 2021. pp. 54-63 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inbook{6a3ec516a71e4c7fbde6c2fc822fa3ba,

title = "Uncertainty-Aware Deep Learning Based Deformable Registration",

abstract = "We introduce an uncertainty-aware deep learning deformable image registration solution for magnetic resonance imaging multi-channel data. In our proposed framework, the contributions of structural and microstructural data to the displacement field are weighted with spatially varying certainty maps. We produce certainty maps by employing a conditional variational autoencoder image registration network, which enables us to generate uncertainty maps in the deformation field itself. Our approach is quantitatively evaluated on pairwise registrations of 36 neonates to a standard structural and/or microstructural template, and compared with models trained on either single modality, or both modalities together. Our results show that by incorporating uncertainty while fusing the two modalities, we achieve superior alignment in cortical gray matter and white matter regions, while also achieving a good alignment of the white matter tracts. In addition, for each of our trained models, we show examples of average uncertainty maps calculated for 10 neonates scanned at 40 weeks post-menstrual age.",

keywords = "Certainty maps, Multi-channel registration, Uncertainty",

author = "Irina Grigorescu and Alena Uus and Daan Christiaens and Lucilio Cordero-Grande and Jana Hutter and Dafnis Batalle and Edwards, {A. David} and Hajnal, {Joseph V.} and Marc Modat and Maria Deprez",

note = "Funding Information: This work was supported by the Academy of Medical Sciences Springboard Award [SBF004\1040], Medical Research Council (Grant no. [MR/K006355/1]), European Research Council under the European Union?s Seventh Framework Programme [FP7/20072013]/ERC grant agreement no. 319456 dHCP project, the EPSRC Research Council as part of the EPSRC DTP (grant Ref: [EP/R513064/1]), the Wellcome/EPSRC Centre for Medical Engineering at King?s College London [WT 203148/Z/16/Z], the NIHR Clinical Research Facility (CRF) at Guy?s and St Thomas?, and by the National Institute for Health Research Biomedical Research Centre based at Guy?s and St Thomas? NHS Foundation Trust and King?s College London. Publisher Copyright: {\textcopyright} 2021, Springer Nature Switzerland AG.; 3rd International Workshop on Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, UNSURE 2021, and the 6th International Workshop on Perinatal, Preterm and Paediatric Image Analysis, PIPPI 2021, held in conjunction with the 24th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2021 ; Conference date: 01-10-2021 Through 01-10-2021",

year = "2021",

doi = "10.1007/978-3-030-87735-4_6",

language = "English",

isbn = "9783030877347",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "54--63",

editor = "Sudre, {Carole H.} and Roxane Licandro and Christian Baumgartner and Andrew Melbourne and Adrian Dalca and Jana Hutter and Ryutaro Tanno and {Abaci Turk}, Esra and {Van Leemput}, Koen and {Torrents Barrena}, Jordina and Wells, {William M.} and Christopher Macgowan",

booktitle = "Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, and Perinatal Imaging, Placental and Preterm Image Analysis - 3rd International Workshop, UNSURE 2021, and 6th International Workshop, PIPPI 2021, Held in Conjunction with MICCAI 2021, Proceedings",

address = "Germany",

}

Grigorescu, I, Uus, A, Christiaens, D , Cordero-Grande, L , Hutter, J , Batalle, D , Edwards, AD , Hajnal, JV , Modat, M & Deprez, M 2021, Uncertainty-Aware Deep Learning Based Deformable Registration. in CH Sudre, R Licandro, C Baumgartner, A Melbourne, A Dalca, J Hutter, R Tanno, E Abaci Turk, K Van Leemput, J Torrents Barrena, WM Wells & C Macgowan (eds), Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, and Perinatal Imaging, Placental and Preterm Image Analysis - 3rd International Workshop, UNSURE 2021, and 6th International Workshop, PIPPI 2021, Held in Conjunction with MICCAI 2021, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12959 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 54-63, 3rd International Workshop on Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, UNSURE 2021, and the 6th International Workshop on Perinatal, Preterm and Paediatric Image Analysis, PIPPI 2021, held in conjunction with the 24th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2021, Virtual, Online, 1/10/2021. https://doi.org/10.1007/978-3-030-87735-4_6

Uncertainty-Aware Deep Learning Based Deformable Registration. / Grigorescu, Irina; Uus, Alena; Christiaens, Daan et al.
Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, and Perinatal Imaging, Placental and Preterm Image Analysis - 3rd International Workshop, UNSURE 2021, and 6th International Workshop, PIPPI 2021, Held in Conjunction with MICCAI 2021, Proceedings. ed. / Carole H. Sudre; Roxane Licandro; Christian Baumgartner; Andrew Melbourne; Adrian Dalca; Jana Hutter; Ryutaro Tanno; Esra Abaci Turk; Koen Van Leemput; Jordina Torrents Barrena; William M. Wells; Christopher Macgowan. Springer Science and Business Media Deutschland GmbH, 2021. p. 54-63 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12959 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference paper › peer-review

TY - CHAP

T1 - Uncertainty-Aware Deep Learning Based Deformable Registration

AU - Grigorescu, Irina

AU - Uus, Alena

AU - Christiaens, Daan

AU - Cordero-Grande, Lucilio

AU - Hutter, Jana

AU - Batalle, Dafnis

AU - Edwards, A. David

AU - Hajnal, Joseph V.

AU - Modat, Marc

AU - Deprez, Maria

N1 - Funding Information: This work was supported by the Academy of Medical Sciences Springboard Award [SBF004\1040], Medical Research Council (Grant no. [MR/K006355/1]), European Research Council under the European Union?s Seventh Framework Programme [FP7/20072013]/ERC grant agreement no. 319456 dHCP project, the EPSRC Research Council as part of the EPSRC DTP (grant Ref: [EP/R513064/1]), the Wellcome/EPSRC Centre for Medical Engineering at King?s College London [WT 203148/Z/16/Z], the NIHR Clinical Research Facility (CRF) at Guy?s and St Thomas?, and by the National Institute for Health Research Biomedical Research Centre based at Guy?s and St Thomas? NHS Foundation Trust and King?s College London. Publisher Copyright: © 2021, Springer Nature Switzerland AG.

PY - 2021

Y1 - 2021

N2 - We introduce an uncertainty-aware deep learning deformable image registration solution for magnetic resonance imaging multi-channel data. In our proposed framework, the contributions of structural and microstructural data to the displacement field are weighted with spatially varying certainty maps. We produce certainty maps by employing a conditional variational autoencoder image registration network, which enables us to generate uncertainty maps in the deformation field itself. Our approach is quantitatively evaluated on pairwise registrations of 36 neonates to a standard structural and/or microstructural template, and compared with models trained on either single modality, or both modalities together. Our results show that by incorporating uncertainty while fusing the two modalities, we achieve superior alignment in cortical gray matter and white matter regions, while also achieving a good alignment of the white matter tracts. In addition, for each of our trained models, we show examples of average uncertainty maps calculated for 10 neonates scanned at 40 weeks post-menstrual age.

AB - We introduce an uncertainty-aware deep learning deformable image registration solution for magnetic resonance imaging multi-channel data. In our proposed framework, the contributions of structural and microstructural data to the displacement field are weighted with spatially varying certainty maps. We produce certainty maps by employing a conditional variational autoencoder image registration network, which enables us to generate uncertainty maps in the deformation field itself. Our approach is quantitatively evaluated on pairwise registrations of 36 neonates to a standard structural and/or microstructural template, and compared with models trained on either single modality, or both modalities together. Our results show that by incorporating uncertainty while fusing the two modalities, we achieve superior alignment in cortical gray matter and white matter regions, while also achieving a good alignment of the white matter tracts. In addition, for each of our trained models, we show examples of average uncertainty maps calculated for 10 neonates scanned at 40 weeks post-menstrual age.

KW - Certainty maps

KW - Multi-channel registration

KW - Uncertainty

UR - http://www.scopus.com/inward/record.url?scp=85117125293&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-87735-4_6

DO - 10.1007/978-3-030-87735-4_6

M3 - Conference paper

AN - SCOPUS:85117125293

SN - 9783030877347

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 54

EP - 63

BT - Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, and Perinatal Imaging, Placental and Preterm Image Analysis - 3rd International Workshop, UNSURE 2021, and 6th International Workshop, PIPPI 2021, Held in Conjunction with MICCAI 2021, Proceedings

A2 - Sudre, Carole H.

A2 - Licandro, Roxane

A2 - Baumgartner, Christian

A2 - Melbourne, Andrew

A2 - Dalca, Adrian

A2 - Hutter, Jana

A2 - Tanno, Ryutaro

A2 - Abaci Turk, Esra

A2 - Van Leemput, Koen

A2 - Torrents Barrena, Jordina

A2 - Wells, William M.

A2 - Macgowan, Christopher

PB - Springer Science and Business Media Deutschland GmbH

T2 - 3rd International Workshop on Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, UNSURE 2021, and the 6th International Workshop on Perinatal, Preterm and Paediatric Image Analysis, PIPPI 2021, held in conjunction with the 24th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2021

Y2 - 1 October 2021 through 1 October 2021

ER -

Grigorescu I, Uus A, Christiaens D , Cordero-Grande L , Hutter J , Batalle D et al. Uncertainty-Aware Deep Learning Based Deformable Registration. In Sudre CH, Licandro R, Baumgartner C, Melbourne A, Dalca A, Hutter J, Tanno R, Abaci Turk E, Van Leemput K, Torrents Barrena J, Wells WM, Macgowan C, editors, Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, and Perinatal Imaging, Placental and Preterm Image Analysis - 3rd International Workshop, UNSURE 2021, and 6th International Workshop, PIPPI 2021, Held in Conjunction with MICCAI 2021, Proceedings. Springer Science and Business Media Deutschland GmbH. 2021. p. 54-63. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-87735-4_6

Uncertainty-Aware Deep Learning Based Deformable Registration

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this