Fitting Segmentation Networks on Varying Image Resolutions Using Splatting

Mikael Brudfors; Yaël Balbastre; John Ashburner; Geraint Rees; Parashkev Nachev; Sébastien Ourselin; M. Jorge Cardoso

doi:10.1007/978-3-031-12053-4_21

Fitting Segmentation Networks on Varying Image Resolutions Using Splatting

Mikael Brudfors^*, Yaël Balbastre, John Ashburner, Geraint Rees, Parashkev Nachev, Sébastien Ourselin, M. Jorge Cardoso

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference paper › peer-review

Abstract

Data used in image segmentation are not always defined on the same grid. This is particularly true for medical images, where the resolution, field-of-view and orientation can differ across channels and subjects. Images and labels are therefore commonly resampled onto the same grid, as a pre-processing step. However, the resampling operation introduces partial volume effects and blurring, thereby changing the effective resolution and reducing the contrast between structures. In this paper we propose a splat layer, which automatically handles resolution mismatches in the input data. This layer pushes each image onto a mean space where the forward pass is performed. As the splat operator is the adjoint to the resampling operator, the mean-space prediction can be pulled back to the native label space, where the loss function is computed. Thus, the need for explicit resolution adjustment using interpolation is removed. We show on two publicly available datasets, with simulated and real multi-modal magnetic resonance images, that this model improves segmentation results compared to resampling as a pre-processing step.

Original language	English
Title of host publication	Medical Image Understanding and Analysis - 26th Annual Conference, MIUA 2022, Proceedings
Editors	Guang Yang, Angelica Aviles-Rivero, Michael Roberts, Carola-Bibiane Schönlieb
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	271-282
Number of pages	12
ISBN (Print)	9783031120527
DOIs	https://doi.org/10.1007/978-3-031-12053-4_21
Publication status	Published - 2022
Event	26th Annual Conference on Medical Image Understanding and Analysis, MIUA 2022 - Cambridge, United Kingdom Duration: 27 Jul 2022 → 29 Jul 2022

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	13413 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	26th Annual Conference on Medical Image Understanding and Analysis, MIUA 2022
Country/Territory	United Kingdom
City	Cambridge
Period	27/07/2022 → 29/07/2022

Keywords

Image resolution
Image segmentation
Pre-processing
Resampling
Splatting

Access to Document

10.1007/978-3-031-12053-4_21

Cite this

Brudfors, M., Balbastre, Y., Ashburner, J., Rees, G., Nachev, P., Ourselin, S., & Cardoso, M. J. (2022). Fitting Segmentation Networks on Varying Image Resolutions Using Splatting. In G. Yang, A. Aviles-Rivero, M. Roberts, & C.-B. Schönlieb (Eds.), Medical Image Understanding and Analysis - 26th Annual Conference, MIUA 2022, Proceedings (pp. 271-282). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13413 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-031-12053-4_21

Brudfors, Mikael ; Balbastre, Yaël ; Ashburner, John et al. / Fitting Segmentation Networks on Varying Image Resolutions Using Splatting. Medical Image Understanding and Analysis - 26th Annual Conference, MIUA 2022, Proceedings. editor / Guang Yang ; Angelica Aviles-Rivero ; Michael Roberts ; Carola-Bibiane Schönlieb. Springer Science and Business Media Deutschland GmbH, 2022. pp. 271-282 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inbook{c0abaeb5e549425dba04263aa3b5275a,

title = "Fitting Segmentation Networks on Varying Image Resolutions Using Splatting",

abstract = "Data used in image segmentation are not always defined on the same grid. This is particularly true for medical images, where the resolution, field-of-view and orientation can differ across channels and subjects. Images and labels are therefore commonly resampled onto the same grid, as a pre-processing step. However, the resampling operation introduces partial volume effects and blurring, thereby changing the effective resolution and reducing the contrast between structures. In this paper we propose a splat layer, which automatically handles resolution mismatches in the input data. This layer pushes each image onto a mean space where the forward pass is performed. As the splat operator is the adjoint to the resampling operator, the mean-space prediction can be pulled back to the native label space, where the loss function is computed. Thus, the need for explicit resolution adjustment using interpolation is removed. We show on two publicly available datasets, with simulated and real multi-modal magnetic resonance images, that this model improves segmentation results compared to resampling as a pre-processing step.",

keywords = "Image resolution, Image segmentation, Pre-processing, Resampling, Splatting",

author = "Mikael Brudfors and Ya{\"e}l Balbastre and John Ashburner and Geraint Rees and Parashkev Nachev and S{\'e}bastien Ourselin and Cardoso, {M. Jorge}",

note = "Publisher Copyright: {\textcopyright} 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.; 26th Annual Conference on Medical Image Understanding and Analysis, MIUA 2022 ; Conference date: 27-07-2022 Through 29-07-2022",

year = "2022",

doi = "10.1007/978-3-031-12053-4_21",

language = "English",

isbn = "9783031120527",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "271--282",

editor = "Guang Yang and Angelica Aviles-Rivero and Michael Roberts and Carola-Bibiane Sch{\"o}nlieb",

booktitle = "Medical Image Understanding and Analysis - 26th Annual Conference, MIUA 2022, Proceedings",

address = "Germany",

}

Brudfors, M, Balbastre, Y, Ashburner, J, Rees, G, Nachev, P, Ourselin, S & Cardoso, MJ 2022, Fitting Segmentation Networks on Varying Image Resolutions Using Splatting. in G Yang, A Aviles-Rivero, M Roberts & C-B Schönlieb (eds), Medical Image Understanding and Analysis - 26th Annual Conference, MIUA 2022, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 13413 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 271-282, 26th Annual Conference on Medical Image Understanding and Analysis, MIUA 2022, Cambridge, United Kingdom, 27/07/2022. https://doi.org/10.1007/978-3-031-12053-4_21

Fitting Segmentation Networks on Varying Image Resolutions Using Splatting. / Brudfors, Mikael; Balbastre, Yaël; Ashburner, John et al.
Medical Image Understanding and Analysis - 26th Annual Conference, MIUA 2022, Proceedings. ed. / Guang Yang; Angelica Aviles-Rivero; Michael Roberts; Carola-Bibiane Schönlieb. Springer Science and Business Media Deutschland GmbH, 2022. p. 271-282 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13413 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference paper › peer-review

TY - CHAP

T1 - Fitting Segmentation Networks on Varying Image Resolutions Using Splatting

AU - Brudfors, Mikael

AU - Balbastre, Yaël

AU - Ashburner, John

AU - Rees, Geraint

AU - Nachev, Parashkev

AU - Ourselin, Sébastien

AU - Cardoso, M. Jorge

PY - 2022

Y1 - 2022

N2 - Data used in image segmentation are not always defined on the same grid. This is particularly true for medical images, where the resolution, field-of-view and orientation can differ across channels and subjects. Images and labels are therefore commonly resampled onto the same grid, as a pre-processing step. However, the resampling operation introduces partial volume effects and blurring, thereby changing the effective resolution and reducing the contrast between structures. In this paper we propose a splat layer, which automatically handles resolution mismatches in the input data. This layer pushes each image onto a mean space where the forward pass is performed. As the splat operator is the adjoint to the resampling operator, the mean-space prediction can be pulled back to the native label space, where the loss function is computed. Thus, the need for explicit resolution adjustment using interpolation is removed. We show on two publicly available datasets, with simulated and real multi-modal magnetic resonance images, that this model improves segmentation results compared to resampling as a pre-processing step.

AB - Data used in image segmentation are not always defined on the same grid. This is particularly true for medical images, where the resolution, field-of-view and orientation can differ across channels and subjects. Images and labels are therefore commonly resampled onto the same grid, as a pre-processing step. However, the resampling operation introduces partial volume effects and blurring, thereby changing the effective resolution and reducing the contrast between structures. In this paper we propose a splat layer, which automatically handles resolution mismatches in the input data. This layer pushes each image onto a mean space where the forward pass is performed. As the splat operator is the adjoint to the resampling operator, the mean-space prediction can be pulled back to the native label space, where the loss function is computed. Thus, the need for explicit resolution adjustment using interpolation is removed. We show on two publicly available datasets, with simulated and real multi-modal magnetic resonance images, that this model improves segmentation results compared to resampling as a pre-processing step.

KW - Image resolution

KW - Image segmentation

KW - Pre-processing

KW - Resampling

KW - Splatting

UR - http://www.scopus.com/inward/record.url?scp=85135936137&partnerID=8YFLogxK

U2 - 10.1007/978-3-031-12053-4_21

DO - 10.1007/978-3-031-12053-4_21

M3 - Conference paper

AN - SCOPUS:85135936137

SN - 9783031120527

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 271

EP - 282

BT - Medical Image Understanding and Analysis - 26th Annual Conference, MIUA 2022, Proceedings

A2 - Yang, Guang

A2 - Aviles-Rivero, Angelica

A2 - Roberts, Michael

A2 - Schönlieb, Carola-Bibiane

PB - Springer Science and Business Media Deutschland GmbH

T2 - 26th Annual Conference on Medical Image Understanding and Analysis, MIUA 2022

Y2 - 27 July 2022 through 29 July 2022

ER -

Brudfors M, Balbastre Y, Ashburner J, Rees G, Nachev P, Ourselin S et al. Fitting Segmentation Networks on Varying Image Resolutions Using Splatting. In Yang G, Aviles-Rivero A, Roberts M, Schönlieb CB, editors, Medical Image Understanding and Analysis - 26th Annual Conference, MIUA 2022, Proceedings. Springer Science and Business Media Deutschland GmbH. 2022. p. 271-282. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-031-12053-4_21