Inter Extreme Points Geodesics for End-to-End Weakly Supervised Image Segmentation

Reuben Dorent; Samuel Joutard; Jonathan Shapey; Aaron Kujawa; Marc Modat; Sébastien Ourselin; Tom Vercauteren

doi:10.1007/978-3-030-87196-3_57

Inter Extreme Points Geodesics for End-to-End Weakly Supervised Image Segmentation

Reuben Dorent^*, Samuel Joutard, Jonathan Shapey, Aaron Kujawa, Marc Modat, Sébastien Ourselin, Tom Vercauteren

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference paper › peer-review

26 Citations (Scopus)

Abstract

We introduce InExtremIS, a weakly supervised 3D approach to train a deep image segmentation network using particularly weak train-time annotations: only 6 extreme clicks at the boundary of the objects of interest. Our fully-automatic method is trained end-to-end and does not require any test-time annotations. From the extreme points, 3D bounding boxes are extracted around objects of interest. Then, deep geodesics connecting extreme points are generated to increase the amount of “annotated” voxels within the bounding boxes. Finally, a weakly supervised regularised loss derived from a Conditional Random Field formulation is used to encourage prediction consistency over homogeneous regions. Extensive experiments are performed on a large open dataset for Vestibular Schwannoma segmentation. InExtremIS obtained competitive performance, approaching full supervision and outperforming significantly other weakly supervised techniques based on bounding boxes. Moreover, given a fixed annotation time budget, InExtremIS outperformed full supervision. Our code and data are available online.

Original language	English
Title of host publication	Medical Image Computing and Computer Assisted Intervention – MICCAI 2021 - 24th International Conference, Proceedings
Editors	Marleen de Bruijne, Philippe C. Cattin, Stéphane Cotin, Nicolas Padoy, Stefanie Speidel, Yefeng Zheng, Caroline Essert
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	615-624
Number of pages	10
ISBN (Print)	9783030871956
DOIs	https://doi.org/10.1007/978-3-030-87196-3_57
Publication status	Published - 2021
Event	24th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2021 - Virtual, Online Duration: 27 Sept 2021 → 1 Oct 2021

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	12902 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	24th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2021
City	Virtual, Online
Period	27/09/2021 → 1/10/2021

Access to Document

10.1007/978-3-030-87196-3_57

https://arxiv.org/abs/2107.00583

Cite this

Dorent, R., Joutard, S., Shapey, J., Kujawa, A., Modat, M., Ourselin, S., & Vercauteren, T. (2021). Inter Extreme Points Geodesics for End-to-End Weakly Supervised Image Segmentation. In M. de Bruijne, P. C. Cattin, S. Cotin, N. Padoy, S. Speidel, Y. Zheng, & C. Essert (Eds.), Medical Image Computing and Computer Assisted Intervention – MICCAI 2021 - 24th International Conference, Proceedings (pp. 615-624). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12902 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-030-87196-3_57

Dorent, Reuben ; Joutard, Samuel ; Shapey, Jonathan et al. / Inter Extreme Points Geodesics for End-to-End Weakly Supervised Image Segmentation. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021 - 24th International Conference, Proceedings. editor / Marleen de Bruijne ; Philippe C. Cattin ; Stéphane Cotin ; Nicolas Padoy ; Stefanie Speidel ; Yefeng Zheng ; Caroline Essert. Springer Science and Business Media Deutschland GmbH, 2021. pp. 615-624 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inbook{90be56721bea41f5bef295bc9addf8cc,

title = "Inter Extreme Points Geodesics for End-to-End Weakly Supervised Image Segmentation",

abstract = "We introduce InExtremIS, a weakly supervised 3D approach to train a deep image segmentation network using particularly weak train-time annotations: only 6 extreme clicks at the boundary of the objects of interest. Our fully-automatic method is trained end-to-end and does not require any test-time annotations. From the extreme points, 3D bounding boxes are extracted around objects of interest. Then, deep geodesics connecting extreme points are generated to increase the amount of “annotated” voxels within the bounding boxes. Finally, a weakly supervised regularised loss derived from a Conditional Random Field formulation is used to encourage prediction consistency over homogeneous regions. Extensive experiments are performed on a large open dataset for Vestibular Schwannoma segmentation. InExtremIS obtained competitive performance, approaching full supervision and outperforming significantly other weakly supervised techniques based on bounding boxes. Moreover, given a fixed annotation time budget, InExtremIS outperformed full supervision. Our code and data are available online.",

author = "Reuben Dorent and Samuel Joutard and Jonathan Shapey and Aaron Kujawa and Marc Modat and S{\'e}bastien Ourselin and Tom Vercauteren",

note = "Funding Information: Acknowledgement. This work was supported by the Engineering and Physical Sciences Research Council (EPSRC) [NS/A000049/1] and Wellcome Trust [203148/Z/16/Z]. TV is supported by a Medtronic/Royal Academy of Engineering Research Chair [RCSRF1819\7\34]. Publisher Copyright: {\textcopyright} 2021, Springer Nature Switzerland AG.; 24th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2021 ; Conference date: 27-09-2021 Through 01-10-2021",

year = "2021",

doi = "10.1007/978-3-030-87196-3_57",

language = "English",

isbn = "9783030871956",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "615--624",

editor = "{de Bruijne}, Marleen and Cattin, {Philippe C.} and St{\'e}phane Cotin and Nicolas Padoy and Stefanie Speidel and Yefeng Zheng and Caroline Essert",

booktitle = "Medical Image Computing and Computer Assisted Intervention – MICCAI 2021 - 24th International Conference, Proceedings",

address = "Germany",

}

Dorent, R, Joutard, S, Shapey, J, Kujawa, A, Modat, M , Ourselin, S & Vercauteren, T 2021, Inter Extreme Points Geodesics for End-to-End Weakly Supervised Image Segmentation. in M de Bruijne, PC Cattin, S Cotin, N Padoy, S Speidel, Y Zheng & C Essert (eds), Medical Image Computing and Computer Assisted Intervention – MICCAI 2021 - 24th International Conference, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12902 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 615-624, 24th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2021, Virtual, Online, 27/09/2021. https://doi.org/10.1007/978-3-030-87196-3_57

Inter Extreme Points Geodesics for End-to-End Weakly Supervised Image Segmentation. / Dorent, Reuben; Joutard, Samuel; Shapey, Jonathan et al.
Medical Image Computing and Computer Assisted Intervention – MICCAI 2021 - 24th International Conference, Proceedings. ed. / Marleen de Bruijne; Philippe C. Cattin; Stéphane Cotin; Nicolas Padoy; Stefanie Speidel; Yefeng Zheng; Caroline Essert. Springer Science and Business Media Deutschland GmbH, 2021. p. 615-624 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12902 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference paper › peer-review

TY - CHAP

T1 - Inter Extreme Points Geodesics for End-to-End Weakly Supervised Image Segmentation

AU - Dorent, Reuben

AU - Joutard, Samuel

AU - Shapey, Jonathan

AU - Kujawa, Aaron

AU - Modat, Marc

AU - Ourselin, Sébastien

AU - Vercauteren, Tom

N1 - Funding Information: Acknowledgement. This work was supported by the Engineering and Physical Sciences Research Council (EPSRC) [NS/A000049/1] and Wellcome Trust [203148/Z/16/Z]. TV is supported by a Medtronic/Royal Academy of Engineering Research Chair [RCSRF1819\7\34]. Publisher Copyright: © 2021, Springer Nature Switzerland AG.

PY - 2021

Y1 - 2021

N2 - We introduce InExtremIS, a weakly supervised 3D approach to train a deep image segmentation network using particularly weak train-time annotations: only 6 extreme clicks at the boundary of the objects of interest. Our fully-automatic method is trained end-to-end and does not require any test-time annotations. From the extreme points, 3D bounding boxes are extracted around objects of interest. Then, deep geodesics connecting extreme points are generated to increase the amount of “annotated” voxels within the bounding boxes. Finally, a weakly supervised regularised loss derived from a Conditional Random Field formulation is used to encourage prediction consistency over homogeneous regions. Extensive experiments are performed on a large open dataset for Vestibular Schwannoma segmentation. InExtremIS obtained competitive performance, approaching full supervision and outperforming significantly other weakly supervised techniques based on bounding boxes. Moreover, given a fixed annotation time budget, InExtremIS outperformed full supervision. Our code and data are available online.

AB - We introduce InExtremIS, a weakly supervised 3D approach to train a deep image segmentation network using particularly weak train-time annotations: only 6 extreme clicks at the boundary of the objects of interest. Our fully-automatic method is trained end-to-end and does not require any test-time annotations. From the extreme points, 3D bounding boxes are extracted around objects of interest. Then, deep geodesics connecting extreme points are generated to increase the amount of “annotated” voxels within the bounding boxes. Finally, a weakly supervised regularised loss derived from a Conditional Random Field formulation is used to encourage prediction consistency over homogeneous regions. Extensive experiments are performed on a large open dataset for Vestibular Schwannoma segmentation. InExtremIS obtained competitive performance, approaching full supervision and outperforming significantly other weakly supervised techniques based on bounding boxes. Moreover, given a fixed annotation time budget, InExtremIS outperformed full supervision. Our code and data are available online.

UR - http://www.scopus.com/inward/record.url?scp=85116482463&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-87196-3_57

DO - 10.1007/978-3-030-87196-3_57

M3 - Conference paper

AN - SCOPUS:85116482463

SN - 9783030871956

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 615

EP - 624

BT - Medical Image Computing and Computer Assisted Intervention – MICCAI 2021 - 24th International Conference, Proceedings

A2 - de Bruijne, Marleen

A2 - Cattin, Philippe C.

A2 - Cotin, Stéphane

A2 - Padoy, Nicolas

A2 - Speidel, Stefanie

A2 - Zheng, Yefeng

A2 - Essert, Caroline

PB - Springer Science and Business Media Deutschland GmbH

T2 - 24th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2021

Y2 - 27 September 2021 through 1 October 2021

ER -

Dorent R, Joutard S, Shapey J, Kujawa A, Modat M , Ourselin S et al. Inter Extreme Points Geodesics for End-to-End Weakly Supervised Image Segmentation. In de Bruijne M, Cattin PC, Cotin S, Padoy N, Speidel S, Zheng Y, Essert C, editors, Medical Image Computing and Computer Assisted Intervention – MICCAI 2021 - 24th International Conference, Proceedings. Springer Science and Business Media Deutschland GmbH. 2021. p. 615-624. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-87196-3_57