Adaptive Multi-scale Online Likelihood Network for AI-Assisted Interactive Segmentation

Muhammad Asad; Helena Williams; Indrajeet Mandal; Sarim Ather; Jan Deprest; Jan D’hooge; Tom Vercauteren

doi:10.1007/978-3-031-43895-0_53

Adaptive Multi-scale Online Likelihood Network for AI-Assisted Interactive Segmentation

Muhammad Asad^*, Helena Williams, Indrajeet Mandal, Sarim Ather, Jan Deprest, Jan D’hooge, Tom Vercauteren

^*Corresponding author for this work

Surgical & Interventional Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference paper › peer-review

3 Citations (Scopus)

26 Downloads (Pure)

Abstract

Existing interactive segmentation methods leverage automatic segmentation and user interactions for label refinement, significantly reducing the annotation workload compared to manual annotation. However, these methods lack quick adaptability to ambiguous and noisy data, which is a challenge in CT volumes containing lung lesions from COVID-19 patients. In this work, we propose an adaptive multi-scale online likelihood network (MONet) that adaptively learns in a data-efficient online setting from both an initial automatic segmentation and user interactions providing corrections. We achieve adaptive learning by proposing an adaptive loss that extends the influence of user-provided interaction to neighboring regions with similar features. In addition, we propose a data-efficient probability-guided pruning method that discards uncertain and redundant labels in the initial segmentation to enable efficient online training and inference. Our proposed method was evaluated by an expert in a blinded comparative study on COVID-19 lung lesion annotation task in CT. Our approach achieved 5.86% higher Dice score with 24.67% less perceived NASA-TLX workload score than the state-of-the-art. Source code is available at: https://github.com/masadcv/MONet-MONAILabel.

Original language	English
Title of host publication	Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 - 26th International Conference, Proceedings
Editors	Hayit Greenspan, Hayit Greenspan, Anant Madabhushi, Parvin Mousavi, Septimiu Salcudean, James Duncan, Tanveer Syeda-Mahmood, Russell Taylor
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	564-574
Number of pages	11
ISBN (Print)	9783031438943
DOIs	https://doi.org/10.1007/978-3-031-43895-0_53
Publication status	Published - Oct 2023
Event	26th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2023 - Vancouver, Canada Duration: 8 Oct 2023 → 12 Oct 2023

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	14221 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	26th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2023
Country/Territory	Canada
City	Vancouver
Period	8/10/2023 → 12/10/2023

Access to Document

10.1007/978-3-031-43895-0_53Licence: Unspecified

2023 - Lect. Notes Comput. Sci. - Asad, Williams, Mandal et al. - Adaptive Multi-scale Online Likelihood Network for AI-Assisted Interactive SegmentationAccepted author manuscript, 5.11 MBLicence: CC BY

Cite this

Asad, M., Williams, H., Mandal, I., Ather, S., Deprest, J., D’hooge, J., & Vercauteren, T. (2023). Adaptive Multi-scale Online Likelihood Network for AI-Assisted Interactive Segmentation. In H. Greenspan, H. Greenspan, A. Madabhushi, P. Mousavi, S. Salcudean, J. Duncan, T. Syeda-Mahmood, & R. Taylor (Eds.), Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 - 26th International Conference, Proceedings (pp. 564-574). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14221 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-031-43895-0_53

Asad, Muhammad ; Williams, Helena ; Mandal, Indrajeet et al. / Adaptive Multi-scale Online Likelihood Network for AI-Assisted Interactive Segmentation. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 - 26th International Conference, Proceedings. editor / Hayit Greenspan ; Hayit Greenspan ; Anant Madabhushi ; Parvin Mousavi ; Septimiu Salcudean ; James Duncan ; Tanveer Syeda-Mahmood ; Russell Taylor. Springer Science and Business Media Deutschland GmbH, 2023. pp. 564-574 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inbook{482103a3dce94b36b7130ee601eac30f,

title = "Adaptive Multi-scale Online Likelihood Network for AI-Assisted Interactive Segmentation",

abstract = "Existing interactive segmentation methods leverage automatic segmentation and user interactions for label refinement, significantly reducing the annotation workload compared to manual annotation. However, these methods lack quick adaptability to ambiguous and noisy data, which is a challenge in CT volumes containing lung lesions from COVID-19 patients. In this work, we propose an adaptive multi-scale online likelihood network (MONet) that adaptively learns in a data-efficient online setting from both an initial automatic segmentation and user interactions providing corrections. We achieve adaptive learning by proposing an adaptive loss that extends the influence of user-provided interaction to neighboring regions with similar features. In addition, we propose a data-efficient probability-guided pruning method that discards uncertain and redundant labels in the initial segmentation to enable efficient online training and inference. Our proposed method was evaluated by an expert in a blinded comparative study on COVID-19 lung lesion annotation task in CT. Our approach achieved 5.86% higher Dice score with 24.67% less perceived NASA-TLX workload score than the state-of-the-art. Source code is available at: https://github.com/masadcv/MONet-MONAILabel.",

author = "Muhammad Asad and Helena Williams and Indrajeet Mandal and Sarim Ather and Jan Deprest and Jan D{\textquoteright}hooge and Tom Vercauteren",

note = "Funding Information: This project has received funding from the European Union{\textquoteright}s Horizon 2020 research and innovation programme under grant agreement No 101016131 (icovid project). This work was also supported by core and project funding from the Wellcome/EPSRC [WT203148/Z/16/Z; NS/A000049/1; WT101957; NS/A000027/1]. This project utilized scribbles-based interactive segmentation tools from opensource project MONAI Label (https://github.com/Project-MONAI/MONAILabel) [7]. Publisher Copyright: {\textcopyright} The Author(s), under exclusive license to Springer Nature Switzerland AG 2023. Funding Information: Acknowledgment. This project has received funding from the European Union{\textquoteright}s Horizon 2020 research and innovation programme under grant agreement No 101016131 (icovid project). This work was also supported by core and project funding from the Wellcome/EPSRC [WT203148/Z/16/Z; NS/A000049/1; WT101957; NS/A000027/1]. Publisher Copyright: {\textcopyright} The Author(s), under exclusive license to Springer Nature Switzerland AG 2023.; 26th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2023 ; Conference date: 08-10-2023 Through 12-10-2023",

year = "2023",

month = oct,

doi = "10.1007/978-3-031-43895-0_53",

language = "English",

isbn = "9783031438943",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "564--574",

editor = "Hayit Greenspan and Hayit Greenspan and Anant Madabhushi and Parvin Mousavi and Septimiu Salcudean and James Duncan and Tanveer Syeda-Mahmood and Russell Taylor",

booktitle = "Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 - 26th International Conference, Proceedings",

address = "Germany",

}

Asad, M, Williams, H, Mandal, I, Ather, S, Deprest, J, D’hooge, J & Vercauteren, T 2023, Adaptive Multi-scale Online Likelihood Network for AI-Assisted Interactive Segmentation. in H Greenspan, H Greenspan, A Madabhushi, P Mousavi, S Salcudean, J Duncan, T Syeda-Mahmood & R Taylor (eds), Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 - 26th International Conference, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 14221 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 564-574, 26th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2023, Vancouver, Canada, 8/10/2023. https://doi.org/10.1007/978-3-031-43895-0_53

Adaptive Multi-scale Online Likelihood Network for AI-Assisted Interactive Segmentation. / Asad, Muhammad; Williams, Helena; Mandal, Indrajeet et al.
Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 - 26th International Conference, Proceedings. ed. / Hayit Greenspan; Hayit Greenspan; Anant Madabhushi; Parvin Mousavi; Septimiu Salcudean; James Duncan; Tanveer Syeda-Mahmood; Russell Taylor. Springer Science and Business Media Deutschland GmbH, 2023. p. 564-574 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14221 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference paper › peer-review

TY - CHAP

T1 - Adaptive Multi-scale Online Likelihood Network for AI-Assisted Interactive Segmentation

AU - Asad, Muhammad

AU - Williams, Helena

AU - Mandal, Indrajeet

AU - Ather, Sarim

AU - Deprest, Jan

AU - D’hooge, Jan

AU - Vercauteren, Tom

N1 - Funding Information: This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 101016131 (icovid project). This work was also supported by core and project funding from the Wellcome/EPSRC [WT203148/Z/16/Z; NS/A000049/1; WT101957; NS/A000027/1]. This project utilized scribbles-based interactive segmentation tools from opensource project MONAI Label (https://github.com/Project-MONAI/MONAILabel) [7]. Publisher Copyright: © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023. Funding Information: Acknowledgment. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 101016131 (icovid project). This work was also supported by core and project funding from the Wellcome/EPSRC [WT203148/Z/16/Z; NS/A000049/1; WT101957; NS/A000027/1]. Publisher Copyright: © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023.

PY - 2023/10

Y1 - 2023/10

N2 - Existing interactive segmentation methods leverage automatic segmentation and user interactions for label refinement, significantly reducing the annotation workload compared to manual annotation. However, these methods lack quick adaptability to ambiguous and noisy data, which is a challenge in CT volumes containing lung lesions from COVID-19 patients. In this work, we propose an adaptive multi-scale online likelihood network (MONet) that adaptively learns in a data-efficient online setting from both an initial automatic segmentation and user interactions providing corrections. We achieve adaptive learning by proposing an adaptive loss that extends the influence of user-provided interaction to neighboring regions with similar features. In addition, we propose a data-efficient probability-guided pruning method that discards uncertain and redundant labels in the initial segmentation to enable efficient online training and inference. Our proposed method was evaluated by an expert in a blinded comparative study on COVID-19 lung lesion annotation task in CT. Our approach achieved 5.86% higher Dice score with 24.67% less perceived NASA-TLX workload score than the state-of-the-art. Source code is available at: https://github.com/masadcv/MONet-MONAILabel.

AB - Existing interactive segmentation methods leverage automatic segmentation and user interactions for label refinement, significantly reducing the annotation workload compared to manual annotation. However, these methods lack quick adaptability to ambiguous and noisy data, which is a challenge in CT volumes containing lung lesions from COVID-19 patients. In this work, we propose an adaptive multi-scale online likelihood network (MONet) that adaptively learns in a data-efficient online setting from both an initial automatic segmentation and user interactions providing corrections. We achieve adaptive learning by proposing an adaptive loss that extends the influence of user-provided interaction to neighboring regions with similar features. In addition, we propose a data-efficient probability-guided pruning method that discards uncertain and redundant labels in the initial segmentation to enable efficient online training and inference. Our proposed method was evaluated by an expert in a blinded comparative study on COVID-19 lung lesion annotation task in CT. Our approach achieved 5.86% higher Dice score with 24.67% less perceived NASA-TLX workload score than the state-of-the-art. Source code is available at: https://github.com/masadcv/MONet-MONAILabel.

UR - http://www.scopus.com/inward/record.url?scp=85174715979&partnerID=8YFLogxK

U2 - 10.1007/978-3-031-43895-0_53

DO - 10.1007/978-3-031-43895-0_53

M3 - Conference paper

AN - SCOPUS:85174715979

SN - 9783031438943

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 564

EP - 574

BT - Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 - 26th International Conference, Proceedings

A2 - Greenspan, Hayit

A2 - Madabhushi, Anant

A2 - Mousavi, Parvin

A2 - Salcudean, Septimiu

A2 - Duncan, James

A2 - Syeda-Mahmood, Tanveer

A2 - Taylor, Russell

PB - Springer Science and Business Media Deutschland GmbH

T2 - 26th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2023

Y2 - 8 October 2023 through 12 October 2023

ER -

Asad M, Williams H, Mandal I, Ather S, Deprest J, D’hooge J et al. Adaptive Multi-scale Online Likelihood Network for AI-Assisted Interactive Segmentation. In Greenspan H, Greenspan H, Madabhushi A, Mousavi P, Salcudean S, Duncan J, Syeda-Mahmood T, Taylor R, editors, Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 - 26th International Conference, Proceedings. Springer Science and Business Media Deutschland GmbH. 2023. p. 564-574. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). Epub 2023 Oct 1. doi: 10.1007/978-3-031-43895-0_53