LNQ 2023 challenge: Benchmark of weakly-supervised techniques for mediastinal lymph node quantification

Reuben Dorent; Roya Khajavi; Tagwa Idris; Erik Ziegler; Bhanusupriya Somarouthu; Heather Jacene; Ann LaCasce; Jonathan Deissler; Jan Ehrhardt; Sofija Engelson; Stefan M. Fischer; Yun Gu; Heinz Handels; Satoshi Kasai; Satoshi Kondo; Klaus Maier-Hein; Julia A. Schnabel; Guotai Wang; Litingyu Wang; Tassilo Wald; Guang-Zhong Yang; Hanxiao Zhang; Minghui Zhang; Steve Pieper; Gordon Harris; Ron Kikinis; Tina Kapur

doi:10.59275/j.melba.2025-d482

LNQ 2023 challenge: Benchmark of weakly-supervised techniques for mediastinal lymph node quantification

Reuben Dorent, Roya Khajavi, Tagwa Idris, Erik Ziegler, Bhanusupriya Somarouthu, Heather Jacene, Ann LaCasce, Jonathan Deissler, Jan Ehrhardt, Sofija Engelson, Stefan M. Fischer, Yun Gu, Heinz Handels, Satoshi Kasai, Satoshi Kondo, Klaus Maier-Hein, Julia A. Schnabel, Guotai Wang, Litingyu Wang, Tassilo WaldGuang-Zhong Yang, Hanxiao Zhang, Minghui Zhang, Steve Pieper, Gordon Harris, Ron Kikinis, Tina Kapur

Research output: Contribution to journal › Article › peer-review

10 Downloads (Pure)

Abstract

Accurate assessment of lymph node size in 3D CT scans is crucial for cancer staging, therapeutic management, and monitoring treatment response. Existing state-of-the-art segmentation frameworks in medical imaging often rely on fully annotated datasets. However, for lymph node segmentation, these datasets are typically small due to the extensive time and expertise required to annotate the numerous lymph nodes in 3D CT scans. Weakly-supervised learning, which leverages incomplete or noisy annotations, has recently gained interest in the medical imaging community as a potential solution. Despite the variety of weakly-supervised techniques proposed, most have been validated only on private datasets or small publicly available datasets. To address this limitation, the Mediastinal Lymph Node Quantification (LNQ) challenge was organized in conjunction with the 26th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2023). This challenge aimed to advance weakly-supervised segmentation methods by providing a new, partially annotated dataset and a robust evaluation framework. A total of 16 teams from 5 countries submitted predictions to the validation leaderboard, and 6 teams from 3 countries participated in the evaluation phase. The results highlighted both the potential and the current limitations of weakly-supervised approaches. On one hand, weakly-supervised approaches obtained relatively good performance with a median Dice score of $61.0\%$. On the other hand, top-ranked teams, with a median Dice score exceeding $70\%$, boosted their performance by leveraging smaller but fully annotated datasets to combine weak supervision and full supervision. This highlights both the promise of weakly-supervised methods and the ongoing need for high-quality, fully annotated data to achieve higher segmentation performance.

Original language	Undefined/Unknown
Journal	Machine.Learning.for.Biomedical.Imaging.
DOIs	https://doi.org/10.59275/j.melba.2025-d482
Publication status	Published - 19 Aug 2024

Keywords

cs.CV

Access to Document

10.59275/j.melba.2025-d482

2408.10069v2

Cite this

Dorent, R., Khajavi, R., Idris, T., Ziegler, E., Somarouthu, B., Jacene, H., LaCasce, A., Deissler, J., Ehrhardt, J., Engelson, S., Fischer, S. M., Gu, Y., Handels, H., Kasai, S., Kondo, S., Maier-Hein, K., Schnabel, J. A., Wang, G., Wang, L., ... Kapur, T. (2024). LNQ 2023 challenge: Benchmark of weakly-supervised techniques for mediastinal lymph node quantification. Machine.Learning.for.Biomedical.Imaging. https://doi.org/10.59275/j.melba.2025-d482

@article{9ff4db33392d4888aa8fc24d9609ab81,

title = "LNQ 2023 challenge: Benchmark of weakly-supervised techniques for mediastinal lymph node quantification",

abstract = "Accurate assessment of lymph node size in 3D CT scans is crucial for cancer staging, therapeutic management, and monitoring treatment response. Existing state-of-the-art segmentation frameworks in medical imaging often rely on fully annotated datasets. However, for lymph node segmentation, these datasets are typically small due to the extensive time and expertise required to annotate the numerous lymph nodes in 3D CT scans. Weakly-supervised learning, which leverages incomplete or noisy annotations, has recently gained interest in the medical imaging community as a potential solution. Despite the variety of weakly-supervised techniques proposed, most have been validated only on private datasets or small publicly available datasets. To address this limitation, the Mediastinal Lymph Node Quantification (LNQ) challenge was organized in conjunction with the 26th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2023). This challenge aimed to advance weakly-supervised segmentation methods by providing a new, partially annotated dataset and a robust evaluation framework. A total of 16 teams from 5 countries submitted predictions to the validation leaderboard, and 6 teams from 3 countries participated in the evaluation phase. The results highlighted both the potential and the current limitations of weakly-supervised approaches. On one hand, weakly-supervised approaches obtained relatively good performance with a median Dice score of $61.0\%$. On the other hand, top-ranked teams, with a median Dice score exceeding $70\%$, boosted their performance by leveraging smaller but fully annotated datasets to combine weak supervision and full supervision. This highlights both the promise of weakly-supervised methods and the ongoing need for high-quality, fully annotated data to achieve higher segmentation performance.",

keywords = "cs.CV",

author = "Reuben Dorent and Roya Khajavi and Tagwa Idris and Erik Ziegler and Bhanusupriya Somarouthu and Heather Jacene and Ann LaCasce and Jonathan Deissler and Jan Ehrhardt and Sofija Engelson and Fischer, {Stefan M.} and Yun Gu and Heinz Handels and Satoshi Kasai and Satoshi Kondo and Klaus Maier-Hein and Schnabel, {Julia A.} and Guotai Wang and Litingyu Wang and Tassilo Wald and Guang-Zhong Yang and Hanxiao Zhang and Minghui Zhang and Steve Pieper and Gordon Harris and Ron Kikinis and Tina Kapur",

note = "Submitted to MELBA; Accepted for publication at the Journal of Machine Learning for Biomedical Imaging (MELBA) https://melba-journal.org/2025:001",

year = "2024",

month = aug,

day = "19",

doi = "10.59275/j.melba.2025-d482",

language = "Undefined/Unknown",

journal = "Machine.Learning.for.Biomedical.Imaging.",

}

Dorent, R, Khajavi, R, Idris, T, Ziegler, E, Somarouthu, B, Jacene, H, LaCasce, A, Deissler, J, Ehrhardt, J, Engelson, S, Fischer, SM, Gu, Y, Handels, H, Kasai, S, Kondo, S, Maier-Hein, K, Schnabel, JA , Wang, G, Wang, L, Wald, T, Yang, G-Z, Zhang, H, Zhang, M, Pieper, S, Harris, G, Kikinis, R & Kapur, T 2024, 'LNQ 2023 challenge: Benchmark of weakly-supervised techniques for mediastinal lymph node quantification', Machine.Learning.for.Biomedical.Imaging. https://doi.org/10.59275/j.melba.2025-d482

TY - JOUR

T1 - LNQ 2023 challenge

T2 - Benchmark of weakly-supervised techniques for mediastinal lymph node quantification

AU - Dorent, Reuben

AU - Khajavi, Roya

AU - Idris, Tagwa

AU - Ziegler, Erik

AU - Somarouthu, Bhanusupriya

AU - Jacene, Heather

AU - LaCasce, Ann

AU - Deissler, Jonathan

AU - Ehrhardt, Jan

AU - Engelson, Sofija

AU - Fischer, Stefan M.

AU - Gu, Yun

AU - Handels, Heinz

AU - Kasai, Satoshi

AU - Kondo, Satoshi

AU - Maier-Hein, Klaus

AU - Schnabel, Julia A.

AU - Wang, Guotai

AU - Wang, Litingyu

AU - Wald, Tassilo

AU - Yang, Guang-Zhong

AU - Zhang, Hanxiao

AU - Zhang, Minghui

AU - Pieper, Steve

AU - Harris, Gordon

AU - Kikinis, Ron

AU - Kapur, Tina

N1 - Submitted to MELBA; Accepted for publication at the Journal of Machine Learning for Biomedical Imaging (MELBA) https://melba-journal.org/2025:001

PY - 2024/8/19

Y1 - 2024/8/19

N2 - Accurate assessment of lymph node size in 3D CT scans is crucial for cancer staging, therapeutic management, and monitoring treatment response. Existing state-of-the-art segmentation frameworks in medical imaging often rely on fully annotated datasets. However, for lymph node segmentation, these datasets are typically small due to the extensive time and expertise required to annotate the numerous lymph nodes in 3D CT scans. Weakly-supervised learning, which leverages incomplete or noisy annotations, has recently gained interest in the medical imaging community as a potential solution. Despite the variety of weakly-supervised techniques proposed, most have been validated only on private datasets or small publicly available datasets. To address this limitation, the Mediastinal Lymph Node Quantification (LNQ) challenge was organized in conjunction with the 26th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2023). This challenge aimed to advance weakly-supervised segmentation methods by providing a new, partially annotated dataset and a robust evaluation framework. A total of 16 teams from 5 countries submitted predictions to the validation leaderboard, and 6 teams from 3 countries participated in the evaluation phase. The results highlighted both the potential and the current limitations of weakly-supervised approaches. On one hand, weakly-supervised approaches obtained relatively good performance with a median Dice score of $61.0\%$. On the other hand, top-ranked teams, with a median Dice score exceeding $70\%$, boosted their performance by leveraging smaller but fully annotated datasets to combine weak supervision and full supervision. This highlights both the promise of weakly-supervised methods and the ongoing need for high-quality, fully annotated data to achieve higher segmentation performance.

AB - Accurate assessment of lymph node size in 3D CT scans is crucial for cancer staging, therapeutic management, and monitoring treatment response. Existing state-of-the-art segmentation frameworks in medical imaging often rely on fully annotated datasets. However, for lymph node segmentation, these datasets are typically small due to the extensive time and expertise required to annotate the numerous lymph nodes in 3D CT scans. Weakly-supervised learning, which leverages incomplete or noisy annotations, has recently gained interest in the medical imaging community as a potential solution. Despite the variety of weakly-supervised techniques proposed, most have been validated only on private datasets or small publicly available datasets. To address this limitation, the Mediastinal Lymph Node Quantification (LNQ) challenge was organized in conjunction with the 26th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2023). This challenge aimed to advance weakly-supervised segmentation methods by providing a new, partially annotated dataset and a robust evaluation framework. A total of 16 teams from 5 countries submitted predictions to the validation leaderboard, and 6 teams from 3 countries participated in the evaluation phase. The results highlighted both the potential and the current limitations of weakly-supervised approaches. On one hand, weakly-supervised approaches obtained relatively good performance with a median Dice score of $61.0\%$. On the other hand, top-ranked teams, with a median Dice score exceeding $70\%$, boosted their performance by leveraging smaller but fully annotated datasets to combine weak supervision and full supervision. This highlights both the promise of weakly-supervised methods and the ongoing need for high-quality, fully annotated data to achieve higher segmentation performance.

KW - cs.CV

U2 - 10.59275/j.melba.2025-d482

DO - 10.59275/j.melba.2025-d482

M3 - Article

JO - Machine.Learning.for.Biomedical.Imaging.

JF - Machine.Learning.for.Biomedical.Imaging.

ER -