UM-CAM: Uncertainty-weighted multi-resolution class activation maps for weakly-supervised segmentation

TY - JOUR

T1 - UM-CAM

T2 - Uncertainty-weighted multi-resolution class activation maps for weakly-supervised segmentation

AU - Fu, Jia

AU - Wang, Guotai

AU - Lu, Tao

AU - Yue, Qiang

AU - Vercauteren, Tom

AU - Ourselin, Sébastien

AU - Zhang, Shaoting

N1 - Publisher Copyright: © 2024 The Authors

PY - 2025/4

Y1 - 2025/4

N2 - Weakly-supervised medical image segmentation methods utilizing image-level labels have gained attention for reducing the annotation cost. They typically use Class Activation Maps (CAM) from a classification network but struggle with incomplete activation regions due to low-resolution localization without detailed boundaries. Differently from most of them that only focus on improving the quality of CAMs, we propose a more unified weakly-supervised segmentation framework with image-level supervision. Firstly, an Uncertainty-weighted Multi-resolution Class Activation Map (UM-CAM) is proposed to generate high-quality pixel-level pseudo-labels. Subsequently, a Geodesic distance-based Seed Expansion (GSE) strategy is introduced to rectify ambiguous boundaries in the UM-CAM by leveraging contextual information. To train a final segmentation model from noisy pseudo-labels, we introduce a Random-View Consensus (RVC) training strategy to suppress unreliable pixel/voxels and encourage consistency between random-view predictions. Extensive experiments on 2D fetal brain segmentation and 3D brain tumor segmentation tasks showed that our method significantly outperforms existing weakly-supervised methods. Code is available at: https://github.com/HiLab-git/UM-CAM.

AB - Weakly-supervised medical image segmentation methods utilizing image-level labels have gained attention for reducing the annotation cost. They typically use Class Activation Maps (CAM) from a classification network but struggle with incomplete activation regions due to low-resolution localization without detailed boundaries. Differently from most of them that only focus on improving the quality of CAMs, we propose a more unified weakly-supervised segmentation framework with image-level supervision. Firstly, an Uncertainty-weighted Multi-resolution Class Activation Map (UM-CAM) is proposed to generate high-quality pixel-level pseudo-labels. Subsequently, a Geodesic distance-based Seed Expansion (GSE) strategy is introduced to rectify ambiguous boundaries in the UM-CAM by leveraging contextual information. To train a final segmentation model from noisy pseudo-labels, we introduce a Random-View Consensus (RVC) training strategy to suppress unreliable pixel/voxels and encourage consistency between random-view predictions. Extensive experiments on 2D fetal brain segmentation and 3D brain tumor segmentation tasks showed that our method significantly outperforms existing weakly-supervised methods. Code is available at: https://github.com/HiLab-git/UM-CAM.

KW - Brain tumor

KW - Class activation map

KW - Exponential geodesic distance

KW - Noisy label

KW - Segmentation

UR - http://www.scopus.com/inward/record.url?scp=85210135546&partnerID=8YFLogxK

U2 - 10.1016/j.patcog.2024.111204

DO - 10.1016/j.patcog.2024.111204

M3 - Article

AN - SCOPUS:85210135546

SN - 0031-3203

VL - 160

JO - PATTERN RECOGNITION

JF - PATTERN RECOGNITION

M1 - 111204

ER -