Graph Neural Networks: A Suitable Alternative to MLPs in Latent 3D Medical Image Classification?

TY - JOUR

T1 - Graph Neural Networks

T2 - 6th International Workshop on Graphs in Biomedical Image Analysis, GRAIL 2024

AU - Kiechle, Johannes

AU - Lang, Daniel M.

AU - Fischer, Stefan M.

AU - Felsner, Lina

AU - Peeken, Jan C.

AU - Schnabel, Julia A.

N1 - Publisher Copyright: © The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.

PY - 2025/3/1

Y1 - 2025/3/1

N2 - Recent studies have underscored the capabilities of natural imaging foundation models to serve as powerful feature extractors, even in a zero-shot setting for medical imaging data. Most commonly, a shallow multi-layer perceptron (MLP) is appended to the feature extractor to facilitate end-to-end learning and downstream prediction tasks such as classification, thus representing the de facto standard. However, as graph neural networks (GNNs) have become a practicable choice for various tasks in medical research in the recent past, we direct attention to the question of how effective GNNs are compared to MLP prediction heads for the task of 3D medical image classification, proposing them as a potential alternative. In our experiments, we devise a subject-level graph for each volumetric dataset instance. Therein latent representations of all slices in the volume, encoded through a DINOv2 pretrained vision transformer (ViT), constitute the nodes and their respective node features. We use public datasets to compare the classification heads numerically and evaluate various graph construction and graph convolution methods in our experiments. Our findings show enhancements of the GNN in classification performance and substantial improvements in runtime compared to an MLP prediction head. Additional robustness evaluations further validate the promising performance of the GNN, promoting them as a suitable alternative to traditional MLP classification heads. Our code is publicly available at: https://github.com/compai-lab/2024-miccai-grail-kiechle.

AB - Recent studies have underscored the capabilities of natural imaging foundation models to serve as powerful feature extractors, even in a zero-shot setting for medical imaging data. Most commonly, a shallow multi-layer perceptron (MLP) is appended to the feature extractor to facilitate end-to-end learning and downstream prediction tasks such as classification, thus representing the de facto standard. However, as graph neural networks (GNNs) have become a practicable choice for various tasks in medical research in the recent past, we direct attention to the question of how effective GNNs are compared to MLP prediction heads for the task of 3D medical image classification, proposing them as a potential alternative. In our experiments, we devise a subject-level graph for each volumetric dataset instance. Therein latent representations of all slices in the volume, encoded through a DINOv2 pretrained vision transformer (ViT), constitute the nodes and their respective node features. We use public datasets to compare the classification heads numerically and evaluate various graph construction and graph convolution methods in our experiments. Our findings show enhancements of the GNN in classification performance and substantial improvements in runtime compared to an MLP prediction head. Additional robustness evaluations further validate the promising performance of the GNN, promoting them as a suitable alternative to traditional MLP classification heads. Our code is publicly available at: https://github.com/compai-lab/2024-miccai-grail-kiechle.

KW - Classification

KW - Graph Neural Networks

KW -  Graph Topology

UR - https://www.scopus.com/pages/publications/105000265757

U2 - 10.1007/978-3-031-83243-7_2

DO - 10.1007/978-3-031-83243-7_2

M3 - Conference paper

AN - SCOPUS:105000265757

SN - 0302-9743

VL - 15182

SP - 12

EP - 22

JO - Lecture Notes in Computer Science

JF - Lecture Notes in Computer Science

Y2 - 6 October 2024 through 6 October 2024

ER -