General Vision Encoder Features as Guidance in Medical Image Registration

Fryderyk Kögl; Anna Reithmeir; Vasiliki Sideri-Lampretsa; Ines Machado; Rickmer Braren; Daniel Rückert; Julia A. Schnabel; Veronika A. Zimmer

General Vision Encoder Features as Guidance in Medical Image Registration

Fryderyk Kögl, Anna Reithmeir, Vasiliki Sideri-Lampretsa, Ines Machado, Rickmer Braren, Daniel Rückert, Julia A. Schnabel, Veronika A. Zimmer

Research output: Working paper/Preprint › Preprint

11 Downloads (Pure)

Abstract

General vision encoders like DINOv2 and SAM have recently transformed computer vision. Even though they are trained on natural images, such encoder models have excelled in medical imaging, e.g., in classification, segmentation, and registration. However, no in-depth comparison of different state-of-the-art general vision encoders for medical registration is available. In this work, we investigate how well general vision encoder features can be used in the dissimilarity metrics for medical image registration. We explore two encoders that were trained on natural images as well as one that was fine-tuned on medical data. We apply the features within the well-established B-spline FFD registration framework. In extensive experiments on cardiac cine MRI data, we find that using features as additional guidance for conventional metrics improves the registration quality. The code is available at github.com/compai-lab/2024-miccai-koegl.

Original language	Undefined/Unknown
Publication status	Published - 18 Jul 2024

Keywords

cs.CV

Access to Document

2407.13311v1

Cite this

@techreport{04aec13945a64741af02703308ec6cfe,

title = "General Vision Encoder Features as Guidance in Medical Image Registration",

abstract = "General vision encoders like DINOv2 and SAM have recently transformed computer vision. Even though they are trained on natural images, such encoder models have excelled in medical imaging, e.g., in classification, segmentation, and registration. However, no in-depth comparison of different state-of-the-art general vision encoders for medical registration is available. In this work, we investigate how well general vision encoder features can be used in the dissimilarity metrics for medical image registration. We explore two encoders that were trained on natural images as well as one that was fine-tuned on medical data. We apply the features within the well-established B-spline FFD registration framework. In extensive experiments on cardiac cine MRI data, we find that using features as additional guidance for conventional metrics improves the registration quality. The code is available at github.com/compai-lab/2024-miccai-koegl.",

keywords = "cs.CV",

author = "Fryderyk K{\"o}gl and Anna Reithmeir and Vasiliki Sideri-Lampretsa and Ines Machado and Rickmer Braren and Daniel R{\"u}ckert and Schnabel, {Julia A.} and Zimmer, {Veronika A.}",

note = "Accepted at WBIR MICCAI 2024",

year = "2024",

month = jul,

day = "18",

language = "Undefined/Unknown",

type = "WorkingPaper",

}

TY - UNPB

T1 - General Vision Encoder Features as Guidance in Medical Image Registration

AU - Kögl, Fryderyk

AU - Reithmeir, Anna

AU - Sideri-Lampretsa, Vasiliki

AU - Machado, Ines

AU - Braren, Rickmer

AU - Rückert, Daniel

AU - Schnabel, Julia A.

AU - Zimmer, Veronika A.

N1 - Accepted at WBIR MICCAI 2024

PY - 2024/7/18

Y1 - 2024/7/18

N2 - General vision encoders like DINOv2 and SAM have recently transformed computer vision. Even though they are trained on natural images, such encoder models have excelled in medical imaging, e.g., in classification, segmentation, and registration. However, no in-depth comparison of different state-of-the-art general vision encoders for medical registration is available. In this work, we investigate how well general vision encoder features can be used in the dissimilarity metrics for medical image registration. We explore two encoders that were trained on natural images as well as one that was fine-tuned on medical data. We apply the features within the well-established B-spline FFD registration framework. In extensive experiments on cardiac cine MRI data, we find that using features as additional guidance for conventional metrics improves the registration quality. The code is available at github.com/compai-lab/2024-miccai-koegl.

AB - General vision encoders like DINOv2 and SAM have recently transformed computer vision. Even though they are trained on natural images, such encoder models have excelled in medical imaging, e.g., in classification, segmentation, and registration. However, no in-depth comparison of different state-of-the-art general vision encoders for medical registration is available. In this work, we investigate how well general vision encoder features can be used in the dissimilarity metrics for medical image registration. We explore two encoders that were trained on natural images as well as one that was fine-tuned on medical data. We apply the features within the well-established B-spline FFD registration framework. In extensive experiments on cardiac cine MRI data, we find that using features as additional guidance for conventional metrics improves the registration quality. The code is available at github.com/compai-lab/2024-miccai-koegl.

KW - cs.CV

M3 - Preprint

BT - General Vision Encoder Features as Guidance in Medical Image Registration

ER -