Learning to Match 2D Keypoints Across Preoperative MR and Intraoperative Ultrasound

Hassan Rasheed; Reuben Dorent; Maximilian Fehrentz; Tina Kapur; William M. Wells III; Alexandra Golby; Sarah Frisken; Julia A. Schnabel; Nazim Haouchine

Learning to Match 2D Keypoints Across Preoperative MR and Intraoperative Ultrasound

Hassan Rasheed, Reuben Dorent, Maximilian Fehrentz, Tina Kapur, William M. Wells III, Alexandra Golby, Sarah Frisken, Julia A. Schnabel, Nazim Haouchine

Research output: Working paper/Preprint › Preprint

9 Downloads (Pure)

Abstract

We propose in this paper a texture-invariant 2D keypoints descriptor specifically designed for matching preoperative Magnetic Resonance (MR) images with intraoperative Ultrasound (US) images. We introduce a matching-by-synthesis strategy, where intraoperative US images are synthesized from MR images accounting for multiple MR modalities and intraoperative US variability. We build our training set by enforcing keypoints localization over all images then train a patient-specific descriptor network that learns texture-invariant discriminant features in a supervised contrastive manner, leading to robust keypoints descriptors. Our experiments on real cases with ground truth show the effectiveness of the proposed approach, outperforming the state-of-the-art methods and achieving 80.35% matching precision on average.

Original language	Undefined/Unknown
Publication status	Published - 12 Sept 2024

Keywords

cs.CV

Access to Document

2409.08169v1

Cite this

@techreport{6e35feb39c334dfba5771a588d970436,

title = "Learning to Match 2D Keypoints Across Preoperative MR and Intraoperative Ultrasound",

abstract = "We propose in this paper a texture-invariant 2D keypoints descriptor specifically designed for matching preoperative Magnetic Resonance (MR) images with intraoperative Ultrasound (US) images. We introduce a matching-by-synthesis strategy, where intraoperative US images are synthesized from MR images accounting for multiple MR modalities and intraoperative US variability. We build our training set by enforcing keypoints localization over all images then train a patient-specific descriptor network that learns texture-invariant discriminant features in a supervised contrastive manner, leading to robust keypoints descriptors. Our experiments on real cases with ground truth show the effectiveness of the proposed approach, outperforming the state-of-the-art methods and achieving 80.35% matching precision on average.",

keywords = "cs.CV",

author = "Hassan Rasheed and Reuben Dorent and Maximilian Fehrentz and Tina Kapur and III, {William M. Wells} and Alexandra Golby and Sarah Frisken and Schnabel, {Julia A.} and Nazim Haouchine",

note = "Accepted for publication at the International Workshop of Advances in Simplifying Medical UltraSound (ASMUS) at MICCAI 2024",

year = "2024",

month = sep,

day = "12",

language = "Undefined/Unknown",

type = "WorkingPaper",

}

TY - UNPB

T1 - Learning to Match 2D Keypoints Across Preoperative MR and Intraoperative Ultrasound

AU - Rasheed, Hassan

AU - Dorent, Reuben

AU - Fehrentz, Maximilian

AU - Kapur, Tina

AU - III, William M. Wells

AU - Golby, Alexandra

AU - Frisken, Sarah

AU - Schnabel, Julia A.

AU - Haouchine, Nazim

N1 - Accepted for publication at the International Workshop of Advances in Simplifying Medical UltraSound (ASMUS) at MICCAI 2024

PY - 2024/9/12

Y1 - 2024/9/12

N2 - We propose in this paper a texture-invariant 2D keypoints descriptor specifically designed for matching preoperative Magnetic Resonance (MR) images with intraoperative Ultrasound (US) images. We introduce a matching-by-synthesis strategy, where intraoperative US images are synthesized from MR images accounting for multiple MR modalities and intraoperative US variability. We build our training set by enforcing keypoints localization over all images then train a patient-specific descriptor network that learns texture-invariant discriminant features in a supervised contrastive manner, leading to robust keypoints descriptors. Our experiments on real cases with ground truth show the effectiveness of the proposed approach, outperforming the state-of-the-art methods and achieving 80.35% matching precision on average.

AB - We propose in this paper a texture-invariant 2D keypoints descriptor specifically designed for matching preoperative Magnetic Resonance (MR) images with intraoperative Ultrasound (US) images. We introduce a matching-by-synthesis strategy, where intraoperative US images are synthesized from MR images accounting for multiple MR modalities and intraoperative US variability. We build our training set by enforcing keypoints localization over all images then train a patient-specific descriptor network that learns texture-invariant discriminant features in a supervised contrastive manner, leading to robust keypoints descriptors. Our experiments on real cases with ground truth show the effectiveness of the proposed approach, outperforming the state-of-the-art methods and achieving 80.35% matching precision on average.

KW - cs.CV

M3 - Preprint

BT - Learning to Match 2D Keypoints Across Preoperative MR and Intraoperative Ultrasound

ER -