Deep Reinforcement Learning Based System for Intraoperative Hyperspectral Video Autofocusing

Charlie Budd*, Jianrong Qiu, Oscar MacCormac, Martin Huber, Christopher Mower, Mirek Janatka, Théo Trotouin, Jonathan Shapey, Mads S. Bergholt, Tom Vercauteren

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference paperpeer-review

Abstract

Hyperspectral imaging (HSI) captures a greater level of spectral detail than traditional optical imaging, making it a potentially valuable intraoperative tool when precise tissue differentiation is essential. Hardware limitations of current optical systems used for handheld real-time video HSI result in a limited focal depth, thereby posing usability issues for integration of the technology into the operating room. This work integrates a focus-tunable liquid lens into a video HSI exoscope, and proposes novel video autofocusing methods based on deep reinforcement learning. A first-of-its-kind robotic focal-time scan was performed to create a realistic and reproducible testing dataset. We benchmarked our proposed autofocus algorithm against traditional policies, and found our novel approach to perform significantly (p< 0.05 ) better than traditional techniques (0.070 ±. 098 mean absolute focal error compared to 0.146 ±. 148 ). In addition, we performed a blinded usability trial by having two neurosurgeons compare the system with different autofocus policies, and found our novel approach to be the most favourable, making our system a desirable addition for intraoperative HSI.

Original languageEnglish
Title of host publicationMedical Image Computing and Computer Assisted Intervention – MICCAI 2023
Subtitle of host publication26th International Conference, Vancouver, BC, Canada, October 8–12, 2023, Proceedings, Part IX
EditorsHayit Greenspan, Anant Madabhushi, Parvin Mousavi, Septimiu Salcudean, James Duncan, Tanveer Syeda-Mahmood, Russell Taylor
Place of PublicationCham
PublisherSpringer Nature
Pages658-667
Number of pages10
ISBN (Electronic)9783031439964
ISBN (Print)9783031439957
DOIs
Publication statusPublished - 2 Oct 2023
Event26th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2023 - Vancouver, Canada
Duration: 8 Oct 202312 Oct 2023

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume14228 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference26th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2023
Country/TerritoryCanada
CityVancouver
Period8/10/202312/10/2023

Keywords

  • Autofocus
  • Computer Assisted Intervention
  • Deep Reinforcement Learning
  • Hyperspectral Imaging

Fingerprint

Dive into the research topics of 'Deep Reinforcement Learning Based System for Intraoperative Hyperspectral Video Autofocusing'. Together they form a unique fingerprint.

Cite this