Learned Optical Flow for Intra-Operative Tracking of the Retinal Fundus

Claudio Ravasio, Theodoros Pissas, Edward Bloch, Blanca Flores, Sepehr Jalali, Danail Stoyanov, M. Jorge Cardoso, Lyndon Da Cruz, Christos Bergeles

Research output: Contribution to journalArticlepeer-review

7 Citations (Scopus)


Sustained delivery of regenerative retinal therapies by robotic systems requires intra-operative tracking of the retinal fundus. We propose a supervised deep convolutional neural network to densely predict semantic segmentation and optical flow of the retina as mutually supportive tasks, implicitly inpainting retinal flow information missing due to occlusion by surgical tools.

As manual annotation of optical flow is infeasible, we propose a flexible algorithm for generation of large synthetic training datasets on the basis of given intra-operative retinal images. We evaluate optical flow estimation by tracking a grid and sparsely annotated ground truth points on a benchmark of challenging real intra-operative clips obtained from an extensive internally acquired dataset encompassing representative vitreoretinal surgical cases.

The U-Net-based network trained on the synthetic dataset is shown to generalise well to the benchmark of real surgical videos. When used to track retinal points of interest, our flow estimation outperforms variational baseline methods on clips containing tool motions which occlude the points of interest, as is routinely observed in intra-operatively recorded surgery videos.

The results indicate that complex synthetic training datasets can be used to specifically guide optical flow estimation. Our proposed algorithm therefore lays the foundation for a robust system which can assist with intra-operative tracking of moving surgical targets even when occluded.
Original languageEnglish
Pages (from-to)827-836
Number of pages10
JournalInternational Journal of Computer Assisted Radiology and Surgery
Issue number5
Publication statusPublished - 1 May 2020


  • Deep learning
  • Optical flow
  • Retinal tracking
  • Synthetic data


Dive into the research topics of 'Learned Optical Flow for Intra-Operative Tracking of the Retinal Fundus'. Together they form a unique fingerprint.

Cite this