Optical Flow-Based Organ Tracking System for Robotic Surgery Support

This study aims to develop an intraoperative navigation system for in-vivo procedures leveraging optical flow techniques. Traditional methods for organ tracking often fail in intraoperative scenarios due to domain complexity and, in the case of machine learning methods, by lack of annotated data. To address this challenge, we present a novel organ tracking system based on optical flow and evaluate the performance of four state-of-the-art neural networks for optical flow to determine the most suitable one to use in our system. The proposed system combines semantic segmentation and optical flow estimation. Segmentation networks identify the organ, surgical tools, and background in each frame. Then, deep learning-based optical flow networks estimate its motion across frames. The resulting motion is used to compute the organ’s rotation and translation and update the 3D virtual model accordingly. The performances of the four neural networks for optical flow are tested on two video sequences of a robot-assisted partial nephrectomy, where the tracked organ of interest is the kidney. Among the tested networks, and for the concerns of this specific domain of use, RAFT and GMFlow achieved the most promising results in terms of IoU accuracy.