Toward human-out-of-the-loop endoscope navigation based on context awareness for enhanced autonomy in robotic surgery

Although the da Vinci surgical system enhances manipulation dexterity and restores 3D vision in robotic surgery, it requires surgeons to asynchronously control surgical instruments and the endoscope, which hinders a smooth operation. Surgeons frequently position the endoscope to maintain a good field of view during operation, potentially increasing surgical time and workload. In this paper, a Human-Out-Of-The-Loop (HOOTL) endoscope navigation control with the assistance of context awareness is proposed to enhance surgical autonomy. A comprehensive comparison study using 8 state-of-the-art networks was conducted to find out the best model for surgical phase recognition. Ten human subjects were invited to participate in a classic ring transferring task based on three different endoscope navigation pipelines on a da Vinci research kit platform, including standard endoscope navigation, semi-autonomous endoscope navigation with manual pedal control, and HOOTL endoscope navigation supported by vision-based phase recognition. The experimental results showed that the proposed endoscope navigation approach releases the operation need of controlling the pedals, and it significantly reduces the execution time compared to the other two navigation pipelines. The result of the NASA Task Load Index (NASA-TLX) questionnaire indicates that the proposed endoscope navigation can reduce the physical and mental load for the users.