Dynamic Augmented Reality Cues for Telementoring in Laparoscopic Surgery: Usability Study

Background: Surgical telementoring enables a remote expert surgeon (mentor) to guide an operating surgeon (mentee) during surgery and facilitates the transfer of surgical skills. However, commonly used audio and static visual cues are inadequate to demonstrate complex tool-tissue interactions. To overcome this limitation, dynamic augmented reality (AR)–based visual cues are overlayed on the operative field to demonstrate precise instrument movements. Objective: The objective of this work was to evaluate dynamic AR cues and identify the most suitable cue that effectively demonstrates the required motions of surgical instruments during laparoscopy. Methods: A user study was conducted in a simulated environment among mentor-mentee pairs using 3 dynamic AR cues (hand gestures, a 3D pointer, and a virtual tool). The task assessed how closely the mentee was able to follow the mentor. The outcomes measured were (1) dynamic time warping distance, representing the closeness of the paths followed; (2) angular error in tooltip orientation; and (3) the NASA Task Load Index, assessing cognitive workload during telementoring. Results: Telementoring using the virtual tool resulted in a reduced dynamic time warping distance compared with hand gestures (P<.01 for 12/13, 92.3% of the trials) and the 3D pointer (P<.05 for 11/13, 84.6% of the trials). Lower orientation error was also noted while using the virtual tool as compared with hand gestures (P<.05 for 5 of 6 poses). There were no substantial differences in the NASA Task Load Index scores. Conclusions: Use of a virtual tool (as a dynamic AR cue) enabled the mentee to follow the mentor’s instructions with fewer errors than both hand gestures and a 3D pointer without increasing cognitive workload. Further research is needed to assess the clinical effectiveness of the virtual tool during live surgery.