3D Neurosurgical Simulator for Training Robotic Steerable Catheter Agents Using Generative Adversarial Imitation Learning

The complex anatomical structure of the brain and the vulnerability of its tissues make difficult to reach targets located in deep brain areas with rigid needles without causing damages to the adjacent tissues. Steerable catheter technology allows accessibility to the difficult anatomy. Since a steerable catheter can only unfold its full potential when following the best trajectory within the brain, there is the need of pre-operatively planning the path of the probe minimizing length of the trajectory and maximizing distance to vessels, while respecting the kinematic constraints of the catheter. Among path planning methods for surgical steerable catheters, graph based and the sampling based methods, are unable in approaching human level, generating a trajectory that does not take into account all the preferences of the expert surgeons. The approaches that may overcome these limitations are Learning Based methods, making a system more robust since system parameters are adjusted automatically. In this work an imitation learning method is applied in which the agent learns to perform the desired trajectory thanks to a set of demonstrations given by an expert surgeon. The approach chosen to accomplish this task is the exploitation of Generative Adversarial Imitation Learning (GAIL) that is a technique that takes advantage of demonstrations executed by experts, and learns both the policy and reward function of the unknown environment.