Q-Learning-Based Model Predictive Variable Impedance Control for Physical Human-Robot Collaboration (Extended Abstract)

Physical human-robot collaboration is increasingly required in many contexts. To implement an effective collaboration, the robot should be able to recognize the human's intentions and guarantee safe and adaptive behavior along the desired directions of motion. The robot-control strategies with such attributes are particularly demanded in the industrial field. Indeed, with this aim, this work proposes a Q-Learning-based Model Predictive Variable Impedance Control (Q-LMPVIC) to assist the operators in physical human-robot collaboration (pHRC) tasks. A Cartesian impedance control loop is designed to implement the decoupled compliant robot dynamics. The impedance control parameters (i.e., setpoint and damping parameters) are then optimized in an online manner to maximize the performance of the pHRC. First, an ensemble of neural networks is designed to learn the model of human-robot interaction dynamics while capturing the associated uncertainties. The derived model is then used by the model predictive controller (MPC), enhanced with stability guarantees through Lyapunov constraints. The MPC is solved by making use of a Q-Learning method that, in its online implementation, uses an actor-critic algorithm to approximate the exact solution. The Q-learning method provides an accurate and highly efficient solution (in terms of computational time and resources). The proposed approach has been validated through experimental tests on a Franka EMIKA panda robot.