Reinforced Model Predictive Guidance and Control for Spacecraft Proximity Operations

An increased level of autonomy is attractive above all in the framework of proximity operations, and researchers are focusing more and more on artificial intelligence techniques to improve spacecraft's capabilities in these scenarios. This work presents an autonomous AI-based guidance algorithm to plan the path of a chaser spacecraft for the map reconstruction of an artificial uncooperative target, coupled with Model Predictive Control for the tracking of the generated trajectory. Deep reinforcement learning is particularly interesting for enabling spacecraft's autonomous guidance, since this problem can be formulated as a Partially Observable Markov Decision Process and because it leverages domain randomization well to cope with model uncertainty, thanks to the neural networks' generalizing capabilities. The main drawback of this method is that it is difficult to verify its optimality mathematically and the constraints can be added only as part of the reward function, so it is not guaranteed that the solution satisfies them. To this end a convex Model Predictive Control formulation is employed to track the DRL-based trajectory, while simultaneously enforcing compliance with the constraints. Two neural network architectures are proposed and compared: a recurrent one and the more recent transformer. The trained reinforcement learning agent is then tested in an end-to-end AI-based pipeline with image generation in the loop, and the results are presented. The computational effort of the entire guidance and control strategy is also verified on a Raspberry Pi board. This work represents a viable solution to apply artificial intelligence methods for spacecraft's autonomous motion, still retaining a higher level of explainability and safety than that given by more classical guidance and control approaches.