Network architecture and action space analysis for deep reinforcement learning towards spacecraft autonomous guidance

The growing ferment towards enhanced autonomy on-board spacecrafts is driving the research of leading space agencies. Concurrently, the rapid developments of Artificial Intelligence (AI) are strongly influencing the aerospace researches, regarding on-orbit servicing (OOS) activities above all. Within the wide spectrum of OOS and proximity operations, this work focuses on autonomous guidance of a chaser spacecraft for the map reconstruction of an artificial uncooperative target. Adaptive guidance is framed as an active Simultaneous Localization and Mapping (SLAM) problem and modeled as a Partially Observable Markov Decision Process (POMDP). A state-of-the-art Deep Reinforcement Learning (DRL) method, Proximal Policy Optimization (PPO), is investigated to develop an agent capable of cleverly planning the shape reconstruction of the uncooperative space object. The guidance algorithm performance are evaluated in terms of target map reconstruction, by rendering the space object with a triangular mesh and then considering the number of quality images for each face. A major differentiation in the algorithm implementation is provided by the employment of either a discrete or a continuous action space. The main differences between the two cases are critically commented and the benefits of a continuous action space are highlighted. The proposed model is trained and then extensively tested, always starting from random initial conditions, to verify the generalizing capabilities of the DRL agent, by means of the neural network architecture. On this note, a comparison analysis between a Feed-forward Neural Networks (FFNN) and a Recurrent Neural Network (RNN) is performed. The better performing model is retrieved from the aforementioned comparisons, and its robustness and sensitivity are sharply analyzed. This work confirms and develops further the applicability of DRL techniques for autonomous guidance, highlighting in a critical way its possible implementation in future close proximity scenarios.