A Reinforcement Learning-Based Continuation Strategy for Autonomous On-Orbit Assembly

Autonomous on-orbit assembly operations are critical technologies for space exploration and long-term colonization. Such operations are usually translated into complex optimal control problems requiring advanced methods to solve the resulting nonlinear programming. Nonlinear programming problems are prominent to be highly sensitive to the initial guess provided to the solver. To alleviate this situation, this paper presents a continuation strategy that significantly desensitizes the optimal control problem to the given initial guess. It became possible by forming a continuation space between a trivial problem with a known optimal solution and the desired optimal control problem using the allocation of some state, control, or constraint variables as the continuation parameters. To this end, a reinforcement learning search agent was defined based on the Sarsa(��) method to search the continuation space in real time for a continuation trajectory satisfying some desired criteria. Numerical experiments concerning the autonomous assembly of a robotic spacecraft with a complex target structure were performed to demonstrate the efficiency and capability of the proposed technique. Through various simulations, it has been shown how the complexity of the components’ shapes and the size of the problem affect the solver’s performance.