Meta-Reinforcement Learning with Transformer Networks for Space Guidance Applications

Transformer neural networks have revolutionized machine learning, excelling in text and image processing. Their self-attention mechanism captures sequence dependencies, facilitating feature extraction and avoiding gradient problems of recurrent networks. Transformers naturally implement a meta-reinforcement learning framework when used in reinforcement learning, using self-attention weights as context-dependent parameters for task inference. This paper proposes a meta-reinforcement learning algorithm based on the gated transformer-XL model for autonomous spacecraft guidance during a planetary landing, by considering the presence of unmodeled dynamics, inaccurate navigation data, and control errors. The method will be compared with standard reinforcement learning via a feed-forward network to demonstrate the potential of transformers for real-time and robust spacecraft guidance in uncertain mission scenarios.