Research/ SPACE AI & AUTONOMY
Fast model inference and training on-board of Satellites
Vít Růžička, Gonzalo Mateo-García, Chris Bridges, Chris Brunskill, Cormac Purcell, Nicolas Longépé, Andrew Markham
Learning-based MethodsOpenAlex topic: Advanced Neural Network Applications · Perception & State EstimationarXiv: cs.CV · Design, Mechanisms & HardwareOpenAlex keyword: edge computing
arXivcs.AI · cs.CVOpenAlexAdvanced Neural Network Applications · Spacecraft Design and Technology · Space Technology and Applications
Artificial intelligence onboard satellites has the potential to reduce data transmission requirements, enable real-time decision-making and collaboration within constellations. This study deploys a lightweight foundational model called RaVAEn on D-Orbit's ION SCV004 satellite. RaVAEn is a variational auto-encoder (VAE) that generates compressed latent vectors from small image tiles, enabling several downstream tasks. In this work we demonstrate the reliable use of RaVAEn onboard a satellite, achieving an encoding time of 0.110s for tiles of a 4.8x4.8 km$^2$ area. In addition, we showcase fast few-shot training onboard a satellite using the latent representation of data. We compare the deployment of the model on the on-board CPU and on the available Myriad vision processing unit (VPU) accelerator. To our knowledge, this work shows for the first time the deployment of a multi-task model on-board a CubeSat and the on-board training of a machine learning model.
Comments: 4 pages, 4 figures, International Geoscience and Remote Sensing Symposium (IGARSS) 2023