Stereo Vision Based Navigation for Lunar Rover: A Hardware-Software Co-Design Approach

Autonomous navigation of rovers on the Lunar South Pole is necessitated due to intermittent communication via relay satellites, signal propagation delays and limited bandwidth for teleoperation. Such an autonomous operation may be divided into four inter-connected processes: perception, localization, mapping, and control. All these processes must be executed onboard within a severely constrained computation and power budget. Modern CMOS camera sensors are well suited for perception under strict power and mass restrictions but require substantial computation power. However, the rover may move at a slow pace, which reduces the rate of computation required. Previous missions, including Mars Exploration Rovers (Spirit and Opportunity), Mars Science Laboratory (Curiosity) and Mars 2020 (Perseverance) have demonstrated progressively enhanced capabilities in autonomous driving using stereo images. We propose a solution that utilizes a pair of stereo cameras and an Inertial Measurement Unit (IMU), as primary sensors, to construct a visual-inertial odometry (VIO) and stereo 3D reconstruction pipeline. To address the limited power availability onboard the rover, the algorithms are implemented on a power efficient System-on-Chip (SoC) that incorporates a quad-core RISC-V processor and an FPGA fabric, both tightly integrated with AXI-4 interface. All algorithms are architected such that the most compute-intensive and data-parallelizable components are delegated to the FPGA, while control and floating-point intensive tasks are executed on the CPU. The components of the pipeline, in the order of image data flow, are respectively: histogram equalization, feature detection, feature matching and tracking, Multi-State Constraint Kalman Filter (MSCKF) pose estimation. Further, Semi-Global Matching (SGM) is implemented to generate a disparity map from the stereo camera to produce a 3D point cloud. It is used to perform local mapping, traversability analysis, and rover path planning. Histogram equalization, feature detection, feature matching and tracking, rectification and SGM stereo disparity are implemented on the FPGA fabric. The FPGA algorithms are initialized, controlled, and synchronized with the CPU using a DDR memory mapped Advanced Peripheral Bus (APB) interface. The CPU runs an embedded Linux operating system with custom drivers facilitating the FPGA interface. The selection of algorithms are driven by a trade-off between mission requirements, data throughput and system power consumption. Additionally, the solution is validated in a terrestrial environment with moon-like surface features. Although the current solution is implemented on commercial off-the-shelf (COTS) components, we are anticipating a High Performance Space Compute (HPSC) infrastructure to enable reliable onboard processing.