Zero-shot semantic landmark-based visual odometry using foundation models for unstructured planetary exploration

Precise autonomous navigation on unstructured planetary surfaces is a critical prerequisite for future exploration missions, particularly in GNSS-denied environments such as the Lunar South Pole or Martian deserts. Traditional Visual Odometry (VO) methods, which rely on tracking low-level geometric features (e.g., corners), often fail under the extreme illumination contrast of the Moon or the textural monotony of the Martian regolith. In this work, we present a zero-shot semantic landmark-based visual odometry approach that leverages the generalization capabilities of modern Foundation Models. Our approach uses the Segment Anything Model (SAM) to extract geological landmarks (rocks) and DINOv2 to generate view-invariant semantic descriptors that are matched across frames. We evaluate our pipeline across two distinct domains: a high-fidelity synthetic lunar environment (LuSNAR dataset) to test robustness against extreme lighting, and a real-world Martian analog dataset (Katwijk Beach) to assess sim-to-real transfer. Experimental results show that the proposed approach achieves a decimeter-level trajectory accuracy ( R M S E ≈ 0.14 m) on the Martian analog and an R M S E = 1.93 m on the most stable lunar traverse, without any domain-specific fine-tuning. Our results suggest that Foundation-Model-based semantic landmarks are a promising alternative to low-level features for zero-shot VO in planetary-like environments.