Precise landing site detection for UAVs using deep learning

Autonomous landing site detection is a core requirement for UAV operations in unstructured planetary-analog terrain. This thesis presents a monocular vision-based deep learning framework that combines joint semantic segmentation and optical flow estimation with a tile-level safe landing site detection module, validated as part of the Rover Aerial Mars Support and Exploration System (RAMSES) experiment in the AMADEE-24 Mars analog mission. The perception core, RAMSES AI Network (RAN), is implemented as a dual-branch deep model based on SegFlow and adapted for Mars-analog terrain analysis. The Landing Site Assessment Module (LSAM) converts branch outputs into interpretable safety heatmaps and ranks candidate landing tiles. To address the lack of suitable multi-modal aerial datasets, a custom synthetic dataset was generated in NVIDIA Isaac Sim, comprising over 170,000 time-synchronized RGB frames with semantic segmentation and optical flow ground truth across eight environments, plus one unseen environment for out-of-domain testing. The model was first trained and validated on TartanAir, then benchmarked on Isaac Sim, and finally assessed on real-world imagery from the AMADEE-24 mission. In the Isaac Sim in-domain evaluation, RAN achieved 62.70% mean IoU for segmentation and 1.28 px mean EPE for optical flow. On the unseen environment, mean IoU dropped to 22.27% as expected under domain shift, while the safety-critical ground class remained strong at 80.55% IoU and optical flow remained robust at0.44 px mean EPE. On AMADEE-24, the analysis identified three dominant failures: shadow sensitivity, velocity saturation from large inter-frame displacement, and texture-domain gap. Targeted mitigations, namely orientation-aware inference, dense temporal resampling, and lighting randomization, improved LSAM performance from 63.5% to 69.6% accuracy and from 68.2% to 78.7% F1 on the refined evaluation set. Branch complementarity analysis further confirmed the fault tolerance of the dual-branch design. On the unrefined AMADEE-24 dataset, when one branch failed, the other successfully compensated in 45.2% of instances, with simultaneous failure of both branches occurring in only 9.21% of cases. The results show that combining appearance and motion cues yields robust, fault-tolerant landing assessment under challenging real-world analog conditions.