Geo-LoFTR: Geometry-aided Vision-based Localization of Future Mars Helicopters in Challenging Illumination Conditions
We present Geo-LoFTR, a geometry-enhanced deep image matcher for robust map-based localization of future Mars rotorcraft under severe illumination variations. By incorporating digital terrain models alongside visual appearance, Geo-LoFTR improves image registration between onboard navigation images and orbital maps, providing higher localization accuracy over prior methods.
☆ The model is trained using large-scale synthetic Mars imagery generated with our Mars Aerial Rendering Tool for Imaging And Navigation (MARTIAN) framework.
Abstract
Planetary exploration using aerial assets has the potential for unprecedented scientific discoveries on Mars. While NASA's Mars helicopter Ingenuity proved flight in Martian atmosphere is possible, future Mars rotorcraft will require advanced navigation capabilities for long-range flights. One such critical capability is Map-based Localization (MbL) which registers an onboard image to a reference map during flight to mitigate cumulative drift from visual odometry. However, significant illumination differences between rotocraft observations and a reference map prove challenging for traditional MbL systems, restricting the operational window of the vehicle. In this work, we investigate a new MbL system and propose Geo-LoFTR, a geometry-aided deep learning model for image registration that is more robust under large illumination differences than prior models. The system is supported by a custom simulation framework that uses real orbital maps to produce large amounts of realistic images of the Martian terrain. Comprehensive evaluations show that our proposed system outperforms prior MbL efforts in terms of localization accuracy under significant lighting and scale variations. Furthermore, we demonstrate the validity of our approach across a simulated Martian day, and on real Mars imagery.
Matching under harsh lighting
Geo-LoFTR extends LoFTR by incorporating geometric context derived from digital terrain models alongside visual appearance cues. This geometry-aware formulation improves correspondence robustness under large illumination and scale variations between onboard imagery and the reference orbital maps.
Robust matching under illumination changes. The figure above compares Geo-LoFTR against SuperPoint+SuperGlue and SuperPoint+LightGlue (both pre-trained on MegaDepth), as well as SIFT, on a representative query image (left in each panel; AZ=0°, EL=10°) matched to map search-area images (right) rendered with three different sun azimuths and 0° elevation offset. Match lines are color-coded by confidence (red = higher).
Integration within the Map-Based Localization pipeline
Within the Map-Based Localization (MbL) framework, Geo-LoFTR operates on overlapping windows sampled from a map search area defined by a pose prior. For each region, correspondences between the onboard navigation image and yhe geo-referenced orbital map are estimated. These matches are filtered and used in a RANSAC-PnP stage to recover the rotorcraft pose in the global reference frame.
Map-Based Localization pipeline. Given geo-referenced orthographic grayscale and depth maps and an onboard navigation image from a Mars rotorcraft, the goal is to estimate the vehicle pose in the global frame. A noisy pose prior defines a localized search area, which is divided into map and depth windows. Each window is processed together with the query image by Geo-LoFTR to establish correspondences. The filtered matches are then used within a RANSAC-PnP stage to recover the rotorcraft pose.
MbL Evaluation
Localization over a simulated Martian day
To evaluate localization robustness under realistic illumination changes, we simulated a full Martian day over the Jezero crater site, using HiRISE orbital maps (0.25 mpx) and Digital Terrain Models (1 m post spacing). Sun position was varied according to Mars ephemerides on 2031-05-10, and nadir-pointing query images were generated from 5:30 to 17:00 Local Mean Solar Time (LMST) across altitudes ranging from 64 to 200 meters. A fixed orthographic map rendered at 15:00 LMST served as HiRISE-like reference.
Illumination variation across a simulated Martian day. Sample query observations rendered at different Local Mean Solar Times (LMST) on 2031-05-10 over Jezero crater. The orthographic HiRISE-like reference map is rendered at 15:00 LMST, while the Sun reaches zenith at 11:29 LMST. Significant changes in shadowing and contrast occur throughout the day.
Localization accuracy across a simulated Martian day. @1m localization accuracy as a function of Local Mean Solar Time (LMST) for simulated test observations over Jezero crater on 2031-05-10, across the 64–200 m altitude range. The reference HiRISE-like map is rendered at 15:00 LMST (dashed line), while the Sun reaches zenith at 11:29 LMST.
Robustness to changing solar angles
We further evaluate robustness to variations in Sun elevation and azimuth by registering query observations onto orthographic maps rendered under different lighting configurations. These experiments isolate the effects of solar geometry while keeping terrain and viewpoint fixed.
Across a wide range of elevation offsets — including extreme low-elevation cases not seen during training — Geo-LoFTR consistently outperforms SuperPoint+SuperGlue, SuperPoint+LightGlue, and SIFT. While all methods degrade under very low Sun elevations due to severe shadowing, Geo-LoFTR remains the most accurate, indicating that incorporating depth information mitigates ambiguities in purely appearance-based matching.
Similarly, under varying Sun azimuth angles, Geo-LoFTR maintains stable localization performance across the full 360° range. In contrast to purely visual approaches, the geometry-enhanced formulation demonstrates improved robustness to both lighting shifts and altitude-induced scale variations.
Localization accuracy under varying solar geometry. @1m accuracy as a function of map Sun elevation (top) and azimuth (bottom) for simulated observations across three altitude ranges (64–200 m). Sun azimuth spans the full [−180°, 180°] range. The dashed line indicates lighting conditions matching the query observations.
Validation on Mars2020 Descent Imagery
In the absence of representative aerial imagery from Mars, we validate our approach using real LCAM images captured during the Mars 2020 Entry, Descent, and Landing (EDL) phase. Descent frames acquired between approximately 6 km and 960 m altitude are registered to a CTX orbital reference map (6 m/px resolution), producing scale differences comparable to those expected in future rotorcraft missions.
Real Mars 2020 descent imagery registration. Samples from the LCAM imaging sequence during the Mars 2020 EDL phase, overlaid with Geo-LoFTR correspondences. Inlier matches (green) are retained by the RANSAC-PnP solution with reprojection errors below 1 pixel, while rejected matches are shown in red.
Reconstructed Mars 2020 descent trajectory. Localization results between 6 km and 960 m altitude over a 9 km² CTX map crop of Jezero crater, comparing Geo-LoFTR, Pre-LoFTR, and SIFT. Ground truth (GT) EDL trajectory is shown for reference.