Moonstone: A Multimodal Foundation Model & Benchmark for Lunar Remote Sensing

Overview

Half a century of orbital missions have imaged the Moon in optical, spectral, thermal, radar, gravity, and elemental data, yet those archives stay fragmented and no benchmark exists for machine learning on them. Moonstone brings them together as one 28-channel global dataset, trains MG-MAE (a modality-grouped masked autoencoder that reads across instruments and handles missing coverage), and evaluates it on six downstream tasks. Pretrained MG-MAE features beat from-scratch, ImageNet, and vanilla-MAE baselines on every task.

Dataset

One place on the Moon, seven ways of seeing it

Every instrument is reprojected onto the same 128 pixels-per-degree grid (46,080 × 23,040, ~237 m/pixel) and tiled into 256 × 256 patches, so a single lunar coordinate carries 28 co-registered measurements. Pick a modality to see the same terrain through a different instrument.

Every column is one modality group, split into its individual channels. A missing instrument (radar covers only 16% of the surface) is handled as signal, not error.

Model · MG-MAE

Reconstruct the Moon from a quarter of the pixels

MG-MAE groups the 28 channels into seven modalities, tokenizes each with its own convolutional stem, and encodes them with a shared Vision Transformer. Coverage-adaptive masking hides most of every patch, heavily for common modalities and gently for rare ones, and a per-group decoder fills the gaps. Pick a modality below: given only 25% of the tiles, the model recovers coherent structure, cleanly for a smooth gravity field, and impressively for cratered terrain.

Gravity Surface Thermal

Masked input · 25% visibleMG-MAE reconstructionGround truth

Grouped tokenizers let the encoder learn structure within a modality, such as the shape of a reflectance spectrum or the falloff of a gravity anomaly, while cross-modal attention lets one instrument inform another. A spectral-continuity term keeps the eight hyperspectral bands physically smooth.

Benchmark

Six downstream tasks

Pretrained features are evaluated against a from-scratch encoder, frozen ImageNet features, and a vanilla per-channel MAE, via linear probing and fine-tuning. MG-MAE wins on every task, with the largest gains on geology (+16.2% accuracy) and craters (+14.7 mIoU).

Open release

Citation

@article{prasad2026moonstone,
  title         = {Moonstone: A Multimodal Foundation Model and Benchmark
                   for Lunar Remote Sensing},
  author        = {Prasad, Ayush and Mazumder, Swarnalee},
  journal       = {arXiv preprint arXiv:2607.03644},
  year          = {2026}
}