Uncertainty-Aware Open-Vocabulary Change Analysis in Remote Sensing: Characterization, Calibration, and Deployment

Abstract

Locating a change described in free text between two remote-sensing images is an open problem: most published change detection is closed-set, open-vocabulary change detectors return masks without calibrated confidence, and open-vocabulary segmentation models take a single image.

First, a method. We adapt a single-image reasoning-segmentation vision–language model (SegEarth-R1) to an image pair and a free-text query with a lightweight adapter over the frozen backbone, initialized to reproduce the unadapted model exactly. In distribution and at full scale, it roughly triples to quadruples the backbone's average precision and exceeds a parameter-free CLIP-difference baseline.

Second, a characterization. Against a chance floor and with false-discovery control, transfer to six held-out SECOND land-cover classes is class-dependent: four of six rank above chance, and two reach a certified operating point. Adaptation creates ranking transfer on the three classes where the backbone has none under either significance test, and erodes two on which the backbone's mean average precision already clears chance. A calibration gap separates ranking from certification; it persists on the one additional backbone tested and, except at one budget that larger budgets did not sustain, under few-shot supervision. On an independent dataset, single-class training yields no held-out transfer.

Third, a safe operating envelope. In a pre-registered selective-prediction study on rare per-item change, calibration-aware, abstaining tool use beats a naive tier-blind baseline but does not beat a base-rate predictor, for a segmenter and a retriever alike. Re-analyses leave the retriever's decisions statistically indistinguishable from the base rate's; on three configurations, in an exploratory analysis, its confidence orders the answers better than prevalence by under two thousandths of mean selective risk. Cost-aware re-thresholding saves at most about two hundredths of a unit per item (about three hundredths with contextual patch features), only at some cost ratios, and costs significantly more at high ones on the retrieval stream, so the base-rate comparator stays binding under asymmetric costs.

Fourth, a working uncertainty-aware change-reasoning agent. It answers where its tools are certified, hedges where they only rank, abstains where they are at chance, and withholds certification under detected distribution shift. Its honesty advantage was measured on 10,164 real tool outputs from the test pairs on which its tiers were assigned: the calibrated policy roughly halves the confident-error rate of the uncalibrated baseline (0.29 versus 0.59) at no gain in accuracy and, abstaining on every at-chance item by rule, misses no required abstention. Given the parsed intent, the decision does not depend on the language model or the prompt; intent class accuracy ranged from 0.84 to 0.96 across the four language models tested on a templated benchmark, and was lower on natural captions for the adopted model.