ENCP: Episode-Normalized Conformal Prediction for Vision-and-Language Navigation
1Monash University, Indonesia 2SEACrowd
01
Abstract
Uncertainty estimation for Vision-Language-Navigation (VLN) models is a critical task since it can help identify ambiguous and unreliable predictions, enabling agents to make safer navigation decisions. As one of the most advanced uncertainty estimation frameworks, conformal prediction (CP) offers a promising approach for uncertainty estimation in VLN. However, as CP is not specifically designed for sequential prediction problems like VLN, it often fails to provide the coverage guarantees it promises.
To address this issue, we propose Episode-Normalized Conformal Prediction (ENCP), which repairs the coverage guarantee of standard CP by rescaling any base nonconformity score using the policy's residual confidence and calibrating the maximum score over entire episodes rather than individual steps. This episode-level construction provides coverage across all steps of a variable-length trajectory while allowing arbitrary dependence within each episode. Across 4 VLN models and 3 nonconformity scores, ENCP achieves the target coverage on the R2R and REVERIE datasets, while other baselines fail to do so. These results demonstrate that ENCP can provide actionable, model-agnostic uncertainty estimates for determining when a VLN agent should defer to a more capable predictor, including human assistance.
02
From uncertainty to intervention
ENCP leaves the navigation policy fixed. Score normalization adjusts each action score to the current policy confidence; episode calibration then replaces dependent steps with one maximum per episode. The parameter-free variant uses recorded policy probabilities, whereas the learning-based variant fits a weight before conformal calibration.
