Researchers have introduced PanoVLN, a method for vision-and-language navigation (VLN) that uses panoramic observations instead of standard perspective images. The premise, as described in the paper, is that a fuller view of the surroundings should support better navigation decisions — for instance, a panorama can expose a passage lying outside a perspective camera's field of view, letting a model identify the intended route without extra exploration.
However, the authors report that simply swapping perspective images for panoramas produces only limited improvements. Their diagnosis points to three areas that need adjustment: action prediction, training supervision, and visual representation.
To address longer-horizon planning enabled by wider visibility, PanoVLN predicts longer action sequences, allowing larger turns and subsequent movement from a single panorama. A confidence-guided execution (CGE) strategy dynamically decides how many predicted actions to carry out before replanning.
Because wider visibility also introduces more complex route choices, the team built training routes with frequent branching points and clear instructions to provide targeted supervision for route selection. For spatial understanding across viewing directions, PanoVLN combines semantic and geometric features from RGB panoramas to capture scene content and spatial layout without adding visual tokens.
With a 4B backbone and RGB-only input, PanoVLN is reported to surpass the previous state of the art by 11.9% and 8.7% in success rate on R2R-CE and RxR-CE Val-Unseen. Real-world experiments on a quadruped are said to show faster navigation with fewer pauses than prior VLN methods.