Recent Vision Foundation Models (VFMs) predict depth, camera pose, and pointmaps in a single forward pass, achieving strong generalization without per-scene optimization. However, they do not enforce explicit multi-view geometric consistency during pretraining, as doing so (e.g., via bundle adjustment) is computationally expensive. This can lead to inconsistencies in predictions.
To address this, researchers from CMLab Korea propose Self-Geometry, a plug-and-play test-time adaptation (TTA) pipeline that directly imposes explicit multi-view geometric constraints using 2D pixel correspondences as pseudo ground-truth. Unlike prior TTA methods that rely on implicit self-consistency from model outputs—which offer limited gains where the pretrained VFM is most inaccurate—Self-Geometry supervises with explicit geometry.
The method comprises three key components: Geometric Disentanglement Optimization, which combines Multi-View Consistency and Epipolar Consistency losses with Gradient Disentanglement to prevent gradient conflicts; Frame Angular-Neighbor, a view sampler based on SO(3) geodesic distances; and Lightweight TTA, which adapts VFMs via LoRA. The adaptation completes in a few minutes on a single GPU.
Experiments show consistent improvements in both pose and geometry estimation across six VFMs (VGGT, π³, DA3-Giant/Large/Base/Small) and four benchmarks (7Scenes, ETH3D, ScanNet++, HiRoom). The project page offers interactive side-by-side comparisons.