Streaming 3D reconstruction from extremely long videos is a challenging task that requires estimating camera motion and scene geometry online under bounded memory and computation. Early models achieved causal, bounded-cost inference using finite context buffers or compact recurrent states, but their estimates often deteriorated as sequences grew. Recent methods improved long-horizon stability by coupling short-range context with persistent or multi-level long-range memory.
Hugging Face researchers propose a different approach: keeping the learned temporal state strictly local and formulating predictions whose targets remain independent of sequence length. Their model, ABot-Recon, caches key-value (KV) features from only the preceding 11 frames, predicting a point map in the current camera coordinate system along with an adjacent-frame relative pose. These predictions are equivariant under changes of reference frame, and global poses and geometry are recovered through sequential composition.
To reduce accumulated drift, ABot-Recon employs a lightweight temporal refiner that improves relative rotations using recent visual and motion context, and a composition-aware pose loss that supervises multi-step pose composition. Extensive evaluations on challenging long-sequence benchmarks demonstrate superior long-horizon performance. On Oxford Spires, ABot-Recon achieves an ATE of 4.35 meters and an RPE-R of 0.12 degrees, reducing both errors by approximately 40% relative to the best prior results.
ABot-Recon can turn a single continuous video into a globally consistent 3D reconstruction in real time, using only a fixed 12-frame local context. It scales to sequences of over 10,000 frames at 24.45 FPS with only 6.71 GB of GPU memory, enabling efficient real-time reconstruction of large-scale environments such as buildings, city streets, and campus flyovers.