Large Vision-Language Models (VLMs) have advanced embodied navigation, but their direct deployment often forces them into unnatural action spaces that misalign with their 2D pre-training priors. Rigid reasoning schedules and inefficient memory management further compound the issue. To address these challenges, researchers from Hugging Face propose TAMP-Nav, a unified framework for efficient embodied navigation.
The framework introduces a Pixel-to-3D Action Formulation (Point) that reformulates navigation into 2D visual prompting. The VLM selects 2D pixels, which are then projected into 3D coordinates for a low-level SLAM controller, aligning embodied execution with the VLM's inherent 2D visual capabilities.
Additionally, TAMP-Nav integrates Selective Reasoning and Anchor-Trajectory Memory (Think and Memorize). This mechanism dynamically triggers Chain-of-Thought reasoning only at critical decision points and retains high-fidelity memory at key nodes, compressing redundant trajectories into lightweight Space-Time Indicators. This preserves critical historical information and enhances spatio-temporal perception.
Finally, a Two-Level Alignment Paradigm (Align) uses Group Relative Policy Optimization (GRPO) to combine global outcome rewards with fine-grained process rewards. This dense supervision tightly aligns the agent's cognitive planning with physical environmental feedback, enabling adaptive reasoning.
Experiments show that TAMP-Nav achieves state-of-the-art performance with high runtime and sample efficiency. It reaches 66.2% success rate on R2R-CE and 65.7% on RxR-CE, while reducing Chain-of-Thought calls by 73.7%, and requires only 90k training trajectories. The policy transfers zero-shot to real-world indoor and outdoor environments without real-robot fine-tuning.