Most end-to-end autonomous driving systems learn by imitating human driving logs, which limits their behavior to the quality and coverage of recorded trajectories. In a new paper, Hugging Face researchers present DriveZero, a system designed to learn driving behaviors that go beyond human demonstrations.
DriveZero decomposes driving into a perception model and an action model, pretraining each in the regime best suited to it. The perception model, DriveVFM, consolidates multiple frozen vision foundation models—including DINOv3, SigLIP2, SAM, and Depth Anything V2—into a single backbone using raw images alone, with no task-specific annotations. The action model, DriveRL, is a mixed-agent closed-loop reinforcement learning framework that converts real driving logs into interactive worlds, where a privileged teacher policy is trained with PPO.
The two models are unified through distillation: DriveZero encodes multi-view images with DriveVFM, decodes trajectory proposals, and learns them from DriveRL rollouts instead of human trajectories. This includes rollouts under augmented goals that human logs never contained, enabling goal-consistent supervision that logged data cannot provide.
On the nuPlan benchmark, DriveRL with value-guided test-time action search achieves a mean score of 93.57 across the Val14, Test14-hard, and Test14-random community splits in both non-reactive and reactive modes, exceeding the Log-Replay expert on all three splits. DriveZero achieves state-of-the-art performance on NAVSIMv1, NAVSIMv2, and the closed-loop HUGSIM benchmark without any human trajectory supervision.