Reinforcement learning (RL) post-training of Vision-Language-Action (VLA) models has shown promise for robotic manipulation, but a fundamental bottleneck persists: value estimation under partial observability. Existing critic models typically rely on single-frame observations or single-frame VLM backbone latents, which fail to capture the temporal dynamics essential for accurate decision-making in robot control.
To address this, researchers at Hugging Face introduce the World Critic Model (WCM), built on a lightweight LeJEPA architecture. WCM jointly predicts future latent states and estimates values, explicitly training the critic's representation to capture temporal dynamics rather than merely regressing scalar returns. This predictive state representation unifies world modeling with critic learning, enabling the model to understand how the environment evolves.
WCM integrates seamlessly into both on-policy and off-policy training pipelines and is compatible with state-of-the-art VLA backbones including Pi0, Pi0.5, and OpenVLA-OFT. Extensive experiments on 149 tasks across four benchmarks demonstrate that WCM consistently achieves state-of-the-art performance in both in-distribution and out-of-distribution settings, with particularly strong generalization gains.
The team further validated WCM on seven real-world manipulation tasks using OpenVLA-OFT and Pi0.5 with off-policy RL, confirming stable deployment across diverse settings. The paper and project page are available for deeper exploration.