Vision-Language-Action (VLA) models have shown impressive visuomotor skills, but they often struggle with challenging and out-of-domain tasks. Recent test-time steering and scaling methods offer improvements without extensive data collection, yet they tend to produce action samples that cluster around similar behaviors, inheriting correlated failure modes. Additionally, these methods apply the same intervention at every timestep, regardless of whether the base policy is already likely to succeed.
To address these issues, researchers from Hugging Face and collaborators introduce RL^2, an adaptive inference-time steering framework that leverages Reinforcement Learning on VLA Latents. The approach trains a lightweight offline RL policy conditioned on expressive latents extracted from the VLA action expert. During inference, the flow velocity of this RL policy is composed with that of the frozen VLA, combining the behavioral priors of large-scale imitation learning with the action diversity induced by offline RL.
A key insight is that inference-time steering follows different scaling laws under success and failure states. Action diversity is most beneficial when the base VLA is likely to fail, but can unnecessarily perturb already-accurate actions when success is likely. Therefore, RL^2 activates compositional steering only when failure is predicted.
Across the SIMPLER and PolaRiS benchmarks, RL^2 improves success rates by up to +17.3% in out-of-domain settings. Ablations and scaling studies highlight the importance of latent representations and RL training. Real-world experiments confirm that these gains transfer beyond simulation, establishing RL^2 as a practical and modular steering framework for VLA deployment.