A new paper from Hugging Face researchers introduces Anchor-Align, a finetuning approach for vision-language-action (VLA) policies that addresses two key weaknesses of standard behavior cloning (BC): the gradual overwriting of pretrained representations and the misalignment between language and action predictions.
Standard BC finetuning on robot demonstrations progressively overwrites the representations learned during vision-language model (VLM) pretraining, which are crucial for visual and semantic generalization. Co-training on web image-text data does not solve this because it applies language and action losses to separate observations, leading to language-action misalignment that standard benchmarks fail to expose.
Anchor-Align augments BC with two objectives: Vision-Language Anchoring uses layer-wise distillation from a frozen VLM copy to prevent representation drift, and Language-Action Alignment converts each action target into a discrete motion-direction label, training language and action prediction jointly on the same robot observation.
On a physical xArm7 robot, Anchor-Align improved real-robot success rates from 28% to 54% and from 37% to 60% across two widely used VLA architectures. In simulation, it demonstrated consistent improvements on out-of-distribution (OOD) perturbations, perceptual robustness, and long-horizon control across LIBERO-PRO, LIBERO-Plus, and CALVIN benchmarks.
The results suggest that preserving pretrained representations and effective action learning are not fundamentally at odds. The project page is available at anchoralignvla.github.io.