Action tokenizers are a critical component of autoregressive vision-language-action (VLA) models, shaping both the training targets for policies and the executable commands recovered from predicted tokens. Their fidelity is typically assessed with pointwise reconstruction metrics such as mean squared error (MSE), but the authors argue that small individual errors do not fully capture whether action adjustments across demonstrations are preserved.
After compression, similar actions may still cluster around a representative motion, while context-specific adjustments can be diminished, distorted, or even reversed. To address this, the paper introduces physical rank consistency (PRC), a metric that measures how well tokenization preserves local physical distance rankings after reconstruction. Evaluating decoded actions provides a common reference across token vocabularies and decoder architectures, complementing pointwise accuracy with a measure of relational fidelity.
The authors also present ActionPiece, a method that preserves physical action relationships through joint supervision of representation learning and quantization. Physical rank preservation supervises near-far ordering in encoder and quantized feature distances, while quantization regularization applies the same ordering to codeword assignment distributions. Both objectives augment reconstruction, producing discrete action tokens for standard autoregressive policy learning and execution through a frozen decoder.
Under the same Qwen3-VL-4B policy training setup, ActionPiece achieves 94.8% on LIBERO and 68.8% on unseen LIBERO-Plus, with additional evaluations reaching 71.9% on SimplerEnv and 51.5% across VLA-Arena L0-L2. Component ablations show that the two objectives jointly improve PRC and policy success, demonstrating the value of physical relationship supervision for action tokenization.