Vision-language model (VLM) agents can drive robots using visual feedback and action primitives, but repeated model calls and redundant observations create heavy token overhead. A new paper on Hugging Face proposes PyRUA-Lean, an interactive code-execution framework designed to reduce that cost.
The framework combines feedback-driven primitive composition with selective observation. Instead of repeatedly querying the model, the agent composes classical robot primitives and learned vision-language-action (VLA) policies into Python cells that handle conditional checks and local retries. Only explicitly requested images and state feedback are returned for replanning.
The researchers evaluated PyRUA-Lean against a tool-calling baseline using the same GPT-6 Astra planner and underlying robot primitives. Tests covered 700 simulated task instances from LIBERO-PRO, RoboTwin 2.0, and RoboCasa365.
Under equal LLM-call budgets, PyRUA-Lean raised overall success from 63.1% to 71.7%. On tasks solved by both agents, it used 49% fewer LLM calls and 65% fewer input tokens.