Vision-language-action (VLA) models have pushed robotic manipulation forward, but they still struggle to generalize zero-shot to unfamiliar tasks and environments. Because they depend on specialized training, they also cannot directly inherit the rapid progress of general-purpose vision-language models (VLMs).
Existing agentic robotics approaches try to close that gap by pairing VLMs with high-level reasoning or coding agents for control. According to the authors, these systems typically rely on extensive external models and tools, which adds complexity and cost.
The paper asks a pointed question: can a general-purpose VLM operate a robot more like a human teleoperator — reasoning from observations, issuing actions, and continuously adapting to execution feedback — without learned action experts, coding agents, or grounding tools such as SAM3?
Their answer is MotorMind, a robot manipulation harness that links VLM-proposed mid-level actions to deterministic robot control and feedback, supported by asynchronous monitoring and background memory updates.
Without task-specific policy training, coding agents, or additional grounding tools, MotorMind reports 66.7% success on the base LIBERO-PRO suites and 53.8% under perturbations, versus at most 13.3% and 19.2% for the prior zero-shot methods the authors evaluated. The same interface reaches 95% average success on a real xArm6 robot across direct manipulation and human-perturbation settings.
Swapping in a stronger VLM backbone improves results further, and the authors note that remaining failures — mainly tied to visual grounding, embodied reasoning, and action knowledge — shrink as VLM capability grows. The project page is available at motor-mind.github.io.