Foundation vision-language models (VLMs) carry broad knowledge about the world, but turning that knowledge into reliable robot control remains difficult. A new paper proposes Show-Harness, described as an "Embodied Harness" that lets VLMs operate robots through a compact semantic interface linking intent to action.
The design exposes discrete semantic action units that VLMs can reason over naturally, while embodiment-specific interpreters deterministically translate those units into local robot actions. According to the authors, this keeps the VLM directly responsible for fine-grained physical decisions rather than hiding them behind a separate controller.
Through the same interface, the work demonstrates two deployment paths: directly unlocking closed-source frontier VLMs for zero-shot robot control, and adapting small open-source VLMs for low-cost deployment with only a few GPU-hours of fine-tuning.
The team also introduces GUMI (GUI Manipulation Interface), which extends the same semantic action space to GUI-based demonstration collection. This allows humans and agents to "play" robots across different embodiments without specialized teleoperation hardware.
Extensive experiments reportedly show that Show-Harness-equipped VLM agents generalize robustly across tasks, embodiments, and environments, outperforming representative agentic and vision-language-action (VLA) paradigms. The authors conclude that the right interface can unlock substantial embodied capability from foundation VLMs without additional model capacity or costly embodiment-specific pretraining.