Enabling robots to adapt to unfamiliar environments as readily as humans remains a moonshot goal of embodied AI. According to a new research paper, no finite collection of demonstrations can cover every task and situation a robot will encounter, which makes the ability to learn from context at deployment essential for generalization.
The paper notes that such in-context learning (ICL) remains largely beyond the reach of existing robotic policies. It raises the question of whether the broad agentic capabilities of commercial vision-language models (VLMs), such as GPT-6 Astra, can allow these models to learn from demonstrations, examples, and interaction feedback, then translate that information into executable and verifiable robot behavior from a new initial state — without gradient updates or persistent changes to task-specific parameters.
To address this, the authors introduce GPT-Policy, a general-agent framework for in-context robot learning. The framework integrates three components: a context compiler that preserves task-relevant visual transitions, a VLM that proposes robot-tool actions, and a constrained controller that verifies and executes each action and reports its outcome.
The researchers evaluate reliability and limitations through task success and efficiency metrics, matched comparisons across models, and controlled context ablations. In real-robot trials, human video demonstrations improved task completion even without robot action labels, while aligned action references yielded further gains on contact-sensitive tasks.
The paper positions GPT-Policy as a step toward robot adaptation through in-context learning, offering an empirical foundation for translating the general-purpose capabilities of VLMs into physical behavior and clarifying the challenges that must be overcome for reliable deployment.