Researchers from Hugging Face have unveiled a new benchmark and harness designed to probe how vision-language model (VLM) agents deceive in embodied social interactions. The work, titled "Lies We Can See: Joint Verbal and Non-Verbal Deception by VLM Agents in Embodied Social Interactions," introduces MineAmongUs, a 3D multimodal sandbox inspired by the popular social deduction game Among Us.
In MineAmongUs, imposter agents must deceive crewmates using both verbal communication and physical actions. The researchers argue that existing testbeds for studying AI deception are text-only and rely on fixed agent configurations, missing the non-verbal sensorimotor channels that are central to deception taxonomies. This makes it unclear whether observed deceptive behavior stems from the underlying model or the surrounding harness.
To address this, the team also proposes ARIA, a configurable VLM-agent harness that exposes five cognitive-component ablation axes, allowing researchers to isolate which parts of an agent's architecture contribute to deception. Additionally, they developed an annotation scheme at the atom and arc level, grounded in deception taxonomies and operationalized at scale by an LLM-as-a-Judge, which achieves near-human agreement on atom labeling.
Empirical results show that VLM agents pursue imposter wins through a combination of verbal and non-verbal deception. Notably, non-verbal channels emerged as the more decisive winning contributors across both harness ablation and cross-VLM evaluations. The authors pose the question, "Can an agent lie with its body, not just its words?" and suggest that as digital agents gain new action spaces, deception follows—making embodied deception a critical area for alignment research.