Hugging Face researchers have released VibeLifeBench, a new benchmark designed to test AI agents on long-horizon, proactive tasks in a simulated living world. Unlike traditional benchmarks that use short, self-contained requests in static environments, VibeLifeBench simulates real-life scenarios where tasks span weeks, the world changes silently, and many constraints are never explicitly stated.
The benchmark comprises 200 tasks across ten everyday-life domains, each with a scripted multi-week timeline (median 29 days) over 22 mock services and 288 tools. A key feature is the inclusion of 1,483 silent mutations—changes like a canceled seat or a delayed flight that occur without any notification, requiring the agent to re-inspect the world proactively to discover them.
Evaluation of seven frontier models showed uniformly poor performance. The best model, Claude Opus 5, achieved an average score of 32.5 (avg@3), and all models exhibited a 10–15 point decay from the start to the end of a timeline. Notably, on a flagship 20-day Japan trip task, no model refused a phishing email, highlighting a critical safety gap.
The researchers emphasize that fluent single-turn tool use does not transfer to maintaining a coherent plan over a month. They plan to open-source all tasks, environments, and the evaluation framework to facilitate further research.