Hugging Face has unveiled τ^τ-Bench (pronounced hyper-tau-bench), a new benchmark designed to measure how well coding agents can construct real-world customer-service agents. Unlike traditional benchmarks that focus on isolated tasks, τ^τ-Bench simulates the full scope of a client engagement, providing the agent with the business records a company actually keeps, a client with specific requirements, a production API, an existing codebase, and constraints on serving cost and models.
The benchmark spans 53 tasks across four domains, and the evaluation is rigorous: the developed agent is deployed against held-out simulated users to score its performance. The results reveal a significant gap between current AI capabilities and expert-level performance. The strongest configuration tested, Claude Opus 5 under Claude Code, passed only 23.9% of evaluation simulations, while an expert-authored reference solution achieved 82.2%.
The authors identify recurring failure patterns that mirror challenges faced by human agent developers. Models often issue shallow queries instead of deeply understanding the provided records, communicate insufficiently with the client, and show limited experimentation with agent architecture and serving budget, often shipping the first design that merely runs.
τ^τ-Bench aims to turn the complex work of cooperative agent building into a measurable target for coding agents, pushing the field toward more reliable and realistic agent development. The paper is available on Hugging Face and has attracted recommendations for related research in long-context instruction following, stateful business workflows, and backend code generation.