Long-horizon agentic tasks require more than chaining short tasks; they demand continuous exploration, learning, and policy adaptation over thousands of steps in dynamic environments. To address this, researchers at Hugging Face have introduced E-Commerce Bench, the first open-source benchmark that integrates multi-round counterpart negotiation and dynamic events into a year-long business operation.
In the simulation, an LLM agent manages multiple online stores over 365 days, researching markets, negotiating with suppliers, optimizing sales strategies, fulfilling orders, handling returns, and managing cash flow to maximize year-end total assets. The environment is built on real e-commerce data, with a calendar of promotions, natural disasters, and supply-chain shocks that continually reshape demand. Both sides of the market are deterministic for reproducibility: customer behavior follows a fixed demand model, and a negotiation kernel determines supplier pricing and concessions, with an LLM only verbalizing them.
The benchmark evaluates 18 frontier models across seven dimensions, including year-end assets. Notably, no single model dominates. GPT-5.6 Sol earned the most, growing a 100,000 opening stake into 1,431,425, but ranked 16th of 18 on fraud avoidance and trailed Fable5 in operational efficiency. Among open-weight models, Qwen3.8-Max-Preview led with 416,252, 38% above GLM 5.2 (high), and demonstrated the strongest learning over the horizon, progressively bargaining down prices across repeated orders.
The code is available on GitHub, and the benchmark is presented as part of the Qwen3.8-Max Blog. This work provides a realistic and reproducible environment for advancing research on long-horizon autonomous business operations.