Black-box optimization (BBO) is a recurring challenge across science and engineering, where each evaluation of an objective function is costly and budgets are limited. A newly posted paper on Hugging Face introduces AgenticBBO-Bench, a benchmark designed to bring consistency to how large language model (LLM) agents are tested on these problems.
According to the paper, prior studies of agentic BBO relied on different task domains and system configurations, making results hard to compare and individual design choices difficult to isolate. AgenticBBO-Bench addresses this by spanning five domains — synthetic functions, hyperparameter optimization, database tuning, chip design, and molecular design — under a single finite-budget evaluation protocol.
In the reported experiments, agentic BBO achieved higher family-averaged scores than direct LLM-based methods across all five domains, and outperformed the best numerical optimizers in four of the five.
The authors also examined three factors shaping agent performance: optimization tools, task information and prior knowledge, and the role of the LLM during search. Their findings indicate that adding numerical tools does not consistently improve results, task semantics are broadly useful while more specific priors are less reliable, and numerical optimizers can effectively absorb gains from search trajectories established by the agent.
The paper additionally introduces a five-task frontier challenge within the benchmark and evaluates seven LLMs under the Codex agent harness, reporting that GPT-6 Astra and DeepSeek-V4.1-Flash sit on the Pareto frontier of performance and cost among the evaluated models. Code is available at github.com/lamda-bbo/agentic-bbo.