Researchers have introduced HyperBrowseComp, a multilingual and multimodal benchmark designed to stress-test the web-browsing abilities of large language models. According to the paper, the benchmark contains 423 manually authored and human-validated questions spanning 13 languages, each written by native or highly proficient speakers.
The questions are deliberately hard. Every item targets a concise, publicly verifiable answer that can only be found by locating obscure evidence, following multi-step clue chains, or inspecting heterogeneous sources such as videos, scanned documents, images, or maps.
To keep the benchmark focused on genuine search rather than memorized knowledge, the authors filtered out easier questions by testing them with models that had no internet access, reducing the chance that a question could be answered from parametric knowledge alone.
Evaluation was carried out across several models using provider-native search and a shared external retrieval harness under a common agent protocol. The team also ran a human evaluation on a sample of the questions to contextualize model performance and effort.
The authors describe HyperBrowseComp as a challenging testbed for persistent information seeking across languages and evidence modalities, with difficulty stemming from discovering and connecting evidence on the open web.