Researchers and developers building large language models and other AI systems often struggle to locate relevant evaluations, find the underlying benchmark datasets and code, and understand the settings behind reported scores. A new paper introduces Benchmark Radar, described as a living database and search engine for the retrieval and discovery of AI benchmarks.
The system spans LLM evaluation as well as agentic and tool-use benchmarks, coding, reasoning, safety, and domain-specific evaluations. It combines daily discovery of benchmark papers, repositories, datasets, and releases with a searchable benchmark catalog, mentions in model cards and technical reports, and score histories. Source identities and citations are retained so readers can inspect candidate benchmarks and their evaluation evidence directly.
According to the paper, daily discovery draws on 37 sources: 13 direct connectors and 24 first-party research and engineering feeds. The catalog contains 1,283 source records drawn from 4 benchmark catalogs, along with 12,916 numeric observations on 790 records.
The authors describe their collection and retrieval methods, audit the full catalog, and examine benchmark saturation, adoption trends, and the limits of score comparisons. A worked example walks through a complete prior-art search, showing how to query the catalog and inspect benchmark evidence when designing a new evaluation.
The release includes a web dashboard with a benchmark leaderboard, a Pareto frontier view of score against measured use, saturation and trend views, daily feeds, downloadable evidence, a command-line interface for offline queries, and reproducible analysis. The authors note the CLI can help produce LaTeX versions of benchmark-related work in minutes from the 12k+ benchmark, eval, and dataset records.