AI research agents draw on prior knowledge, public sources, and experimental feedback to generate results that are often presented as discoveries. A new paper proposes the Discovery Certification Protocol (DCP) as a way to turn such claims into executable evidence rather than relying on scores alone.
The protocol is structured around gated tests. Gate 1 validates useful improvement on a sealed evaluation. Gate 2 gives matched agents the registered starting information and observed web content while withholding the target research history; any valid method that reaches the numerical target supplies a recovery witness and triggers the Core veto.
Under DCP Core, a claim must satisfy three conditions: adequate controls, zero observed recoveries, and a finite-sample bound on recovery in one fresh registered episode. An optional Gate 3 measures the average effect of truthful feedback relative to a specified neutral policy from a shared checkpoint, with DCP Evidence adding this effect only after independent null calibration and a registered effect margin.
The authors report two controlled audits exercising the full protocol in SQLite optimization and virtual catalyst control under different models. Each produced zero recoveries in 96 episodes, with an upper bound of 0.0468. Each paired study yielded 30 truthful recoveries and zero neutral recoveries, alongside passing 60-pair null studies. Additional cases exercise Core, recovered, and audit-incomplete decisions.
A deterministic, LLM-free verifier reproduces the decisions from frozen evidence. The authors position DCP as a common evidence language for useful outcomes, alternative routes, and feedback effects across AI research: validate the gain, challenge its recovery, and measure the contribution of feedback.