Researchers have released DataFlex-RL, an evaluation platform for comparing data policies in reinforcement learning with verifiable rewards (RLVR). Such policies govern which rollouts enter training, how heavily they are weighted, and which domains contribute to each batch. The platform holds the underlying recipe constant, using a common GRPO setup so that differences can be attributed to the data policy itself.
The main experiment compared 13 configurations across 12 matched seeds using Qwen2.5-7B-Base and 12 mathematics, logic, and science benchmarks. Uniform GRPO raised the domain-balanced average accuracy by 7.76 percentage points over the untrained checkpoint. However, none of the eight rollout-selection or reweighting methods achieved a paired 95% confidence interval excluding zero relative to uniform sampling, and none of the three adaptive mixtures outperformed a fixed equal mixture at the same level of precision.
A corrected 12-seed extension on Llama-3.1-8B-Base placed the additional methods on the same score scale as the original controls, but still did not reveal a consistent winner by observed mean performance.
The paper also examines evaluation sensitivity. Rescoring nine Qwen2.5-7B-Instruct runs with a math-heavy six-benchmark summary — five mathematics benchmarks plus GPQA-Diamond, with no logic benchmark — produced rankings negatively correlated with the domain-balanced 12-benchmark summary, at a correlation coefficient of -0.33. Summaries retaining all 12 benchmarks largely agreed with each other.
The authors conclude that, in the controlled settings studied, changing the data policy measurably alters the training process but does not yield a reproducible improvement over uniform training, underscoring the need for balanced and reproducible evaluation in RLVR.