Reinforcement learning (RL) has become a central technique for improving large language models, but a new paper posted on Hugging Face argues that its benefits are distributed unevenly. According to the authors, RL produces large gains on easy problems that a model can already solve, while yielding only small improvements on hard problems. They describe this imbalance as the Matthew Effect in RL for LLMs, borrowing the economics and network-science idea of cumulative advantage — commonly summarized as "the rich get richer."
The paper suggests the obvious explanation — that hard problems simply require more compute to find a solution — is incomplete. The authors contend that modern RL methods make the problem worse by spending too much compute on easy problems, and that training should instead dynamically reallocate where compute is used.
To address this, the researchers introduce Never Give Up (NGU), described as a simple adaptive sampling method that keeps generating samples for a problem until one is correct. By leveraging asynchronous RL, NGU naturally uses fewer samples to filter out easy problems and directs more compute toward solving harder ones. The paper also examines design choices that affect NGU, including off-policy robustness, and develops a set of best practices.
On the math benchmark Deepscaler, the authors report that NGU improves performance per compute, especially on harder problems. On a recent coding task, Manufactoria, they note that standard GRPO with a per-test reward fails to fully solve problems containing a range of easy and difficult tests. NGU, by contrast, iteratively improves — solving harder and harder tests until it learns to fully solve the coding problems.
The work is accompanied by a blog post and a public GitHub repository, and the Hugging Face page also lists related recent papers on difficulty-adaptive and sampling-based RL for LLMs.