Hugging Face researchers have unveiled Skill-α, a reinforcement learning (RL) approach for progressively generating agent skills. Unlike existing methods that rely on heuristics or pipeline-style consolidation—which require special design for different evidence sources—Skill-α offers a unified, learning-based framework that works across heterogeneous sources.
The core challenge addressed is the lack of natural supervision signals for skill generation. Skills are not inherently labeled as relevant or correct; their value is only measurable by whether they improve the agent's performance on downstream tasks. Skill-α tackles this by formulating skill generation as a sequential editing process, breaking down skill construction into individually evaluable edits.
A key innovation is the rollback reward, which evaluates each edit by comparing downstream execution results under the original versus the edited skill on an anchored query. This allows the model to assess the contribution of each edit in a targeted manner.
In experiments, Skill-α generated more effective skills than heuristic and pipeline baselines in both document-to-skill and experience-to-skill settings. Using a GPT-4o worker, Skill-α improved average downstream success rates by 3.3 points on CL-Bench and 6.7 points on tau2-bench over the strongest baseline. Ablations confirmed the importance of the rollback reward and progressive generation.
The code is available on GitHub at https://github.com/ejhshen/skill-alpha.