Adversarial games have long driven progress in AI, from heuristic search to reinforcement learning, yet learning and adapting strategies from limited samples remains difficult. A new paper on Hugging Face proposes turning AI agents themselves into learning engines that revise executable game policies while keeping the underlying model weights fixed.
The work formalizes this idea as Adversarial Heuristic Learning (AHL), building on prior discussions of heuristic learning in which a learner converts experience and feedback into directly improved executable structures such as programs. In AHL, the agent interprets rules, selects opponents, analyzes replays, and generates and maintains the game-playing agent that enters each match.
To test the paradigm, the authors introduce AAArena, a benchmark drawn from competition archives that includes 12 authentic adversarial games and 1,920 human-written program submissions. Each game provides a common evaluation budget of 128 units for small matches and 16 evaluations against the full opponent pool, with agents deciding when to learn and when to submit.
Across seven AI models and their runtime configurations, Opus5.5 with Claude Code earned six gold medals, while no evaluated configuration topped the remaining six human leaderboards. Performance was generally weaker in games with more complex rule specifications.
Further experiments indicate that opponent selection and dense feedback support policy improvement, and that agents learn from both on-policy replays of their own matches and off-policy replays of other players' matches. The authors highlight HL's potential in adversarial games while identifying persistent challenges in game understanding, strategy implementation, and long-horizon policy development.