A new research paper from Hugging Face introduces WHALE (Weight-Harness Alternating LEarning), a simple yet effective recipe for jointly optimizing AI agents. The core insight is that an agent's performance depends on both its model parameters and the executable harness code that manages context, tools, and control flow. Optimizing either in isolation can leave the system bottlenecked by the frozen component.
WHALE alternates between two phases: updating the model under the current harness, and then searching for a better harness under the updated model. The first phase uses online rejection-sampling fine-tuning, while the second leverages a method called Meta-Harness. A key design choice is when to switch between phases; WHALE uses either fixed phase durations or an adaptive patience rule based on training signals to avoid over-optimizing against a changing counterpart.
In experiments with Qwen3.5-2B and 4B agents across three domains—search question answering, mathematical reasoning, and chess puzzles—WHALE outperformed weight-only, harness-only, and Fast-Slow Training approaches by 4.15 to 24.38 percentage points in best mean@8 accuracy. The results highlight that bottlenecks are domain-dependent: harness search alone can match peak weight-only accuracy in SearchQA with far fewer rollouts, but in mathematics, it only becomes effective after a weight update.
The authors also found that small interleaved updates outperform stagewise weight-then-harness optimization, achieving higher accuracy with fewer rollouts. An adaptive version of WHALE removes the need for hand-tuned schedule hyperparameters by using a per-phase patience rule, matching or beating the best fixed schedule. The code is available on GitHub.