Language model agents increasingly depend on human-designed workflows to tackle complex tasks, and good workflows can substantially boost performance by supplying useful intermediate steps and structure. But a new Hugging Face paper asks what happens when the workflow is worse than the model itself.
The authors introduce Box²-Bench, a benchmark designed to isolate how models regulate their reliance on external guidance. The setup holds the model and task fixed while varying workflow reliability, comparing the same model with no workflow, a helpful workflow, and a misleading workflow. This separates two abilities that are usually mixed together: using good guidance and resisting bad guidance.
Across models and tasks, the paper reports a consistent pattern: reliable workflows often improve performance, but misleading workflows can still substantially hurt it. Even capable models remain sensitive to unreliable external guidance, the authors write.
The paper then asks whether selective reliance can be learned. The researchers trained two open-weight models using only bad workflows, reserving good workflows for evaluation. They explored two complementary strategies: counterfactual supervised fine-tuning, which improved robustness to misleading guidance, and outcome-based reinforcement learning, which shifted behavior toward greater use of helpful workflows. Notably, the models never saw good workflows during training, suggesting training can shape a more general strategy for deciding how much to trust external guidance.
The behavior extended beyond workflows. Models trained in this setting also improved peer correction in multi-agent reasoning and showed greater robustness to corrupted memory, according to the paper.
The authors frame selective reliance on fallible external information as a dimension of agent reliability not captured by task performance alone. As agents increasingly operate with workflows, memories, tools, and other agents, they argue that knowing when to listen may become an important measure of reliability.