Hugging Face has published a technical report detailing a new training approach for AI-powered digital avatar agents, particularly those used in live-streaming e-commerce. The paper, titled "Training Agents to Evolve with Their Harness," addresses the challenge of balancing low latency with the ability to adapt to frequently updated system configurations, known as harnesses.
The proposed method, Harness-Aware Training (HAT), is designed for compact models that must meet strict latency requirements. Unlike larger models that can adapt to new harnesses without retraining but are too slow, compact models often overfit to a fixed harness configuration. HAT introduces Harness-State Augmentation (HSA), which applies transformations to skill identifiers, tool schemas, prompt structures, and hook functions during training, simulating a variety of harness states.
The training process consists of three stages: HSA-SFT (supervised fine-tuning with augmented harness states), General On-Policy Distillation to restore generalization, and HSA-RL (reinforcement learning) to improve robustness. The authors report that HAT achieves a score of 94.8 on Live-Stream QA, outperforming both the base model (80.3) and the strongest general LLM (93.0). On Harness-Variant QA, it reaches 94.6, compared to the base model's 75.4.
Importantly, HAT avoids the regression seen with Fixed-Harness SFT, which lowers IFEval by 7.7 points from the base model; HAT instead reaches 83.5. The optimized system, deployed on a single NVIDIA H20 GPU, delivers P50 and P95 latencies of 3.4 seconds and 8.1 seconds respectively. The system has been deployed in Taobao Live's digital-avatar service, yielding positive online A/B test results for GMV and item-page views.