Hugging Face has introduced GPT-Red, an automated red-teaming agent designed to discover novel prompt injection attacks against frontier large language models. The agent is trained using a scalable self-play algorithm, where it attacks a diverse population of simultaneously-trained defender agents, aiming to evaluate and improve the robustness of production systems.
The primary application of GPT-Red is the adversarial training of GPT-5.6, which the company describes as its most robust model to prompt injections to date. The training process leverages compute on the same scale as some of the largest RL post-training runs, making it the single-largest LLM safety training run ever documented.
According to the paper, GPT-Red excels at red-teaming: it reliably breaks past models up to GPT-5.5, finds more successful attacks than human red-teamers, and generalizes to held-out environments, defender models, and harnesses. The authors anticipate a self-improvement flywheel: as each new GPT model becomes more robust, it provides better learning signals for even stronger red-teamer agents.