Post-training is normally understood as a task-specific process: models are refined on data tied to the capability they are meant to improve. A new paper on Hugging Face challenges that assumption by showing that post-training leaves a distributed "behavioral shadow" — tiny preference shifts between ordinary words, even in contexts unrelated to the training task.
The authors introduce Active Taskless Distillation (ATD), a method that transfers capabilities using only a single word from the teacher per prompt. ATD selects prompts where the teacher and the student's shared public ancestor model are nearly indifferent between two ordinary words, then trains the student solely on the resulting prompt-word pairs — with no target-task examples, no teacher logits, and no teacher parameters.
In the primary coding experiment with Qwen2.5-1.5B, 5,664 single-word answers produced a 5.34 percentage point gain on HumanEval+ over an exact nuisance-matched control that disrupts prompt-response alignment. The authors report that the effect also holds in scientific knowledge, commonsense reasoning, and reading comprehension, and across additional model generations, sizes, and families.
Functional analyses indicate that the learned signal is composable and that its strength tracks the teacher's update strength. According to the authors, prior work showed that preferences can be transmitted this way; to their knowledge, this is the first time a capability has been transferred through such task-unrelated text.
The paper's broader implication is that learned capabilities leak far beyond their training task: thousands of seemingly meaningless behavioral changes collectively carry a rich trace of post-training.