A newly published research paper on Hugging Face investigates a fundamental instability in test-time training (TTT), a technique that allows a model to store information in its weights during inference. The study, titled Self-Generated Feedback Destabilizes Test-Time Training: A Causal Decomposition of Long-Horizon Adaptation, examines what happens when a model learns from its own output—a feedback loop where each update also changes the model that generates the next training example.
Across 128K-token streams, the researchers found that retaining generated-text updates worsens prediction on independent human-written text with three TTT-E2E model configurations labeled 125M, 760M, and 3B. The same failure occurs when Adam updates Qwen3-4B's existing weights. Importantly, the same update mechanisms can improve performance on real text, indicating that writing itself is not the cause of the failure.
Three matched comparisons trace the causal pathway. Fixed Generation removes over 98% of the damage at 125M and 760M by using a frozen model to generate training chunks. Recorded Replay separates the loss caused by reading degraded text from the additional loss stored by updating on it. A paired one-update comparison then reveals a local conflict: an update predicts its source better but new real text worse. This cost grows after Closed Loop adaptation, with a few trajectories accounting for most large failures.
Finally, Settlement evaluates the candidate state on independent real text before commitment. It leaves mean endpoint gaps of 0.07 and -0.02 nats at 125M and 760M while retaining real-text adaptation. The authors conclude that these results motivate checking prediction on independent evidence before retaining an update.
The paper's code is available at https://github.com/lingjivoo/ttt-ouroboros, and the authors invite discussion on how continuously adapting models should decide which updates to keep.