A newly posted paper on Hugging Face, titled Your Transformer Can Hold Two Thoughts at Once: Evidence of Linear Superposition in LLMs, proposes what its authors call the Superposition Linearity Hypothesis. The claim is that when inputs drawn from two distinct text streams are combined linearly, the model produces a superposition of the individual next-token distributions rather than a single blended prediction.
According to the paper, this behaviour appears to be an intrinsic property of the Transformer architecture rather than something that emerges from training. The authors report that superposition tends to weaken as pretraining progresses, suggesting that scale and training objectives erode the effect over time.
The work also offers a practical countermeasure: lightweight fine-tuning can substantially restore the linear behaviour, narrowing the gap between the model's predicted next-token distribution and the average of the individual distributions for each stream.
Building on that finding, the authors introduce a guided decoding procedure that disentangles the superposed outputs. This allows two coherent continuations to be generated simultaneously from a single forward pass, a result the paper frames as evidence that an LLM can effectively "hold two thoughts at once."
The paper is accompanied on Hugging Face by automated recommendations from the platform's Librarian Bot, which lists several related 2026 works on hidden-state interpretation, attention sinks, and transformer efficiency. No benchmark figures, model names, or quantitative results were included in the material provided.