Researchers have introduced UNREAL (UNifying REtrieval And Long-Context with a Single Model), a framework that uses a single frozen large language model to both select evidence and generate answers. The work addresses a gap between long-context inference and Retrieval-Augmented Generation (RAG), which typically operate at very different scales — from a single long prompt to an entire corpus.
UNREAL encodes chunks and derives retrieval queries directly from the frozen LLM's internal representations, eliminating the need for an external retrieval model. According to the paper, the approach adds fewer than 500K trainable parameters and leaves the backbone unchanged.
On a 3B-token, 21M-chunk Wikipedia index, all four dense and hybrid UNREAL backbones reportedly outperform state-of-the-art retriever-reranker systems. The best model raises recall from 49.1% to 73.2% on HotpotQA, from 31.7% to 60.1% on 2WikiMultiHopQA, and from 8.8% to 14.4% on MuSiQue.
When applied to long-context tasks, the same selection mechanism removes distractors before generation. The paper reports that this raises NoLiMa accuracy from 1.0% to 24.83% at its maximum context length of 128K tokens, and lifts LV-Eval's F1 score from 49.97% to 54.66% at 256K.
UNREAL also reduces FLOPs and time-to-first-token relative to full-context inference from roughly 32K tokens onward, with larger gains as context grows. The authors position model-internal evidence selection as a common foundation for both corpus retrieval and evidence-sparse long-context inference.