Memory has become a foundational architectural dimension in large language models (LLMs), evolving from an implicit byproduct of computation into a spectrum of explicit, controllable mechanisms. A new survey, hosted on Hugging Face and authored by researchers from Tsinghua University, the National University of Singapore, and Bosch AI, aims to bring order to this rapidly evolving but fragmented research landscape.
The paper, titled "Memory for Large Language Models," proposes a systematic, architecture-centric taxonomy that characterizes memory along three orthogonal axes: representation (implicit versus explicit), update dynamics (offline versus online), and persistence (short-term versus long-term). This framework helps clarify the conceptual boundaries between computation-coupled memory, such as attention KV caches and recurrent hidden states, and independently addressable memory modules like those found in models such as Titans, TTT, and Engram.
The authors formalize the granular mechanisms governing memory writing, routing, state transitions, and consolidation. They also critically analyze hybrid memory architectures, system-level efficiency trade-offs, and multi-dimensional evaluation methodologies. By consolidating scattered advancements into a cohesive framework, the survey charts the trajectory of memory-centric LLM design and provides a principled foundation for future innovations in scalable and adaptive language modeling.
The survey explicitly distinguishes architectural-level memory from external agent-based memory systems, a distinction that is often blurred in the literature. It is recommended for researchers focusing on long-context scaling, hybrid architectures, and algorithm-hardware co-design.