Hugging Face researchers have unveiled Metis, the first prototype of a 'memory foundation model,' a new paradigm that embeds native memory capabilities directly into the architecture of large language models. The work, detailed in a recent paper, challenges the conventional approach of relying on external memory modules for AI agents.
Traditional agent memory systems depend on separate retrieval, storage, and prompt-construction pipelines. Metis instead formalizes native memory through two key components: a persistent, dynamically evolving memory state within the model backbone, and learned memory procedures that autonomously decide what to store and how to use it.
The architecture introduces Metis Blocks, inserted into Transformer layers, combining a Local Memory Block for maintaining the memory state and a Hyper Memory Block for learning storage and utilization. Historical information is compressed into a compact, fixed-size session state, accessed via memory attention, eliminating the need to replay full histories.
Metis operates with gradient-free memory updates, requiring only forward computation. All model weights remain frozen during inference, while the memory state evolves autonomously. The team released implementations and Qwen3.5-based checkpoints at 4B, 9B, and 27B scales, aiming to spur further research into memory-native AI systems.