Memory is becoming essential infrastructure for long-horizon LLM agents, yet choosing the right memory substrate—the underlying medium for storing and representing memory—remains a challenge. A new research paper from Hugging Face presents a controlled evaluation of diverse memory substrates, aiming to provide empirical guidance for building efficient and reliable agent memory systems.
The study covers a wide range of substrates, including dense and sparse indices, text records, structural stores, hierarchical stores, refinement-based memories, parametric updates, and activation-compatible context mechanisms. Using three backbone models and four benchmark suites spanning user-centric QA and agent-centric decision-making, the researchers instrumented 26 performance and efficiency metrics under a unified harness.
Key findings reveal that no single substrate consistently dominates across all conditions. Broad retrieval benefits long-context factual QA, but excessive retrieval can harm sequential decision-making by diverting attention from action-critical context. Scalability introduces another routing axis: substrates that perform well at moderate history lengths may become costly or brittle at longer horizons.
The authors argue that these results motivate substrate routing as a necessary component of adaptive agent memory systems. They provide empirical guidance for designing regime-aware long-term memory, and code will be made available upon acceptance.