Wearable AI assistants face a dual challenge: they must answer questions about past visual experiences while also recognizing when that history is relevant to the present moment. Existing video-memory systems typically handle one of these tasks well, often relying on separate memory and control mechanisms for proactive assistance.
In a new paper, researchers from Hugging Face introduce GROVE (Growing and Reasoning over Temporally Stratified Memory), a training-free framework that unifies both behaviors using a single memory grown causally from a continuous video stream. GROVE retains fine-grained perceptual evidence and incrementally consolidates it into time-stamped moments, coherent episodes, and recurring cross-day patterns.
Each memory stratum is paired with a scale-native retrieval skill: locating a specific observation, replaying an activity, or traversing long-range regularities. Reactive question answering and proactive assistance share this memory and access interface, differing only in whether retrieval is triggered by a user query or by the current situation.
Across multiple benchmarks, including the challenging MM-lifelong and EgoServe, GROVE achieves the best results among compared methods. Controlled ablations show that the temporal strata and their access skills are complementary, with patterns providing the largest benefit when evidence spans multiple days.
The code is expected to be released on GitHub.