Researchers have proposed EM²Mem, a novel framework for long-video question answering that organizes multimodal memory around event anchors rather than isolated fragments. The approach addresses a key limitation of existing methods, which often retrieve captions, frames, transcripts, or graph facts as separate pieces that are not directly usable by language models.
According to the paper, current memory systems require models to reconstruct cross-modal and temporal alignments at inference time, which is challenging with limited context and difficult attribution. EM²Mem instead binds heterogeneous evidence to event anchors during memory construction, creating event-indexed memory cells that align multimodal records, temporal context, graph-linked relations, semantic facts, and provenance.
This design enables compact evidence readout over grounded multimodal events, making the memory generation-ready. The framework was evaluated on three long-video QA benchmarks, where it improved average accuracy by 2.0, 2.4, and 3.7 points over the strongest baseline, and boosted strict event-level Top-5 evidence recall by 7.0 points.
EM²Mem also demonstrated significant efficiency gains, reducing per-query latency by 4.67 times and total inference tokens by 63.66%. The code is planned for integration into the LightMem repository on GitHub.