Answering questions about long videos frequently demands connecting events that involve the same objects across hours or even days. According to a new research paper from Hugging Face, chronological descriptions and text-derived entities often leave physical identity unresolved: different objects can share a description, while observations of the same object remain disconnected across events.
As a result, retrieving relevant events does not necessarily recover the "biography" of the particular entity a question concerns. To address this, the authors introduce Grounded Entity Biographies (GEB), a long-video memory framework that groups visually grounded observations of the same physical instance across clips into retrievable biographies while preserving the context of each moment.
During question answering, the biography is retrieved alongside episodic evidence, allowing the model to follow an entity through events using identity links established during memory construction.
Evaluations across four benchmarks, including day-long and week-long recordings, demonstrate improvements over prior memory frameworks in both multiple-choice and open-ended question answering. On EgoLifeQA, GEB achieves 72.0% accuracy, 4.4 percentage points above the best published result.
Ablations show that grounded identity association and biography reading both contribute to the gains, which additional descriptions alone do not fully recover.