Researchers have introduced OneStreamer, a 4-billion-parameter model designed to unify perception, memory, and proactive response for streaming video interaction. The work addresses a core difficulty in streaming video LLMs: retaining evidence before its relevance to a future task is known, while still responding once sufficient evidence becomes available.
The model jointly learns query-independent evidence recording and task response through a shared proactive generation process. Its Proactive Hierarchical Caption Memory (PHCM) produces time-grounded local-detail captions and summaries of completed events, with streaming caption targets supervising how observed video prefixes are interpreted during training. At inference, these model-generated records complement a recent visual window, offering reusable factual context without revisiting historical visual features.
To handle the tendency of streaming systems to linger in repeated waiting states, the authors propose Proactive State Transition Learning (PSTL). It preserves supervision at all output anchors and selects representative state-change and state-persistence tokens; according to the paper, PSTL outperforms dense state supervision while supervising only 27.5% of annotated state tokens.
The team also built a streaming data synthesis pipeline that aligns output content and timing with available evidence. Combining synthesized streaming captions and QA with cleaned open-source data yields OneStreamer-1M, a broad-coverage streaming video interaction dataset with over one million records spanning diverse tasks.
In evaluations, the 4B model achieved the best results among compared methods across all eight streaming video understanding benchmarks, covering perception, memory, and proactive response. Ablations indicate that retaining generated captions improves historical QA without degrading real-time perception. The authors frame these results as support for proactive generation as a shared learning interface connecting perception, memory formation, and timely response.