Researchers have introduced OmniScope, a training-free token compression framework designed for omnimodal large language models (OmniLLMs). The work addresses a critical flaw in existing compression methods, which typically rely on a single modality to guide token retention across all modalities.
The authors demonstrate that this unidirectional guidance is often suboptimal: for the same query, audio and video relevance can peak at different moments. This cross-modal salience mismatch can lead to the discarding of answer-critical cues, especially under aggressive compression.
OmniScope tackles this by using the query as a shared semantic anchor while estimating relevance separately for audio and video. It allocates modality-specific token budgets, prunes visual tokens with an anchor-delta strategy that preserves both global context and temporal changes, and merges audio tokens within each second to reduce redundancy while maintaining temporal continuity.
Across four audio-video benchmarks and two Qwen2.5-Omni model scales, OmniScope achieved the best average accuracy across all compression settings. At 25% overall token retention, it delivered up to 3.53x prefill speedup and more than 15% GPU memory reduction, with only a 0.35-point drop in average accuracy.
The results suggest a simple design principle for OmniLLM inference: share the query across modalities, but not the salience estimates. The code is available on GitHub.