Omni-modal large language models (Omni-LLMs) excel at audio-visual understanding but face high computational costs due to long, redundant token sequences. Existing token compression methods often fail at low token budgets, either discarding critical structural evidence or underutilizing query-aware audio-visual collaboration.
To address this, researchers propose OmniPack, a training-free framework that combines structural compression before the LLM with task-relevant semantic refinement inside it. The pre-LLM stage removes redundancy via modality-specific importance, global coverage, and similarity-aware merging, while the inner-LLM stage consolidates diverse representations using textual guidance and audio-visual collaboration.
Experiments across five benchmarks and three Omni-LLM backbones show OmniPack consistently achieves the best performance-efficiency trade-off. Notably, on Qwen2.5-Omni-7B, it retains 98.0% of original performance while cutting FLOPs to 16.7%, and still holds 92.9% performance with only 6.8% of original FLOPs.
The framework is model-agnostic and requires no training, making it a practical solution for efficient deployment of omni-modal models in resource-constrained environments.