Multimodal large language models (MLLMs) have become the dominant paradigm for visual understanding, but they pay a heavy price by encoding every input into dense, fixed-size patch tokens. The paper argues that visual information is not distributed evenly: some regions need fine-grained detail, while others can be represented compactly. Downsampling discards that detail, and existing token pruning and adaptive methods remain limited in content-adaptive granularity, task generalization, and integration with modern MLLMs and serving infrastructure.
To address this, the authors introduce VisionWeave, which they describe as establishing a native capability they call elastic visual representation weaving — learning end to end where and at what granularity to allocate visual representations. The method combines two components: a gated spatial pooler that builds coarse-grained representations alongside native fine-grained ones within a shared MRoPE coordinate, and a granularity router that learns their content-adaptive allocation.
The capability was validated through self-distillation alone on Qwen3.5-4B and then scaled to Qwen3.8-27B with over 30K A100 GPU-hours. Based on Qwen3.8-27B, VisionWeave adaptively adjusts token savings to the visual content, saving 43.0% of tokens on average while retaining 98.9% of native performance across eight benchmarks. By comparison, token pruning baselines with a fixed 50% savings target preserved only 88% of performance.
The authors report robust efficiency-quality trade-offs across diverse tasks, resolutions, and video frames. When deployed on the SGLang serving engine, the method achieves a 2.3x throughput gain while reducing mean TTFT by 54.4% and mean TPOT by 60.6%.
The team frames VisionWeave as part of a broader shift, comparing it to sparse attention's evolution from a post-training trick into a native model capability. They argue that visual token efficiency holds the same promise, and that elastic visual weaving could become a built-in ability of future multimodal foundation models, enabling efficient long-horizon interaction with visual environments.