Fine-grained visual perception in multimodal large language models (MLLMs) is typically improved by raising input resolution, but the extra visual tokens increase vision-encoding and language-model prefilling costs. A new paper posted on Hugging Face, Region-Level Policy Optimization for Fine-grained MLLM Perception, argues that the two operations behind fine-grained perception — localizing the region of interest (RoI) and recognizing its content — have different resolution requirements.
In a controlled diagnostic, the authors report that localization tolerates roughly 3 to 4 times stronger token compression than recognition. This motivates a strategy of localizing from a coarse view and concentrating resolution on the selected evidence rather than uniformly increasing resolution.
The paper notes that decoding coordinates directly with the MLLM can be trained end-to-end from answers, but costs a full model pass per query and depends on grounding ability. A lightweight proposal network distilled from the model's attention is faster, yet inherits the noise of its attention targets. Because the RoI reaches the answer through a discrete region choice, its faithfulness to the answer cannot directly supervise the network.
To address this, the authors introduce Vision-RL², which optimizes the proposal network with region-level reinforcement learning. Coherent regions are treated as actions, and a frozen MLLM reader scores each one by how its removal changes the answer likelihood. Complementary subtractive and additive objectives suppress distracting proposals and recover missing evidence, updating only the predictor without region annotations, response sampling, or reasoning trajectories. The refined proposal also enables a sparse visual encoding that magnifies evidence and excludes background tokens.
Across six fine-grained benchmarks (V*, ZoomBench, HR-Bench 4K/8K, MME-RealWorld EN/CN) and four MLLM backbones (Qwen3.5-4B/9B, Qwen2.5-VL-7B, and the encoder-free Gemma-4-12B), Vision-RL² is reported to improve accuracy over the base model at every visual-token budget, and to surpass its largest-budget accuracy with about 4 times fewer visual tokens. The authors also state that it outperforms prior methods that fully fine-tune the model while training an order of magnitude fewer parameters.
Code, checkpoints for all four backbones, training data, and an interactive project page are linked from the paper.