Multimodal retrieval-augmented generation (RAG) systems pull in text, tables, images and videos, but the choice of retrieval granularity does not settle how much context should be kept from each retrieved item. Coarse units drag in irrelevant content, while uniformly fine selection can strip away context needed to interpret the evidence.
Existing compressors tackle this trade-off with modality-specific mechanisms, leaving open a shared procedure for adapting the retained extent region by region across heterogeneous items. The new paper introduces CANOPY (Canonical Projection over Hierarchy), a framework for adaptive-granularity post-retrieval evidence compression.
CANOPY represents retrieved items as hierarchies and uses a node encoder fine-tuned on gold evidence to score regions against the query. A parent-relative refinement step compares these scores to select multiple regions at different granularities, without LLM calls for node-level pruning. Because compression cannot recover evidence that was never retrieved, a critic requests targeted follow-up retrieval when it judges the accumulated evidence insufficient; newly retrieved items are compressed before being added.
Across five QA benchmarks over a 33M-item heterogeneous corpus, CANOPY achieves higher average answer accuracy than the evaluated retrieval baselines. Ablations indicate that additional retrieval drives the main accuracy gains on multi-hop QA.
In the unrouted Qwen3-VL-8B-Instruct setting, compression reduces reader-input evidence tokens by 14.2–27.7% relative to the same iterative pipeline without compression, with comparable answer accuracy. The authors summarize the result as 14–28% token reduction at comparable accuracy, with the best average accuracy among the evaluated retrieval baselines.