Unified multimodal retrieval aims to find candidates that satisfy complex user intent expressed through heterogeneous inputs. While Large Vision-Language Model (LVLM)-based retrievers are efficient and scalable, directly encoding raw multimodal inputs often misses fine-grained discriminative cues, leading to confusion among semantically similar candidates.
Recent methods mitigate this by generating Chain-of-Thought (CoT) rationales to enrich query representation. However, such reasoning is typically derived from the query alone: it explains what the query describes, but not what the retriever misunderstands. The researchers argue that effective retrieval reasoning should be conditioned on retrieval feedback.
Based on this insight, they introduce UniME-R1, an embedder-adviser framework that learns to reason over initially retrieved candidates and generate Retrieval-Centric Chain-of-Thought (RC-CoT). The adviser analyzes candidates individually to identify discriminative cues confused by the embedder. If the target appears in the initial top-k set, UniME-R1 directly reranks; otherwise, it generates RC-CoT to refine retrieval direction and performs full-corpus re-retrieval with a dual-mode embedder.
To train the framework, the authors mine hard negatives to simulate realistic retrieval failures, jointly optimize direct retrieval and RC-CoT-augmented retrieval, and align the adviser with retrieval outcomes through supervised learning and retrieval-oriented reinforcement learning. Extensive experiments on MMEB-V2 and diverse general multimodal retrieval benchmarks show consistent improvements over strong baselines.