Multilingual text embedding models are typically fine-tuned with a single training objective across diverse tasks, despite the fact that different tasks require fundamentally different optimization strategies. This one-size-fits-all approach often leads to suboptimal performance, especially when tasks range from translation to retrieval and classification.
To address this, researchers introduce Task-Conditional Flow Matching (TCFM), a novel adaptation framework that selectively applies Flow Matching—a powerful generative technique—to translation tasks, while optimizing retrieval, classification, and pair-classification tasks with objectives better aligned to their specific learning dynamics. This task-aware strategy ensures that each task is trained with the most suitable objective, leading to more balanced and effective adaptation.
TCFM also incorporates teacher-guided representation preservation and a three-stage curriculum to ensure stable adaptation throughout the training process. This combination helps maintain the integrity of the learned representations while progressively adapting the model to new tasks.
Evaluated on the Indic Massive Text Embedding Benchmark, TCFM establishes a new state-of-the-art, consistently improving embedding quality across a diverse set of multilingual tasks and generalizing across different embedding model families. The authors plan to publicly release the codebase and datasets upon acceptance of the paper.