Researchers have introduced Omni-Embed-Mini, a compact omni-modal embedding model that places text, speech, general audio, images, video and visually-rich documents into a single shared embedding space. The work targets a common failure mode: extending a text embedding model to new modalities usually erodes text retrieval quality, and existing omni-modal embedders compensate by scaling to multi-billion parameters.
The core idea is that the teacher signal needs no separate embedding model. Each media sample is paired with a dense cascaded caption, and the teacher target is simply the frozen backbone's own embedding of that caption. Because teacher and student share the same backbone weights, they occupy byte-identical geometry, so lightweight projectors plus phased LoRA adapters on the modality encoders are enough to achieve alignment.
Training combines a Matryoshka SigLIP contrastive loss with an online hybrid hard-negative miner whose negatives sharpen as the encoder improves. The same recipe scales to a 2.3B variant by swapping in a native vision-language backbone.
According to the authors, the 0.9B model keeps its text weights bit-identical to the backbone, meaning training cannot regress text retrieval — it reports 49.57 nDCG@10 on MTEB-v2 BEIR-8 — while adding five additional modalities. It is described as roughly 2.7x to 9.5x smaller than every open omni embedder compared against, small enough to run on device. The 2.3B variant is said to be competitive with the closed gemini-embedding-2, edging ahead of it on the overall-modality average.
Evaluation spans MTEB-v2, MAEB, MMEB-V2 and ViDoRe-V3. Models, training data and the full training and evaluation code are released on the project page at omniembed.cvmbzuai.com.