Researchers have introduced Ovis-Embedding, a family of omni-modal embedding models designed to encode text, images, video and audio into a common representation space. According to the report, the approach relies on native integration of the four modalities rather than assembling separate modality-specific towers.
The work is built around three stated advances. First, native omni-modal initialization adopts a pretrained Qwen-omni model as the embedding backbone and adapts it through contrastive training with low-rank initialization. Second, data-centric omni-modal training constructs a broad, high-quality corpus spanning text, images, video, audio and interleaved multimodal data, with a homogeneous-source sampling strategy that forms task-consistent batches containing informative in-batch negatives.
Third, the report describes embedding-specific training and inference optimization: focal loss is used to emphasize hard examples, while similarity-based Embedding Distillation transfers fine-grained similarity structure from complementary experts. At inference time, low-rank feature decomposition is said to enable compact embeddings with flexible dimensionality and minimal performance loss.
Empirical evaluations report state-of-the-art performance for the Ovis-Embedding family on MMEB-v3, MMEB-v2, MVEB, MAEB and RTEB, spanning text, image, video and audio modalities. The authors frame these results as evidence that unified omni-modal training can help overcome modality fragmentation and advance universal embedding models for any-to-any retrieval.