Ovis-Embedding: Pushing the Frontiers of Universal Omni-Modal Embeddings
Source
Embedding Team
Author
Embedding Team
Date
Key takeaways · AI-distilled
The backbone is a pretrained Qwen-omni model adapted through contrastive training with low-rank initialization, rather than separate towers per modality.
Training batches use homogeneous-source sampling so each batch is task-consistent and supplies informative in-batch negatives.
Focal loss emphasizes hard examples, and similarity-based embeddingA list of numbers representing a piece of text's meaning, so that similar meanings end up numerically close and can be searched.Full definition →distillationTraining a small, cheap model to imitate a big one's outputs, keeping much of the capability at a fraction of the cost.Full definition → transfers fine-grained similarity structure from complementary expert models.
Low-rank feature decomposition at inference lets users choose smaller embedding dimensions with what the team describes as minimal performance loss; SOTA claims cover MMEB-v3, MMEB-v2, MVEB, MAEB and RTEB.
Terms in this piece · Glossary
embedding — A list of numbers representing a piece of text's meaning, so that similar meanings end up numerically close and can be searched.
distillation — Training a small, cheap model to imitate a big one's outputs, keeping much of the capability at a fraction of the cost.
multimodal — A model that works with more than text — reading images, audio, or video, and sometimes generating them too.
RAG — Retrieval-augmented generation — fetching relevant documents first and pasting them into the model's context so it answers from your data instead of memory.
Why it matters
A single embedding model covering text, image, video and audio could simplify multimodalA model that works with more than text — reading images, audio, or video, and sometimes generating them too.Full definition → search and RAGRetrieval-augmented generation — fetching relevant documents first and pasting them into the model's context so it answers from your data instead of memory.Full definition → pipelines that currently stitch together separate per-modality encoders.
