Ovis-Embedding: Pushing the Frontiers of Universal Omni-Modal Embeddings
Source
arxiv.org
Author
Embedding Team
Date
Context
Ovis-Embedding is designed to make text, pictures, video and audio searchable through a shared representation. It converts different media into numerical descriptions that can be compared, using one multimodalA model that works with more than text — reading images, audio, or video, and sometimes generating them too.Full definition → backbone rather than assembling separate encoders for each medium.
The researchers train these representations to capture similarity across mixed-media data. They also describe a way to produce smaller representations with adjustable dimensions and little reported performance loss. For a search system, the intended benefit is a common basis for matching a query against different kinds of content, reducing the fragmentation that comes from treating each medium as a separate retrieval problem.
Key takeaways · AI-distilled
The backbone is a pretrained Qwen-omni model adapted through contrastive training with low-rank initialization, rather than separate towers per modality.
Training batches use homogeneous-source sampling so each batch is task-consistent and supplies informative in-batch negatives.
Focal loss emphasizes hard examples, and similarity-based embeddingA list of numbers representing a piece of text's meaning, so that similar meanings end up numerically close and can be searched.Full definition →distillationTraining a small, cheap model to imitate a big one's outputs, keeping much of the capability at a fraction of the cost.Full definition → transfers fine-grained similarity structure from complementary expert models.
Low-rank feature decomposition at inferenceRunning a trained model to get answers — the phase where AI is actually used, as opposed to trained.Full definition → lets users choose smaller embedding dimensions with what the team describes as minimal performance loss; SOTA claims cover MMEB-v3, MMEB-v2, MVEB, MAEB and RTEB.
Terms in this piece · Glossary
embedding — A list of numbers representing a piece of text's meaning, so that similar meanings end up numerically close and can be searched.
multimodal — A model that works with more than text — reading images, audio, or video, and sometimes generating them too.
distillation — Training a small, cheap model to imitate a big one's outputs, keeping much of the capability at a fraction of the cost.
inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
