📢 voyage-context-3: contextualized chunk embeddings
- Auto captures of chunk level detail & global doc context, w/o metadata augmentation
- Beats OpenAI-v3-large by 14.24% & Cohere-v4 by 7.89%
- Binary 512-dim matches OpenAI (float, 3072-dim) in accuracy, but 192x cheaper in VDB costs
Before: chunk overlaps, context summaries, metadata augmentation
Now: voyage-context-3 processes the full doc in one pass and generates a distinct embedding for each chunk. Each embedding encodes the chunk-level details AND full doc context, for more semantically aware retrieval.
For chunk-level retrieval, voyage-context-3 outperforms on average OpenAI-v3-large, Cohere-v4, Jina-v3 late chunking, and contextual retrieval on all domains by 14.24%, 7.89%, 23.66%, and 20.54%, respectively.
For document-level retrieval, voyage-context-3 outperforms OpenAI-v3-large, Cohere-v4, Jina-v3 late chunking, and Anthropic contextual retrieval by 12.56%, 5.64%, 6.76%, and 2.40%, respectively.
embedding — A list of numbers representing a piece of text's meaning, so that similar meanings end up numerically close and can be searched.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
RAG — Retrieval-augmented generation — fetching relevant documents first and pasting them into the model's context so it answers from your data instead of memory.
Why it matters
It removes the contextual-retrieval preprocessing step from RAGRetrieval-augmented generation — fetching relevant documents first and pasting them into the model's context so it answers from your data instead of memory.Full definition → pipelines, and binary 512-dimension vectors are claimed to match OpenAI float 3072 accuracy at a fraction of the vector database cost.
Read the full thread on X ↗
