voyage-context-3 embeds chunks with the whole document in view
- Source
- x.com
- Date
📢 voyage-context-3: contextualized chunk embeddings - Auto captures of chunk level detail & global doc context, w/o metadata augmentation - Beats OpenAI-v3-large by 14.24% & Cohere-v4 by 7.89% - Binary 512-dim matches OpenAI (float, 3072-dim) in accuracy, but 192x cheaper in VDB costs
Before: chunk overlaps, context summaries, metadata augmentation Now: voyage-context-3 processes the full doc in one pass and generates a distinct embedding for each chunk. Each embedding encodes the chunk-level details AND full doc context, for more semantically aware retrieval.
For chunk-level retrieval, voyage-context-3 outperforms on average OpenAI-v3-large, Cohere-v4, Jina-v3 late chunking, and contextual retrieval on all domains by 14.24%, 7.89%, 23.66%, and 20.54%, respectively.
For document-level retrieval, voyage-context-3 outperforms OpenAI-v3-large, Cohere-v4, Jina-v3 late chunking, and Anthropic contextual retrieval by 12.56%, 5.64%, 6.76%, and 2.40%, respectively.
Context
Splitting a document into searchable passages can separate a statement from the needed to interpret it. Voyage AI's voyage-context-3 addresses that problem by reading a document's chunks together, then producing a separate numerical representation for each chunk. Each representation combines local detail with broader document context, according to the release explanation.
The article illustrates the problem with a contract clause that specifies an encryption method but leaves the client's identity elsewhere in the document. Joint processing can carry that wider context into the clause's representation while preserving passage-level search. Developers still retrieve focused excerpts, with less reliance on manually attaching background to every one.
- context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
- embedding — A list of numbers representing a piece of text's meaning, so that similar meanings end up numerically close and can be searched.
Checking sign-in…
Loading comments…