When embedding a chunk should include the whole document
- Source
- x.com
- Date
What if your embeddings could see the whole document? Now they can with voyage-context-3, our contextualized chunk embedding model. Learn what contextualized embeddings are, when to use them, and how to evaluate them on your data in our latest tutorial! Link below 👇
💡 Unlike traditional embeddings, which encode chunks in isolation, contextualized chunk embeddings encode both chunk-level detail and document-level context.
💡Contextualized chunk embeddings are particularly valuable for long, unstructured documents—such as legal contracts, financial documents, research papers, and technical documentation—in which queries often depend on document-level context.
Learn more in our tutorial: https://t.co/v1NOvJ9xKK
Context
A passage can contain the detail a search needs while leaving its subject unstated. Voyage AI illustrates this with a hypothetical contract clause about encryption whose client is named elsewhere in the document. that clause alone can lose the connection between the requirement and the client.
Its voyage-context-3 explanation describes processing a document's chunks together, then producing a separate search vector for each chunk. Each representation incorporates both local detail and information from the surrounding document. The intended benefit is precise passage retrieval when understanding a passage also requires from another section.
- embedding — A list of numbers representing a piece of text's meaning, so that similar meanings end up numerically close and can be searched.
- context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
Checking sign-in…
Loading comments…