• benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
• embedding — A list of numbers representing a piece of text's meaning, so that similar meanings end up numerically close and can be searched.
• eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.

Choosing an embedding model on one retrieval score can mislead you. MTEB is the multi-task evaluation behind the public embedding leaderboard, letting you compare candidates on the task family your product actually runs.

• articleThe Data Provenance Initiative A Large Scale Audit Of Dataset Licensing And Attribution In Ai 2023 10 25
• articleUnderstanding And Mitigating Language Confusion In Llms 2024 06 28
• articleProcedural Knowledge In Pretraining Drives Reasoning In Large Language Models 2024 11 20
• articleLanguage Models Don T Always Say What They Think Unfaithful Explanations In Chain Of Thought Prompting 2023 05 07

• postRTEB uses held-out datasets to test embeddings that overfit MTEBVoyage AI by MongoDB
• postvoyage-large-2-instruct takes the MTEB top spot at 1024 dimensionsVoyage AI by MongoDB
• articleGiga-Embeddings: Mixture-of-Experts Encoders for High-Throughput Text EmbeddingsEgor Kolodin, Egor Krasnoperov, Evgeniy Kosarev, Fyodor Minkin

Checking sign-in…

Loading comments…