An Independent Evaluation of TypeSafe's Jev

Shows a model returning probability-weighted choices matches frontier LLMs on claim verification at roughly 1/500th the cost with about 0.1s latency, but trails on legal reasoning.

• articleAn Evaluation of Inception's Mercury Decidewww.vals.ai
• articleJev introduces a new shape of LLM - System One, aka Decision Modelssimonwillison.net
• articleLangChain Benchmarks a Non-Generative Model as an Agent-Eval JudgeLangChain

Checking sign-in…

Loading comments…