OmnisBench
github.com/fortitude-group/omnisbench- Category
- Developer Tools
- Rank
- No. 3623Tools index
Previous survey · No. 3453 ·
- Pricing
- Open Source
- Type
- TOOL
- Use case
- Model & Agent Evaluation
- GitHub
- 1 stars
- Date
About
OmnisBench is an open, reproducible benchmark that measures how close an LLM routing policy gets to the ideal quality-per-dollar cost frontier. It evaluates on fresh task splits the models cannot have memorized, where ideal routing beats using the frontier model alone at roughly 60% lower cost, and every result can be re-graded offline to verify routing-savings claims.
Tags
llmbenchmarkroutingcost-efficiencyopen-sourceevaluation
Tech Stack
Python
Comments (0)
No comments yet
Editorially curated, with community endorsements as a secondary signal. Corrections welcome.