AI Gateway Models & Pricing

Why it's ranked here

The data trail is open. The suite is MIT licensed, every result is committed to the repository as JSON, and runs happen on GitHub Actions rather than on someone's laptop. Each catalogue run writes both a dated snapshot and a rolling latest file, so a reader can diff what changed between runs. It also sits beside a careful latency benchmark for the same routers that separates connection setup from time to first token, which puts availability and speed for one set of routers in a single public repository.

What's good

Targeted reruns cannot overwrite the main dataset: a run limited to one provider writes to its own suffixed files and leaves the full-roster results intact. The catalogue pass sends no chat requests, so reproducing it burns no tokens. The companion latency suite documents its setup down to the prompt, a 200-token output cap, temperature 0 and a 45-second timeout, and it records receipt headers such as request IDs so a disputed number can be traced back to the provider's own logs.

Tradeoffs

Everything is measured from one place. The latency runs execute on Namespace runners in Northern Virginia, and the methodology says so, so routers whose edges sit far from there may look slower than they would for you. The project's own docs warn that results for different target LLMs are not comparable, even for the same router. Neon is both a listed sponsor and a benchmarked router, and Namespace sponsors the runs while also appearing as a benchmarked sandbox provider. The README states sponsors cannot influence results; that is a policy, not something the code enforces.

How to use it well

Treat it as a starting map when picking a router for a specific LLM: confirm the LLM is listed, then check that router's own pricing page before committing, since the table only reflects what each catalogue endpoint publishes. For speed, read the cold numbers if you run serverless functions or CLIs that open a fresh connection per request, and the warm numbers if you keep a connection pool. To check a claim yourself, clone the repository, add your own API keys and run the catalogue job for a single provider. It does not measure answer quality.

Observed

License

MIT

Language

TypeScript, run with tsx in a pnpm workspace

Interface

bench run CLI from the benchsdk runner, with a --provider filter

Measurement method

Queries each router's model-list endpoint; no chat probes are sent

Output format

Normalized JSON committed to the repository, written as a dated file plus a latest file

Run environment

GitHub Actions on Namespace runners, single vantage point in Northern Virginia

Sponsor overlap

Neon is a listed sponsor and a benchmarked router