- Category
- Developer Tools
- Rank
- No. 1205Tools index
- Type
- TOOL
- Use case
- Security & Identity
- GitHub
- 214 stars
- Latest release
- v1.5.5
- Date
About
Busbar is a self-hosted execution control plane that governs what AI agents can do before they act: it sits in front of model requests, MCP tool calls, A2A delegations, and downstream actions to control where an agent can go, what it may use, what authority it receives, what it may cost, and what evidence is retained. It supports six native LLM protocols in and out, letting teams enforce these boundaries without leaving their existing protocols.
What it does
Busbar is a gateway written in Rust that you run on your own servers between your application and the AI model providers it calls. Your existing client library keeps working: you point its base address at Busbar and swap the key. Busbar then decides which provider answers, converts the request between provider formats when needed, retries a failing provider before any reply reaches you, and applies spending and rate caps per issued key. Provider keys stay inside Busbar instead of inside every app.
Why it's ranked here
Apache 2.0, a single static binary with no runtime or database to install, all six request formats accepted on both sides, and a published threat model that maps each defence to the code that implements it. The project also forbids unsafe Rust at compile time and keeps the admin interface on a separate, loopback-only listener by default. Its strongest claims are its own: the speed and memory figures and the competitor comparison come from the vendor's benchmark harness, and the agent and tool governance in the catalogue pitch is not what the fetched material demonstrates. The README itself says it demonstrates only the model traffic layer today.
What's good
Failover is careful about where it can work. A failing provider is swapped out mid request only before the first byte reaches your client, and the docs explain plainly why no gateway can splice a stream after that. The circuit breaker blames the right party: a bad key benches one lane, while a prompt that is too long for a model moves to a bigger one without penalty. Same-format requests are forwarded byte for byte rather than rebuilt. Spend is stored as token counts and priced at read time, so fixing a wrong rate card re-prices history with a config reload. An offline config check fits in a build pipeline.
Tradeoffs
Budget caps are enforced per node. With several Busbar instances sharing one store, the docs say the fleet can admit up to roughly the cap multiplied by the node count between flushes, and token-per-minute limits are best effort. Governance state is in memory unless you add a SQLite, Postgres or Valkey store plugin, so a restart without one loses keys and usage. The provider catalogue is narrower than LiteLLM's, which the README concedes. The security docs disagree on the admin exposure flag: one names an older insecure switch the architecture page says was retired, so check the current configuration reference before exposing the admin port.
How to use it well
It fits a team that runs its own infrastructure, calls more than one model provider, and wants failover and per-team spending caps without rewriting call sites. Start with a single provider in plain passthrough mode to move keys out of application deployments, then add pools and failover rules. Run the config check in your build. If you need multi-node budgets that hold exactly, plan around the per-node limit or route budgeted traffic through one instance. For long streaming replies, keep a client-side retry. It is not an in-process Python library or a hosted router; teams wanting either are better served elsewhere.
Technical notes+
The route table lives in build_router in crates/busbar/src/main.rs, which also declares #![forbid(unsafe_code)] and installs jemalloc as the global allocator on every target except windows-msvc. Cargo.toml defines a 17-crate workspace (gateway, plugin ABI, SDK, loader, signer, secret-ref, example store and auth plugins) and a release profile with fat LTO, one codegen unit and stripped symbols, keeping panic as unwind so a panic aborts one request task rather than the process. crates/busbar/src/proxy/mod.rs splits the forwarding engine into egress, engine, select, wire, usage and response_body submodules and wires in billing-parity and request-short-circuit test modules. docs/architecture.md describes a superset intermediate representation with ProtocolReader and ProtocolWriter traits and smooth weighted round-robin lane selection. docs/failover.md covers the first-byte boundary, attempt_timeout_ms and on_exhausted actions. THREAT_MODEL.md lists nine threats, including ed25519-signed plugin manifests loaded via memfd_create.
Observed
- License
- Apache-2.0
- Language
- Rust, Cargo workspace; unsafe code forbidden at compile time
- Packaging
- Single static binary via install script, container image, Helm chart
- Interface
- HTTP gateway accepting OpenAI, OpenAI Responses, Anthropic, Gemini, Cohere and Bedrock Converse request formats
- Hosting model
- Self-hosted only; no hosted tier described
- Admin plane
- Separate listener, loopback by default; off-loopback requires mTLS or an explicit waiver
- Governance storage
- In-memory by default; SQLite, Postgres or Valkey via store plugins
- Budget enforcement
- Per node; fleet can exceed a cap between flushes (docs/why-busbar.md)
- Plugins
- Dynamic plugins off by default; signed manifests verified against an embedded ed25519 key
- Config check
- Offline validate flag that exits non-zero on errors
Read from README.md, Cargo.toml, LICENSE, docs/architecture.md, docs/why-busbar.md, docs/failover.md, crates/busbar/src/main.rs, crates/busbar/src/proxy/mod.rs, THREAT_MODEL.md.
Tags
Tech Stack
Media
Comments (0)
No comments yet
Editorially curated, with community endorsements as a secondary signal. Corrections welcome.