About
Relay is a self-hostable AI gateway that classifies each incoming request to determine which model capabilities it needs, then routes it across configured providers while tracking each provider's rate limits and available capacity. It uses a queue-first approach, pacing or briefly delaying requests to stay within a preferred provider's limits rather than immediately falling back to a worse model, which the author built after hitting inconsistent rate limits while chaining several providers' free tiers.
Why it made the leaderboard
Relay classifies and routes requests across LLM providers using its own trained classification models, then queues and paces calls against rate limits instead of falling back immediately, useful for anyone stitching together multiple providers.
Intel on Relay
Tags
Tech Stack
Comments (0)
No comments yet
Editorially curated, with community endorsements as a secondary signal. Corrections welcome.