Shows how a large engineering org structures 20+ internal developer tools around AI, which helps teams designing agent infrastructure.
Key takeaways · AI-distilled
Uber decided in 2023 that its AI work must stay model- and agent harnessThe scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.Full definition →-agnostic, so it built a Model Gateway that switches between Anthropic, OpenAI and self-hosted open models, with security checks, per-user spend tracking and audit logs added.
Engineers at Uber get a tokenThe chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.Full definition → spend tier, and a manager can approve going past it for the month. A plugin suggests ways to lower an engineer's costs, and the episode says Uber cut token costs by roughly half over the last few months.
Uday Kiran Medisetty calls code review a major bottleneck. Humans still review every PR, but by then AI review has already run, demo screenshots or videos may be attached, and each change carries a risk score to help reviewers decide how much attentionThe mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.Full definition → it needs.
CI job volume at Uber more than doubled in a year, prompting an internal code red. The fix moved simple checks such as linting and build rules into the AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition →'s local loop, run with an afx validate command before a PR is opened.
Because agents make PR counts easy to inflate, Uber stopped leaning on PR velocity and now labels each PR as feature, bugfix, maintenance, testing or docs. It reports the feature share rising and the maintenance share falling.
Terms in this piece · Glossary
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
attention — The mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.
