Inside GitHub’s new Git storage architecture

GitHub has rebuilt how it stores Git repositories because it expects coding agents to make individual repositories far busier, and in internal testing the new design handled 5,000 concurrent agents sending 7,200 pushes per second to one repository while reads kept consistently low latency (GitHub's engineering post). Today the busiest repositories on GitHub receive at most dozens of writes in a second, in short bursts, and GitHub expects those bursts to become a sustained rate within the next year. Each push also triggers more reads, as CI, deployment systems and other agents fetch the new commit. GitHub set a target of at least two orders of magnitude more scale per repository and announced the architecture at its Universe conference (Universe announcements).

The design rests on one observation about Git. The files and commits a push uploads, called objects, never change and are named by their contents, so many writers can store them at once without coordinating. The only part of a push that needs agreement is the reference update, the small pointer that says which commit a branch now points to. GitHub therefore stores objects, bundled into packs, in Azure Blob Storage, and keeps references and the list of packs in a separate replicated log service called Ref-CAS, short for Reference Compare-and-Swap. Requests are served by virtual replicas, disposable machines whose local disks are only caches. A write uploads its pack to Blob Storage first and then commits one transaction to Ref-CAS, so a reference only ever points at data that is already durable. A read starts with a version check, and if nothing has changed, it costs a single comparison.

GitHub explains why it did not keep references in Blob Storage too. A single log blob for a repository's references allowed only one conditional write at a time and topped out in the mid-hundreds of writes per second. Splitting references across several blobs raised the write ceiling but forced every read to check every blob and wait for the slowest, a poor trade when reads outnumber writes by orders of magnitude. In Ref-CAS, each transaction locks only the references it touches and becomes visible once a majority of replicas record it. Because agents mostly push to their own branches, most updates do not conflict, and GitHub says reference state alone can be updated tens of thousands of times per second. Merging the growing pile of small packs, called compaction, now runs on separate background nodes rather than on the machines serving traffic, which GitHub says keeps the pack count, and latency, flat as writes climb.

For developers, pushing, fetching and branching work the same way. The architecture is in a limited private preview, with a broader rollout planned for early next year, and GitHub lists further work such as fetching single objects, replicas in more regions, an atomic multi-file commit API and letting agents read files without cloning. The throughput figures come from GitHub's own internal testing, not an outside benchmark, and the two-orders-of-magnitude figure is a design target rather than a measured result.