Building Git infrastructure for agent-scale development
- Source
- github.blog
- Author
- Brian Celenza
- Date
GitHub's engineering team explained why its Git storage layer needs a redesign. Under the current Spokes architecture, every replica participates in each write, using a five-replica quorum. Agents that commit after every action strain it with more concurrent writers in the same busy repositories, more merge contention and more read fan-out. GitHub shared usage data including 7.38 billion commits in September and repositories with millions of daily commits. Its stated direction is to decouple durability from read scale, so serving reads no longer depends on the same replica set that confirms writes. For teams running many agents against one repository, the post is a reminder that commit frequency is a load decision, and batching agent commits reduces contention on any Git host.

Context
GitHub's proposed fix is to stop using the same full copies of a repository for both durability and read capacity. In its engineering post, the company says the authoritative copy would sit in Azure Blob Storage, while lightweight workers that cache data answer read requests. Because those workers hold only cached data, GitHub says losing one is closer to a cache miss than losing a replica, and workers can be added during a burst and removed afterward.
The second change limits what each push must wait for. GitHub says only the branch pointer update, called a reference, needs agreement among servers, while storing objects, checking connectivity and scanning can run in parallel. Heavy upkeep such as and garbage collection would move to separate workers.
GitHub reports up to 35 times higher write in internal benchmarks, without describing the setup. That is a company claim about a system still being built; the post defers detail to a later installment.
- Secret — A value such as a password, key or token that lets its holder act as someone; it must be kept out of code and shared files.
- Compaction — Replacing older content in the model's context with a summary of it, usually keeping recent messages, so that a session can go on.
- throughput — How much model output a provider can generate over time, commonly measured in output tokens per second.
- Git and version control — A system for recording and comparing changes to a codebase.
Checking sign-in…
Loading comments…


