Intel / blog

Saving another 100TB of RAM with math (and Rust)

Source: blog.cloudflare.com
Date: Published Sep 18, 2026

Context

Cloudflare reports reclaiming about 100 terabytes of memory by shrinking the routing structures used by one service. Those structures assign cacheable requests to servers, using multiple hash entries per server to distribute work. The company's engineering account says its analysis supported generating 90% fewer entries without appreciably worsening that distribution; a more compact storage layout reduced memory use further.

Changing the routing structures also changes where requests go, potentially sending traffic away from already cached files. Cloudflare therefore kept the old and new versions together during migration and gradually moved traffic by data center. The reported saving came after it retired the old structures, making controlled migration part of the optimization.

Key takeaways · AI-distilled

Cloudflare's Pingora router defaulted to ketama's standard 160 hash points per server (NGINX's default too); the math showed that count could be cut 90% with no meaningful added error, since 32-bit hash collisions become the limiting factor at high hash counts.
A Rust struct- fix (storing each hash/index pair as a packed raw byte array with getters instead of a naturally-aligned struct) cut consistent-hashing memory by 25% on its own, with no change to the hashing algorithm itself.
The two changes together reclaimed over 100TB of RAM globally, on top of the 100TB Cloudflare's DNS team had already freed the month before.
Rather than a global flip, Cloudflare ran the old and new hash rings in memory simultaneously, chosen per request, and rolled out data center by data center, trading temporary extra memory for a clean rollback path and a small blast radius.
Ketama's disk-space-weighted hashing can't be extended to per-request eligibility rules like compliance or feature flags by adding more points to one ring; every combination of constraints needs its own separate hash ring.

Terms in this piece · Glossary

alignment — The work of making AI systems actually pursue what their builders and users intend, rather than something subtly or dangerously different.

Read the source blog.cloudflare.com