Memory Savings Can Require a Routing Migration

Summary

Cloudflare reports reclaiming 100 TB of RAM by changing the routing structures in one Pingora-based service. Its September 18, 2026 engineering report also describes the less visible part of that saving: moving requests to a new hash ring without overwhelming origins with cache misses. The useful lesson is to evaluate memory use and migration behavior together.

Key Updates

Smaller routing structures. Cloudflare reduced each stored hash point from eight bytes to six, then used 90% fewer hashes per server in its configuration. The 25% saving in point storage and the reduction in hash count describe different changes; neither is a percentage reduction in total service memory.

A routing migration accompanied the optimization. Changing the hash ring changes which backend receives a request. Cloudflare kept old and new rings in memory, made the choice stable per request hash, and controlled traffic share separately from data-center scope. It retained a rollback path while monitoring cache behavior and origin traffic.

Available as an optional feature. The report describes the new v2 ring in pingora-ketama as an opt-in Cargo feature, with the existing v1 ring unchanged and both able to run together. The reported 100 TB saving followed the fleet migration and removal of the old rings. It is Cloudflare’s result for this service, not a savings forecast for other deployments.

Why It Matters

A change can preserve the purpose of a load balancer while changing where individual requests land. If cached content remains on the old destination, a smaller routing table can mean more cache misses and extra work for origin servers. A memory benchmark alone would miss that cost.

For builders, this makes rollout behavior part of the optimization’s acceptance criteria. Check how much work moves, where it moves, and whether the system can absorb the transition. A low global traffic percentage can still disturb caches across many locations. This report gives a concrete reason to control location as well as volume; it does not establish a universally safe rollout recipe.

Builder Takeaway

Test the routing change before counting the memory saving.

For one cache-backed service, replay a representative request sample through the current and proposed routing configurations in a test environment. Compare memory use, backend distribution, and the share of requests sent to a different backend. Include the temporary memory needed to keep both configurations available.

If that evaluation supports a canary, limit it to one location and a small traffic share. Set stop thresholds for cache misses, origin load, latency, and errors, and verify rollback before expanding. Use your workload to choose the hash count and rollout limits. Cloudflare’s 90% reduction is a result to investigate, not a default to copy.