Yes, CDN improves latencies by caching things closer to the users, but here’s an interesting optimization they do to optimize on latencies…
TCP suffers from slow starts, i.e., when a new TCP connection is established, it doesn’t immediately operate at full bandwidth. Instead, it begins conservatively with fewer segments and doubles them each round-trip time until it detects congestion.
This ramp-up can take several iterations to achieve optimal throughput, which is problematic for latency-sensitive applications. This is a big problem for CDN, because even a few additional round-trips for a massive scale costs a lot.
CDNs solve this by maintaining persistent connection pools to origin servers. Rather than establishing fresh connections for each user request, they keep a pool of long-lived connections alive between edge nodes and origins.
The clever part is pre-warming during low-traffic periods.
CDNs periodically send small amounts of data (like health checks or cache validation requests) over these idle connections. This keeps the TCP congestion window at max and prevents it from shrinking due to inactivity.
Here’s a simple calculation to quantify the impact.
By keeping the congestion window pre-warmed, it is operating at 64KB instead of 4KB. This eliminates the 3-7 round-trip ramp-up delay that would otherwise occur.
For a connection with 50ms round-trip time, this saves 150-350ms of latency, and that’s pretty significant for something that operates at web scale, literally.
Hope you found this interesting, and like always, keep digging deeper.