More and more businesses are moving their core operations to the cloud, but a common phenomenon is emerging: after migrating, the network experience for some services doesn't improve and sometimes even worsens. Web pages load slower, cross-region access latency increases, and occasional lag during peak hours becomes the norm.
Where's the problem? Cloud providers' backbone networks are demonstrably more powerful than self-built data centers, so why does the real-world experience often fall short?
The answer lies in the fact that cloud architecture fundamentally changes the path and nature of network traffic, and many businesses focus only on compute and storage resources when migrating, neglecting to redesign their network layer.
In the era of traditional self-built data centers, network topology was relatively simple: user requests entered the corporate data center, and application servers and databases communicated within the same local network, with latency typically under a millisecond.
After moving to the cloud, this model is upended.
Many early cloud adoption strategies were simple: just lift and shift virtual machines from the on-premises data center to the cloud, binding the public IP directly to a single cloud server. User requests hit this server first, and if it needs to access the database, traffic might use the cloud's internal network. However, with poor architecture, some traffic might even go out to the public internet and back.
A more typical problem is inter-service communication. In a microservices architecture, a single user request can trigger dozens of internal service calls. If these services are spread across different Availability Zones (AZs) or even different regions, each call traverses the cloud's internal routing. While cloud provider networks are fast, cross-AZ latency typically adds 1-3ms, and cross-region can be 20-50ms. With dozens of calls in a chain, total latency can jump from tens of milliseconds to hundreds, severely degrading user experience.
Cloud providers' bandwidth-based pricing model often leads businesses to choose conservative bandwidth configurations initially. During peak business hours, outbound bandwidth saturates, and new connection requests start queueing or dropping packets. This is especially noticeable during flash sales, live streaming events, or peak enrollment seasons.
Even more subtle is the asymmetry between inbound and outbound bandwidth. Many cloud instances have explicit limits on outbound bandwidth (data sent from the server), while inbound bandwidth (incoming user requests) is relatively generous. If your application is heavy on data egress (e.g., video transcoding and distribution, big data analytics exports), outbound bandwidth can easily become a bottleneck—a problem often overlooked during the design phase.
In cloud environments, components like load balancers, CDNs, and API gateways are layered. The DNS resolution path becomes longer. A user request goes to the CDN, which then points back to the load balancer, which then distributes to the backend instances. Each layer of DNS resolution can add tens to hundreds of milliseconds of latency. If any layer is misconfigured (e.g., overly long TTL causing slow cache invalidation, or cross-region DNS resolution taking a detour), the problem is magnified.
Many businesses deploy applications across multiple regions for high availability. However, without adequate data synchronization and traffic routing mechanisms, such deployment can actually hurt performance.
A classic scenario: a primary database in Beijing, and a read-only replica in Shanghai. Requests from Shanghai users are routed to the application node in Shanghai. But when the application needs to read the latest data, it must either access the primary database in Beijing across regions (high latency) or read the Shanghai replica (which may have stale data). If the business has strong consistency requirements, this architecture creates a dilemma.
A deeper issue is the bandwidth cost of data synchronization. Cross-region data replication continuously consumes cloud internal network bandwidth, and cloud providers typically charge significant fees for cross-region traffic. For data-heavy services, monthly cross-region data transfer costs can easily exceed compute resource costs.
Cloud object storage (e.g., S3, OSS) is known for low cost and high durability, but its latency model is fundamentally different from traditional block storage. Object storage is optimized for large file sequential reads. For small, random file access, latency can reach tens to hundreds of milliseconds.
Many businesses treat object storage as a "universal storage": dumping log files, config files, session data, and even database backups into it. The result is that frequent small file reads/writes drag down overall performance. A typical example: an e-commerce company stored product thumbnail images in object storage. During a flash sale, image loading latency spiked because each thumbnail was only a few dozen kilobytes, and object storage's metadata query and connection setup overhead far exceeded the time to transfer the data itself.
Security Groups and Network ACLs are the first line of traffic control in cloud environments, but too many rules or improper configuration can introduce processing latency.
Each inbound/outbound rule must be matched. When rule counts reach hundreds, packet filtering processing time can rise from microseconds to milliseconds. If rules are poorly designed (e.g., many "deny" rules placed after "allow" rules, causing the system to traverse most of the rule list every time), latency is amplified.
Even more subtle is the overhead of stateful connection tracking. Security Groups track connection states by default. In high-concurrency scenarios, the connection tracking table can be overwhelmed, causing new connection requests to be rejected or queued. This can be especially devastating during burst traffic events like flash sales.
When a business feels "slower," how do you determine if the issue is the cloud network or the application itself?
Use curl -w or browser developer tools to break down an HTTP request into:
DNS resolution time
TCP connection establishment time (including TLS handshake)
Time to First Byte (TTFB)
Content download time
If DNS and TCP connection times are normal but TTFB is high, the problem lies in server-side processing or the backend call chain. If TCP connection time itself is long, the problem is likely at the network layer.
In a microservices architecture, use distributed tracing tools (e.g., OpenTelemetry, SkyWalking) to record the duration of every service call. Pay close attention to:
Proportion of cross-AZ/cross-region calls
Latency distribution of database access (P50, P95, P99)
Cache hit rates and cache access latency
Often, under the surface of "slow network," the real issue is a sudden spike in response time from a specific service, dragging down the entire call chain.
Major cloud providers offer metrics for Virtual Network Interfaces (ENI/NICs), including traffic volume, packet loss, and latency. Focus on:
Whether outbound bandwidth utilization is consistently near the limit
Presence of abnormal packet loss (even 0.1% loss significantly impacts TCP connections)
Security group rule match counts and connection tracking table usage
Access the service from different network environments (different ISPs, different regions, different devices) and compare latency differences. If a specific path is significantly slower, issues might include:
Inadequate CDN node coverage
Poor backbone network quality in a specific region for that cloud provider
DNS resolution being hijacked or taking a detour
A core principle of cloud architecture: compute should be located as close as possible to its data.
For read-heavy, write-light scenarios, deploy caches and read replicas on edge nodes near users.
For real-time data processing, avoid cross-region calls; schedule compute tasks to the region where the data resides.
Utilize cloud providers' edge compute services (e.g., CloudFront Functions, edge containers) to perform lightweight processing closest to users, reducing origin traffic.
"East-West" traffic refers to communication between services within the cloud. Optimization includes:
Deploy the same microservice cluster within the same Availability Zone as much as possible to reduce cross-AZ calls.
If cross-AZ communication is unavoidable, use the cloud provider's high-bandwidth, low-latency internal network or direct connect solutions, avoiding the public internet.
Service Mesh sidecar proxies introduce additional latency. In high-performance scenarios, evaluate if they are necessary or choose a lighter implementation.
Choose storage based on access patterns:
Hot data, small files, low latency required → Cloud block storage or high-performance file storage
Large files, sequential reads/writes, archive/backup → Object storage
Temporary cache, session data → In-memory databases (e.g., Redis)
Don't treat object storage as a general-purpose file system; its latency model makes it unsuitable for high-frequency, small-file random access.
Estimate bandwidth needs during the architecture design phase:
Estimate peak outbound bandwidth requirements, reserving 30%-50% buffer.
Distinguish between "burst" and "sustained" bandwidth needs. Use cloud providers' bandwidth packages or elastic bandwidth products to optimize costs.
For high-traffic services, consider using CDN or P2P distribution to reduce origin server egress pressure.
Prioritize rules in Security Groups by "most frequently matched" to reduce traversal time.
Regularly clean up invalid rules, keeping per-security-group rule counts manageable.
In high-concurrency scenarios, evaluate whether to disable certain connection tracking features (e.g., using stateless firewall rules instead of stateful ones).
Not all workloads are best suited for pure public cloud architecture. In the following cases, a hybrid cloud or dedicated network connection might be better:
Data compliance requirements: Industries like finance and healthcare have strict data sovereignty rules. Core data must remain on-premises; only edge computing or external-facing services go to the cloud.
Ultra-low latency needs: Scenarios like high-frequency trading or industrial control require sub-millisecond latency. Public cloud multi-tenancy and virtualization overhead may be too high; on-premises bare metal or FPGA acceleration might be needed.
Massive data transfers: Scenarios requiring TB-scale daily uploads to the cloud. Public internet bandwidth is costly and unstable; dedicated connections (e.g., MPLS, SD-WAN) offer more predictable quality and cost.
Existing infrastructure investments: If a company has already invested heavily in its own data centers, abandoning them isn't economical. A hybrid architecture lets legacy assets contribute while leveraging the cloud's elasticity.
Moving to the cloud is not the end; it's the beginning of redesigning your network architecture. Cloud providers offer powerful infrastructure, but how to combine these building blocks into an efficient, stable, and cost-effective network still requires businesses to make decisions based on their specific workloads.
Network performance issues often don't appear on day one of deployment. They surface gradually as the business grows, architecture becomes more complex, and traffic patterns change. Understanding cloud network traffic models and potential bottlenecks beforehand, and planning during the architecture design phase, is far more effective than "firefighting" after issues arise.
After all, the cloud network doesn't automatically get faster — it simply gives you the tools. How you use them is still a matter of design.