5 min read
On this page

Modern Networking

This file covers the technologies and architectures that define modern network infrastructure.

Content Delivery Networks (CDN)

CDNs cache content at edge locations geographically close to users, reducing latency and offloading origin servers.

Architecture

User (Tokyo) → CDN Edge (Tokyo) → [Cache HIT: serve from edge]
                                  → [Cache MISS: fetch from origin, cache, serve]

Edge locations: Hundreds to thousands worldwide. Major CDNs: Cloudflare (300+ cities), AWS CloudFront (400+ PoPs), Akamai (4000+ locations), Fastly.

What CDNs Cache

  • Static assets: Images, CSS, JS, fonts, videos
  • API responses (with appropriate cache headers)
  • Entire pages (static sites, SSG)
  • DNS responses
  • TLS termination at the edge

Cache Control

HTTP cache headers drive CDN behavior:

Cache-Control: public, max-age=3600, s-maxage=86400
Cache-Control: private, no-cache  (don't cache in CDN, only browser)
Cache-Control: no-store            (never cache)
ETag: "abc123"                     (conditional requests)

Cache invalidation: The hardest problem. Purge by URL, tag, or wildcard. TTL-based expiry.

Benefits

  • Lower latency: Content served from nearby edge.
  • DDoS absorption: Distributed edge absorbs volumetric attacks.
  • Offload origin: 90%+ of requests served from cache.
  • Global reach: No need for your own global infrastructure.

Load Balancing

Layer 4 (Transport)

Forward based on IP + port. No inspection of application data.

Algorithms: Round-robin, least connections, IP hash, weighted.

Technologies: Linux IPVS, HAProxy (L4 mode), AWS NLB, F5 BIG-IP.

Advantages: Very fast (~1 μs added latency). Handles any TCP/UDP protocol. High throughput.

Layer 7 (Application)

Inspect HTTP headers, URL path, cookies. Route based on content.

/api/*  → API server pool
/static/* → Static file server
/ws/*   → WebSocket server pool

Features: SSL termination, header manipulation, request routing, health checks, rate limiting, WAF.

Technologies: nginx, HAProxy (L7 mode), Envoy, AWS ALB, Traefik.

DNS-Based Load Balancing

Return different IP addresses for the same domain name.

Advantages: Simple. Geographic routing (return nearest server's IP). Disadvantages: DNS caching makes changes slow. No health checking (without smart DNS).

Anycast

Same IP address announced from multiple locations (via BGP). Packets routed to the nearest location.

Used by: CDNs (Cloudflare), DNS (root servers, 1.1.1.1, 8.8.8.8), DDoS mitigation.

Service Mesh

A dedicated infrastructure layer for service-to-service communication in microservices.

Architecture

Service A                    Service B
┌────────────┐              ┌────────────┐
│ App code   │              │ App code   │
├────────────┤              ├────────────┤
│ Sidecar    │─── mTLS ────│ Sidecar    │
│ (Envoy)    │  traffic    │ (Envoy)    │
└────────────┘              └────────────┘
       ↕                          ↕
  Control Plane (Istio/Linkerd)
  - Service discovery
  - Load balancing policy
  - TLS certificate management
  - Traffic routing rules
  - Observability config

Features

  • mTLS (mutual TLS): Encrypted and authenticated service-to-service communication. Automatic certificate rotation.
  • Traffic management: Canary deployments, A/B testing, circuit breaking, retries, timeouts.
  • Observability: Distributed tracing, metrics, access logs — without changing application code.
  • Policy: Rate limiting, access control, quota enforcement.

Implementations

Istio: Most feature-rich. Envoy sidecar proxy. Complex but powerful.

Linkerd: Simpler, lighter. Rust-based proxy (linkerd2-proxy). Lower resource overhead.

Envoy (standalone): High-performance proxy. C++. Used as the data plane in Istio and many other systems.

API Gateways

Entry point for external API traffic. Handles cross-cutting concerns.

Functions: Authentication, rate limiting, request routing, API versioning, request/response transformation, logging, analytics.

Examples: Kong, AWS API Gateway, Apigee, Traefik.

API gateway vs service mesh: API gateway handles north-south traffic (external → internal). Service mesh handles east-west traffic (internal ↔ internal). Often used together.

gRPC

Google's RPC framework. Protocol Buffers over HTTP/2.

service Greeter {
    rpc SayHello (HelloRequest) returns (HelloReply);
    rpc StreamGreetings (HelloRequest) returns (stream HelloReply);
}

message HelloRequest { string name = 1; }
message HelloReply { string message = 1; }

Features:

  • Binary serialization (protobuf): ~10× smaller, ~10× faster than JSON.
  • HTTP/2: Multiplexing, streaming, header compression.
  • Streaming: Unary, server-streaming, client-streaming, bidirectional.
  • Code generation: Generate client/server stubs in many languages from .proto file.
  • Deadlines: Built-in deadline propagation across services.

Used by: Google (internal), many microservice architectures, Kubernetes API.

GraphQL Networking

Single endpoint: POST /graphql. Client specifies exactly what data it needs.

query {
    user(id: "123") {
        name
        email
        posts(limit: 5) {
            title
            comments { count }
        }
    }
}

Benefits: No over-fetching (get exactly what you need). No under-fetching (get everything in one request). Strongly typed schema.

Networking considerations: All requests are POST (no HTTP caching by default). Batching and caching require client-side solutions (Apollo Client, urql). Subscriptions use WebSocket.

WebRTC

Real-time peer-to-peer communication in the browser. Audio, video, and data.

Architecture:

Browser A                        Browser B
    │── Signaling (via server) ──→│  (SDP offer/answer, ICE candidates)
    │                              │
    │←───── Direct P2P ──────────→│  (media/data via SRTP/SCTP)
    │    (STUN/TURN for NAT)      │

Components:

  • Signaling: Exchange connection info (SDP, ICE candidates) via any channel (WebSocket, HTTP).
  • STUN: Discover public IP/port (NAT traversal).
  • TURN: Relay server for when P2P fails (symmetric NAT).
  • ICE: Framework for finding the best P2P path.
  • SRTP: Encrypted media transport.
  • SCTP (over DTLS): Reliable/unreliable data channels.

Use cases: Video conferencing (Google Meet, Zoom web client), voice calls, screen sharing, peer-to-peer file transfer, multiplayer gaming.

Network Function Virtualization (NFV)

Replace dedicated network hardware (firewalls, load balancers, WAN optimizers) with software running on commodity servers.

Benefits: Flexibility, scalability, cost reduction, faster service deployment.

DPDK (Data Plane Development Kit): Bypass the kernel network stack for high-speed packet processing. User-space drivers. 100+ Gbps on commodity hardware.

Network Observability

Three Pillars

Logs: Structured event records. Searchable. Per-request detail.

Metrics: Numerical measurements over time. Aggregated. Dashboards and alerts. Prometheus, StatsD, Datadog.

Traces: End-to-end request flow across services. Distributed tracing.

Distributed Tracing

Track a request through multiple services:

[Frontend: 50ms] → [Auth: 5ms] → [API: 100ms] → [DB: 80ms]
                                       ↓
                                  [Cache: 2ms]

Standards: OpenTelemetry (vendor-neutral), W3C Trace Context (header propagation).

Tools: Jaeger, Zipkin, Grafana Tempo, AWS X-Ray, Datadog APM.

How it works: Each service propagates a trace ID and span ID in request headers. Each service reports its span (start time, duration, metadata) to a collector.

Key Metrics for Networking

  • Latency: p50, p95, p99 response times. Not just averages.
  • Error rate: 5xx responses / total. By endpoint.
  • Throughput: Requests per second. Bytes per second.
  • Saturation: Connection pool utilization, CPU, memory, bandwidth.
  • Availability: Uptime percentage. SLA compliance.

RED method (for services): Rate, Errors, Duration. USE method (for resources): Utilization, Saturation, Errors.

Applications in CS

  • System design: CDN + load balancer + API gateway + service mesh = modern production stack.
  • Performance: Understanding CDN caching, load balancing algorithms, and gRPC vs REST tradeoffs.
  • Debugging: Distributed tracing, network metrics, and log correlation for diagnosing issues.
  • Security: mTLS in service mesh, WAF at API gateway, DDoS mitigation at CDN.
  • Cost optimization: CDN reduces egress costs. gRPC reduces bandwidth. Connection pooling reduces latency.