Scrolls
7 min read
On this page

API Gateways

An API gateway is a reverse proxy that sits between your clients and your backend services. It handles cross-cutting concerns — authentication, rate limiting, routing, logging, and request transformation — so your services do not have to.

Without a gateway, every service implements its own authentication, its own rate limiting, and its own logging. This duplication creates inconsistency, security gaps, and maintenance burden. With a gateway, these concerns are centralized in one place.

What an API Gateway Does

Request Routing

The gateway routes incoming requests to the appropriate backend service based on the URL path, headers, or other request attributes.

Client request: GET /v1/users/123
Gateway routes to: user-service:8080/users/123

Client request: GET /v1/orders/456
Gateway routes to: order-service:8080/orders/456

Client request: POST /v1/payments
Gateway routes to: payment-service:8080/payments

The client sees a single domain (api.example.com). The gateway translates that into requests to the correct internal service. Clients never know about your service topology.

Authentication & Authorization

The gateway validates authentication tokens before requests reach your services. This means your services can trust that every request they receive is already authenticated.

Client → Gateway:
  GET /v1/users/123
  Authorization: Bearer eyJhbGciOiJSUzI1NiIs...

Gateway:
  1. Extract JWT from Authorization header
  2. Verify signature against public key
  3. Check token expiration
  4. Extract user_id and scopes
  5. Forward request with X-User-ID: user_42 header

Gateway → User Service:
  GET /users/123
  X-User-ID: user_42
  X-Scopes: read:users,write:users

The user service receives a verified user ID and scopes. It does not need JWT validation logic, key management, or token parsing.

Rate Limiting

The gateway enforces rate limits per API key, per IP address, or per authenticated user. All services benefit from the same rate limiting logic.

API Key: sk_live_abc123
  Limit: 100 requests per minute
  Current: 98 requests
  
Request 99: 200 OK (allowed)
Request 100: 200 OK (allowed)
Request 101: 429 Too Many Requests
  Retry-After: 12

Rate limiting at the gateway protects all backend services simultaneously. Without it, each service needs its own rate limiter, and a burst of traffic to one service can overwhelm it while others sit idle.

Logging & Monitoring

The gateway logs every request and response, providing a single source of truth for API traffic.

{
  "request_id": "req_abc123",
  "method": "GET",
  "path": "/v1/users/123",
  "status": 200,
  "latency_ms": 45,
  "api_key": "sk_live_abc123",
  "user_agent": "MyApp/1.0",
  "timestamp": "2025-01-15T10:30:00Z"
}

This centralized logging gives you:

  • Request volume by endpoint
  • Latency percentiles (p50, p95, p99)
  • Error rates by status code
  • Usage patterns by API key
  • Geographic distribution of requests

Request & Response Transformation

The gateway can modify requests and responses in flight:

Client sends:
  GET /v1/users/123
  Accept: application/xml

Gateway transforms to:
  GET /users/123
  Accept: application/json

Service responds:
  {"id": "user_123", "name": "Jane"}

Gateway transforms to:
  <user><id>user_123</id><name>Jane</name></user>

This is useful when clients expect a different format than your services produce, or when you need to strip internal fields from responses before they reach external clients.

Gateway Products

Kong

Open-source API gateway built on Nginx. Extensible through plugins for authentication, rate limiting, logging, and transformation. Runs as a standalone proxy or on Kubernetes via the Kong Ingress Controller.

Kong configuration:
  Service: user-service
    Route: /v1/users/*
    Plugins:
      - key-auth (authentication)
      - rate-limiting (100 req/min)
      - response-transformer (strip internal fields)

Kong's plugin ecosystem is its strength. You add capabilities by enabling plugins, not writing code. Custom plugins are written in Lua or Go.

AWS API Gateway

Fully managed gateway service from AWS. Integrates natively with Lambda, ECS, and other AWS services. Handles provisioning, scaling, and patching automatically.

AWS API Gateway:
  REST API: my-api
    Resource: /users/{id}
      Method: GET
        Integration: Lambda (get-user-function)
        Authorizer: Cognito User Pool
        Throttle: 1000 req/sec

The trade-off is vendor lock-in. Your gateway configuration is expressed in AWS-specific terms and does not port to other platforms easily.

Cloudflare Workers

Edge computing platform that can function as an API gateway. Runs your routing, authentication, and transformation logic at Cloudflare's edge locations worldwide.

Cloudflare Worker:
  Route: api.example.com/v1/*
  Logic:
    1. Validate API key
    2. Check rate limit (KV store)
    3. Route to origin server
    4. Cache GET responses (60s TTL)

The advantage is latency. Authentication and rate limiting happen at the edge, milliseconds from the client, not at your origin server. The trade-off is a different programming model (V8 isolates, not traditional servers).

Envoy & Istio

Envoy is a high-performance proxy designed for cloud-native architectures. Istio builds a service mesh on top of Envoy, providing gateway functionality along with service-to-service communication management.

Envoy configuration:
  Listener: 0.0.0.0:443
    Route: /v1/users/* → user-service cluster
    Route: /v1/orders/* → order-service cluster
    Filters:
      - JWT authentication
      - Rate limiting (external service)
      - Access logging

Envoy is the right choice when you need fine-grained control over traffic management, circuit breaking, and observability across a microservices architecture.

When You Need a Gateway

A gateway is justified when you have:

Multiple backend services — if clients need to reach three or more services, a gateway simplifies the client by providing a single entry point.

Consistent auth across services — if every service needs the same authentication logic, centralize it in the gateway instead of duplicating it.

Traffic management — rate limiting, throttling, and circuit breaking are better handled at the gateway level, where you have visibility into total traffic across all services.

External API consumers — if third-party developers integrate with your API, a gateway provides the metering, access control, and documentation infrastructure they need.

Protocol translation — if your internal services use gRPC but external clients use REST, the gateway can translate between protocols.

When You Do Not Need a Gateway

Single service — if you have one backend service, a gateway adds complexity without value. Put authentication and rate limiting in the service itself.

Simple deployment — if your API is a monolith behind a load balancer, the load balancer handles routing and the application handles everything else. A gateway is unnecessary.

Internal-only APIs — if all consumers are internal services in the same network, the overhead of a gateway may not be justified. Service mesh solutions (like Istio) might be more appropriate.

Early-stage startup — if you have two developers and one service, invest your time in the product, not the infrastructure. Add a gateway when you have multiple services and external consumers.

The Gateway as a Single Point of Failure

The gateway sits in the critical path of every API request. If it goes down, every service behind it becomes unreachable. This makes the gateway your most critical infrastructure component.

Mitigation strategies:

Redundancy — run multiple gateway instances behind a load balancer. If one instance fails, traffic routes to the others.

DNS → Load Balancer → Gateway Instance 1
                    → Gateway Instance 2
                    → Gateway Instance 3

Health checks — configure the load balancer to check gateway health every few seconds and remove unhealthy instances automatically.

Circuit breaking — if a backend service is slow or failing, the gateway should circuit-break requests to that service rather than queuing them and consuming resources.

Graceful degradation — if the rate limiting service is down, the gateway should allow requests through (fail open) rather than rejecting all traffic (fail closed). This is a deliberate trade-off: temporary loss of rate limiting is better than total API outage.

Stateless design — gateway instances should be stateless so they can be replaced instantly. Store rate limit counters in Redis, not in gateway memory.

Gateway Anti-Patterns

The Smart Gateway

Putting business logic in the gateway. The gateway should route, authenticate, and rate-limit. It should not validate business rules, transform data models, or orchestrate multi-service workflows.

Bad:
  Gateway receives POST /v1/orders
  Gateway calls user-service to validate user
  Gateway calls inventory-service to check stock
  Gateway calls payment-service to charge card
  Gateway calls order-service to create order
  Gateway assembles response from all four calls

This turns the gateway into an orchestration layer. When the business logic changes, you modify the gateway instead of the services. The gateway becomes a monolith that happens to call microservices.

The Configuration Sprawl

Gateway configuration growing to thousands of lines with service-specific rules, transformations, and exceptions. Each team adds their own rules, and no one understands the full configuration.

Keep gateway configuration simple: routing rules, auth policies, and rate limits. Push everything else to the services.

Common Pitfalls

  • Over-engineering early — adding a gateway when you have one service and two developers. Start without a gateway and add one when you have multiple services.
  • Business logic in the gateway — the gateway handles cross-cutting concerns, not business rules. Keep it thin.
  • Single instance — running one gateway instance without redundancy. The gateway is a single point of failure by nature; make it redundant by design.
  • Ignoring latency — every request through the gateway adds latency for auth validation, rate limit checks, and logging. Measure this overhead and keep it under 10ms.
  • No monitoring of the gateway itself — monitoring backend services but not the gateway. Monitor gateway latency, error rates, and resource usage with the same rigor as any service.
  • Mixing internal and external traffic — routing both internal service-to-service calls and external API calls through the same gateway. Use separate gateways or routes for different traffic types.

Key Takeaways

  • An API gateway centralizes cross-cutting concerns: routing, authentication, rate limiting, logging, and transformation. This eliminates duplication across backend services.
  • You need a gateway when you have multiple backend services, external API consumers, or traffic management requirements. You do not need one for a single service or an early-stage product.
  • Kong (open-source, plugin-based), AWS API Gateway (managed, AWS-native), Cloudflare Workers (edge computing), and Envoy (cloud-native, high-performance) are the main options. Choose based on your infrastructure and vendor preferences.
  • The gateway is a single point of failure. Run multiple instances behind a load balancer, implement health checks, and design for graceful degradation.
  • Keep the gateway thin. It handles routing, auth, and rate limiting. Business logic belongs in the services, not the gateway.