API Gateway Patterns

An API gateway is a single entry point that sits in front of a set of backend services (a monolith split into modules, or a microservices fleet) and handles cross-cutting concerns so individual services don't have to: routing, authentication, rate limiting, request/response transformation, and observability. Clients talk to one host and one contract; the gateway figures out where each request actually goes.

Common implementations range from managed cloud services (AWS API Gateway, Azure API Management, Google Cloud Endpoints) to self-hosted proxies purpose-built for the job (Kong, Tyk, KrakenD) to general-purpose reverse proxies configured as a gateway (Nginx, Envoy, Traefik). The concepts below apply across all of them even though the config syntax differs.

Why put a gateway in front of your services

Without a gateway, every backend service reimplements auth, rate limiting, CORS, TLS termination, and logging — or worse, some services skip it. Every client also needs to know the address of every service it talks to, which breaks the moment you split a service in two or move one behind a new domain. A gateway decouples the client-facing API surface from the internal service topology: services can be renamed, merged, split, or moved without touching client code, as long as the gateway's routes stay stable.

The trade-off is that the gateway becomes a single point of failure and a potential bottleneck if it isn't built for the traffic. Treat it as critical infrastructure: run it stateless and horizontally scaled, keep its own logic thin, and push business logic back into the services it fronts.

Routing and request aggregation

At its core, a gateway maps an incoming path (and sometimes method, host, or header) to an upstream service. Kong's declarative config is a good illustration of the shape most gateways share — services, routes, and plugins attached to either:

_format_version: "3.0"

services:
  - name: users-service
    url: http://users-svc.internal:3000
    routes:
      - name: users-route
        paths:
          - /api/users
        strip_path: false

  - name: orders-service
    url: http://orders-svc.internal:3001
    routes:
      - name: orders-route
        paths:
          - /api/orders
        strip_path: false

plugins:
  - name: rate-limiting
    service: users-service
    config:
      minute: 100
      policy: local
  - name: cors
    config:
      origins:
        - https://app.example.com
      methods:
        - GET
        - POST
        - PATCH
        - DELETE

A more advanced pattern is request aggregation (a.k.a. the backend-for-frontend pattern taken to the edge): the gateway fans a single client request out to multiple upstream services and stitches the responses together, so a mobile client gets one round trip instead of five. Here's a minimal aggregation gateway built on Node.js/Fastify, useful when you want this logic in code rather than a managed product:

import Fastify from 'fastify'

const app = Fastify({ logger: true })

app.get('/api/dashboard/:userId', async (request, reply) => {
  const { userId } = request.params

  const [profile, orders, notifications] = await Promise.all([
    fetch(`http://users-svc.internal:3000/users/${userId}`).then(r => r.json()),
    fetch(`http://orders-svc.internal:3001/orders?userId=${userId}&limit=5`).then(r => r.json()),
    fetch(`http://notify-svc.internal:3002/notifications?userId=${userId}&unread=true`).then(r => r.json()),
  ])

  return reply.send({ profile, recentOrders: orders, unreadNotifications: notifications })
})

app.listen({ port: 8080 })

Use Promise.all rather than sequential awaits so the fan-out requests run concurrently — the aggregation gateway's whole value proposition collapses if it just serializes the same round trips the client would have made anyway.

Authentication and authorization at the edge

A gateway is the natural place to terminate authentication: validate the token once, reject bad requests before they ever reach a backend service, and forward a trusted, already-verified identity downstream. This keeps every backend service from having to re-implement JWT verification and lets you rotate signing keys or swap identity providers in one place.

import jwt from 'jsonwebtoken'

async function authenticate(request, reply) {
  const header = request.headers.authorization
  if (!header?.startsWith('Bearer ')) {
    return reply.code(401).send({ error: 'Missing bearer token' })
  }

  try {
    const token = header.slice('Bearer '.length)
    const payload = jwt.verify(token, process.env.JWT_PUBLIC_KEY, { algorithms: ['RS256'] })

    // Forward a trusted identity downstream instead of the raw token —
    // backend services trust the gateway's internal network, not the client.
    request.headers['x-user-id'] = payload.sub
    request.headers['x-user-roles'] = payload.roles.join(',')
  } catch (err) {
    return reply.code(401).send({ error: 'Invalid or expired token' })
  }
}

app.addHook('onRequest', async (request, reply) => {
  if (request.url.startsWith('/api/public')) return
  await authenticate(request, reply)
})

Coarse-grained authorization ("is this role allowed to call this route at all") also belongs at the gateway. Fine-grained, resource-level authorization ("can this specific user edit this specific order") almost always has to stay in the owning service, because only that service has the data needed to make the call.

Rate limiting, throttling, and resilience

Rate limiting at the gateway protects backend services from being overwhelmed by a single noisy client, and protects the platform as a whole from abusive or runaway traffic. It's usually applied per API key, per user, or per IP, with different tiers for different plans.

import rateLimit from '@fastify/rate-limit'

app.register(rateLimit, {
  max: (request) => (request.headers['x-api-plan'] === 'pro' ? 1000 : 100),
  timeWindow: '1 minute',
  keyGenerator: (request) => request.headers['x-api-key'] || request.ip,
  errorResponseBuilder: () => ({
    error: 'rate_limit_exceeded',
    message: 'Too many requests, slow down and retry after the window resets.',
  }),
})

Beyond throttling, gateways commonly implement circuit breaking: if a backend service starts timing out or erroring above a threshold, the gateway stops sending it traffic for a cooldown period and fails fast instead, protecting the struggling service from a pile-up of retries and giving it room to recover. Envoy and Kong both ship this as a built-in policy; a hand-rolled gateway typically wraps upstream calls with a library like `opossum` (Node.js) to get the same behavior.

Observability and gotchas

Because every request passes through it, the gateway is the best place to emit consistent, structured access logs, propagate a correlation/trace ID to every downstream call, and export latency and error-rate metrics per route. When something is slow, "which upstream and which route" should be answerable from gateway metrics alone, before anyone has to dig into individual service logs.

• Generate (or forward) a correlation/trace ID at the gateway and pass it to every upstream call — without it, debugging a slow request across five services is guesswork.

• Keep the gateway stateless. Rate-limit counters and session data belong in shared storage (Redis), not in-process memory — otherwise scaling the gateway horizontally silently breaks your limits.

• Don't let the gateway become a place where business logic accumulates. Aggregation and auth are fine; "if user.plan === 'pro' then apply this discount" is not — that belongs in a service.

• Set aggressive but sane upstream timeouts. A gateway with no timeout on a hung backend call will exhaust its own connection pool and take down routes that have nothing to do with the failing service.

• Version your routes explicitly (/v1/orders, /v2/orders) rather than mutating a live contract — the gateway is exactly where you can run two versions side by side while clients migrate.

• Test failure modes, not just the happy path: what does a client see when an upstream is down, when auth fails, when the rate limit trips? Each should return a clear, consistent error shape.