The API gateway

Clients should not know how many services there are. A mobile app that calls eleven hosts, handles eleven certificates and eleven authentication schemes is a client that breaks every time a service is split. The gateway is the one door: it routes, it authenticates at the edge, it limits and it observes, and then it gets out of the way. Every gateway that has grown into something more has become the enterprise service bus the industry spent a decade escaping. This lesson is what a gateway does, what Spring Cloud Gateway looks like, the backend-for-frontend variant, and the anti-patterns to refuse.

#What a gateway does

Everything on this list is generic: it knows routes and tokens and limits, not orders or payments. That is the test for whether a piece of logic belongs at the gateway.

#Spring Cloud Gateway

Spring Cloud Gateway is a routing engine on the reactive stack (and, since 4.1, a servlet variant on Spring MVC for teams that do not want WebFlux). A route is a predicate, a target and filters:

spring:
  cloud:
    gateway:
      routes:
        - id: orders
          uri: http://orders.shop.svc.cluster.local:8080
          predicates: [ Path=/api/orders/** ]
          filters:
            - StripPrefix=1                       # /api/orders/42 → /orders/42
            - name: CircuitBreaker
              args: { name: orders, fallbackUri: forward:/unavailable }
            - name: Retry
              args: { retries: 1, methods: GET, statuses: BAD_GATEWAY }
            - name: RequestRateLimiter
              args: { redis-rate-limiter.replenishRate: 50, redis-rate-limiter.burstCapacity: 100 }
        - id: catalogue
          uri: http://catalogue.shop.svc.cluster.local:8080
          predicates: [ Path=/api/catalogue/**, Method=GET ]
          filters: [ StripPrefix=1 ]
      httpclient:
        connect-timeout: 2000
        response-timeout: 5s

Predicates match on path, method, header, host, query and time; filters rewrite paths and headers, add the request id, apply the circuit breaker (Resilience4j), retry, limit (Redis-backed so the limit is shared across gateway instances), and forward. Global filters apply to every route: the one that reads the Authorization header and validates the JWT lives there, with Spring Security's resource-server support on the gateway itself. The gateway is stateless, so it runs as several replicas behind the load balancer and is not a single point of failure unless you make it one.

The alternatives are the ingress controllers and managed gateways (NGINX, Envoy Gateway, Kong, the cloud providers' API gateways). They do the same list with less Java; Spring Cloud Gateway is the choice when the team is a Spring team and the edge needs custom filters written in the language they know.

#Authentication at the edge

The gateway validates the token once: signature against the identity provider's keys, expiry, audience. A request without a valid token never reaches a service. What it then forwards is the decision to be made deliberately:

• Token relay. Forward the original JWT; each service validates it again (cheap, a cached key) and reads the claims it needs. Services remain safe to call without the gateway, which matters for internal callers and for testing.
• Trusted headers. Strip the token and forward X-User-Id and X-Roles. Simpler for services, and it means any process that can reach a service can impersonate anyone by setting a header, so it requires that the network guarantees only the gateway can reach the services (a mesh with mTLS, or network policies). Without that guarantee, it is an open door.

Token relay is the default. Authorisation (may this user cancel this order?) stays in the service that owns the order, because it needs the order.

#The backend-for-frontend

One gateway serving a web app, an iOS app and a partner API tends to serve each badly: the mobile app wants one call that returns a whole screen, the web app wants fine-grained resources, the partner wants stability and versioning. The backend-for-frontend pattern gives each client type its own thin gateway, owned by the team that owns that client, which composes the calls that client needs:

mobile app  →  mobile-bff  ─┐
web app     →  web-bff     ─┼→  orders, catalogue, customers, ...
partners    →  partner-api ─┘

A BFF may aggregate: call three services and shape one response for a screen. That is the one place where "aggregation at the gateway" is legitimate, and it is legitimate because the BFF belongs to the client's team and changes with the client's screens, not with the domain. The BFF still contains no business rules: it composes, it does not decide.

#Anti-patterns

• Business logic. A gateway that checks whether an order is cancellable, applies a discount, or validates a payload against domain rules has become a service with a routing table, owned by nobody, deployed by everybody. Rules live in the service that owns the data.
• Orchestration. A gateway that calls the inventory service, then the payment service, then the order service in sequence to fulfil one request is a saga running at the edge with no state, no compensation and no retries. Orchestration belongs in a service (or a workflow engine) that can own its failures.
• A shared database. A gateway that reads a table to make a routing or authorisation decision has coupled the edge to a service's schema.
• Transformation of payloads. Translating between service formats at the gateway is the ESB; each consumer ends up depending on the gateway's translation, and the translation depends on every service's format.
• The only entry point for internal calls. Service-to-service calls go directly (or through the mesh), not out through the gateway and back in; the gateway is for clients.
• One gateway instance. It is on every path; it runs as replicas or it is the outage.

The rule behind all of them: the gateway handles concerns that are the same for every route. The moment a filter needs to know what an order is, the logic is in the wrong process.

#Under the hood: a request through Spring Cloud Gateway

The gateway is a filter chain over a non-blocking HTTP server. A request arrives on Netty (reactive) or Tomcat (the MVC variant) and enters DispatcherHandler, which hands it to the RoutePredicateHandlerMapping: every route's predicates are evaluated in order against the request, and the first route whose predicates all match is put into the exchange's attributes. Then the FilteringWebHandler builds a chain from the global filters (sorted by @Order) merged with the matched route's gateway filters, and runs the request down it. Each filter is filter(exchange, chain): it may modify the request, short-circuit with a response (the rate limiter's 429, the auth filter's 401), or call chain.filter(exchange) and add "post" behaviour on the returned Mono. The last filter in the chain, NettyRoutingFilter (or the MVC ProxyExchange), opens a connection from the gateway's pooled HttpClient to the route's uri, streams the request body through without buffering, and streams the response back; a filter that needs to read the body (ModifyRequestBody) forces buffering, which is the one place a gateway's memory scales with payload size. Response headers pass back up the chain, so a filter that adds X-Request-Id on the way in can add it to the response on the way out.

Two things decide the gateway's capacity. It is non-blocking: one event-loop thread per core handles thousands of in-flight requests, because a request waiting on an upstream holds no thread, only a small object and a socket. So a blocking call inside a custom filter (a JDBC lookup, a synchronous RestTemplate) stalls an event loop and takes a quarter of the gateway's throughput with it, which is why the token validation uses a cached JWK set and the rate limiter talks to Redis reactively. And the upstream connection pool (spring.cloud.gateway.httpclient.pool.*) is per route target; a slow upstream holds connections rather than threads, but a pool of 500 with a 30 s response-timeout is still 500 requests parked, and the circuit breaker filter is what keeps that from filling on a dead upstream. Timeouts apply per hop: connect-timeout and response-timeout on the client, Retry's attempts inside them, and the client-facing timeout is whatever the load balancer in front of the gateway enforces.

The JWT filter is where auth cost lives: on first use it fetches the identity provider's JWK set (spring.security.oauth2.resourceserver.jwt.jwk-set-uri) and caches it; validation is then a signature check (an RSA verify, ~100 µs), an expiry check, and audience/issuer matching, with no network call. A key rotation at the provider publishes a new kid; the filter's cache misses on the unknown key id and refetches. Token relay is passing the same Authorization header downstream; the "trusted headers" alternative strips it and adds identity headers, and the security of that rests entirely on the upstream refusing any request that did not come through the gateway (a NetworkPolicy, or mesh authorization policies keyed on the gateway's identity).

#Walkthrough: the gateway that became the deployment bottleneck

Three years in, a platform team's gateway had 61 custom filters and shipped twice a week with every product change queued behind it.

public class CancellableOrderFilter implements GatewayFilter {
    public Mono<Void> filter(ServerWebExchange ex, GatewayFilterChain chain) {
        var id = ex.getRequest().getPath().pathWithinApplication().value().replace("/api/orders/", "");
        var order = jdbc.queryForObject("SELECT status FROM orders WHERE id = ?", String.class, id);   // blocking, another team's table
        if (!"PLACED".equals(order)) return respond(ex, 409, "order not cancellable");
        return chain.filter(ex);
    }
}

1. The filter read the orders database from the gateway to save the orders team a round trip. It was blocking JDBC on an event loop; under load the gateway's p99 climbed for every route, including ones that never touched orders, and the filter was found only by thread-dumping the gateway.
2. It also coupled the gateway's deploy to the orders schema: a column rename in orders broke checkout for everyone, because the gateway, not the orders service, threw. The orders team could not ship the rename without the platform team's release.
3. The audit found four more filters reading other teams' tables, nine encoding business rules (discount eligibility, cancellation windows, tier checks), and three doing sequential calls to two services to "pre-validate". The gateway had 40% of the system's business logic and no tests for any of it, because gateway filters are hard to test and nobody had.
4. The unwinding took two quarters: each rule moved into the service that owned the data (the 409 above became the orders service's own response), the sequential pre-validations became a BFF owned by the mobile team, and the gateway was reduced to routing, auth, limits, tracing and per-route resilience. Its deploy frequency dropped to monthly; product teams shipped on their own cadence.
5. The rule the platform team wrote into the gateway repository's CONTRIBUTING.md: a filter may not import a domain type, call a database, or call more than the route's own upstream. Pull requests that violate it are closed with a link to the incident.

A gateway is on every request path and owned by one team; every line added to it is a line every other team waits on. Generic concerns only, or the gateway is the monolith with a routing table.

#Try it yourself

#Misconceptions

• "The gateway is a reverse proxy with extra steps." It is a filter chain on a non-blocking runtime; a blocking call in one filter degrades every route.
• "Validating the JWT at the gateway means services need not." With token relay they validate again, cheaply, and stay safe for internal callers; with trusted headers, the network must guarantee only the gateway can reach them.
• "One aggregation at the gateway is harmless." Aggregation belongs in a BFF owned by the client's team. In the shared gateway it is the first of forty rules.
• "The gateway's timeout is the client's timeout." Each hop has its own; the load balancer in front of the gateway decides when the client gives up, and the gateway's total must fit inside it.
• "Several gateway replicas make it resilient." Replicas remove the single instance; a shared Redis limiter, a shared JWK cache and per-route breakers are what keep a bad upstream from taking all replicas down together.

#Going deeper

• Spring Cloud Gateway reference: "How It Works", "Route Predicate Factories", "GatewayFilter Factories", "Global Filters", and the httpclient configuration.
• Spring Security, "OAuth 2.0 Resource Server → JWT", for the JWK-set cache and validation.
• Sam Newman, "Backends For Frontends" (samnewman.io), the pattern's origin.
• Envoy Gateway and Kong documentation, to compare what a non-Java gateway provides out of the box.
• Kubernetes "Network Policies" and Istio "Authorization Policy", the two ways to make trusted headers actually trusted.