Spring Cloud Gateway — Routing, Filters and Auth in One Place

The Runtime Reality of the Gateway Pattern

When you move logic like authentication, rate limiting, and request transformation out of individual microservices and into Spring Cloud Gateway (SCG), you are creating a single point of failure. In a production environment, the gateway is the most sensitive component in your stack. If the gateway misbehaves, the entire system is effectively offline.

Spring Cloud Gateway is built on Project Reactor. This means it is non-blocking and event-loop driven. Unlike a traditional Tomcat-based Spring Boot app where a thread dump shows exactly where a request is stuck, SCG requires you to think in terms of event loops and non-blocking backpressure. If you block the Netty event loop—for example, by calling a legacy blocking database driver inside a GlobalFilter—every single concurrent request will hang.

This runbook addresses the specific failure modes encountered when managing a gateway handling 10k+ requests per second (RPS).

Symptom: Sudden 503 Service Unavailable During Peak Load

A 503 error usually implies that the gateway cannot find an available instance of the downstream service or the underlying Netty connection pool is exhausted. If your logs show Connection prematurely closed BEFORE response, you are hitting a mismatch between the gateway's idle timeout and the downstream Load Balancer's (or service's) keep-alive timeout.

The Diagnostic Command: Check the connection pool metrics via the Actuator endpoint: ``bash curl -s http://gateway-internal:8081/actuator/metrics/reactor.netty.connection.provider.fixed.active.connections | jq ``

What you should see: If VALUE equals your max-connections setting (default is often 2x CPU cores * 50), the pool is saturated.

The Fix: Tune the pendingAcquireMaxCount. By default, if the pool is full, new requests wait for a connection. If the wait queue fills up, you get a 503. Update your application.yaml:

spring:
  cloud:
    gateway:
      httpclient:
        pool:
          type: fixed
          max-connections: 500
          acquire-timeout: 2000
          pending-acquire-max-count: 1000

The Hard-Won Lesson: In our production cluster, we saw p99 latency drop from 480ms to 90ms simply by aligning the gateway's max-idle-time to be 5 seconds shorter than the AWS ALB timeout. This prevented the "race condition" where the gateway tries to reuse a connection that the ALB has already silently closed.

Symptom: 401 Unauthorized for Valid JWTs

When moving Auth to the gateway, you typically use a GatewayFilterFactory to validate tokens before proxying. A spike in 401s usually points to a failure in the Reactive ReactiveAuthenticationManager or a synchronization issue with the JWKS (Json Web Key Set) endpoint of your Identity Provider.

The Diagnostic Command: Check for WebClient timeouts in your logs when the gateway attempts to fetch the public keys from Google/Okta/Auth0: ``bash kubectl logs -l app=api-gateway | grep "TimeoutException: Did not observe any item or terminal signal within 5000ms" ``

The Implementation Pattern: Do not write a manual JWT filter if you can help it. Use the native OIDC support. Here is a production-grade configuration that handles the heavy lifting:

```java
@Configuration
@EnableWebFluxSecurity
public class SecurityConfig {

@Bean public SecurityWebFilterChain springSecurityFilterChain(ServerHttpSecurity http) { http .authorizeExchange(exchanges -> exchanges .pathMatchers("/actuator/", "/public/").permitAll() .anyExchange().authenticated() ) .oauth2ResourceServer(oauth2 -> oauth2 .jwt(Customizer.withDefaults()) ); return http.build(); } } ```

The Troubleshooting Step: If the 401s persist despite valid tokens, ensure your system clocks are synchronized. A clock skew of even 2 seconds between your gateway and the OIDC provider will cause exp (expiration) or iat (issued at) validation failures. Use ntpdate -q pool.ntp.org to check the drift on the gateway nodes.

Symptom: Gateway Pods Restarting via OOMKill

Spring Cloud Gateway is memory-efficient until it isn't. OOM (Out Of Memory) issues usually stem from one of two things: large request/response bodies being buffered in memory, or a leak in the Netty ByteBuf allocator.

The Topology of Memory Pressure: ``text [ Internet ] | [ Gateway Pod ] --- (Direct Memory / PooledByteBufAllocator) | --- (JVM Heap / Local Cache / Route Definitions) V [ Microservice ] ``

The Diagnostic Command: Compare the JVM resident set size (RSS) against the Max Heap. If RSS is significantly higher, Netty is leaking off-heap memory. ``bash # Get the process ID PID=$(ps -ef | grep java | grep -v grep | awk '{print $2}') # Check dynamic memory mapping pmap -x $PID | sort -k 3 -n -r | head -n 20 ``

The Fix: If you are using the ModifyRequestBodyGatewayFilterFactory, the gateway caches the entire body in memory to transform it. If a client uploads a 100MB file, that's 100MB of heap consumed immediately.

Disable body-heavy filters for high-traffic routes or set strict limits: ``yaml spring: servlet: multipart: max-file-size: 10MB max-request-size: 10MB ` Also, ensure you aren't logging the body of every request in a GlobalFilter. Strings are expensive; DataBufferUtils.join` can easily blow up your heap under load.

Symptom: 429 Too Many Requests (Rate Limit Thrashing)

Spring Cloud Gateway provides a RedisRateLimiter. A 429 spike typically means either a genuine dDoS/retry-storm or that your Redis instance is latent, causing the gateway to fail-closed.

The Implementation Pattern: Define a KeyResolver that uses the JWT's sub claim or the X-Forwarded-For header.

@Bean
public KeyResolver userKeyResolver() {
    return exchange -> Mono.just(
        exchange.getRequest().getHeaders().getFirst("X-User-ID") != null 
        ? exchange.getRequest().getHeaders().getFirst("X-User-ID") 
        : exchange.getRequest().getRemoteAddress().getAddress().getHostAddress()
    );
}

The Diagnostic Command: Monitor Redis latency using the redis-cli --latency command from a pod within the same namespace as the gateway. If latency is >10ms, your rate limiter will become a bottleneck.

The Expected Output: ``text min: 0, max: 12, avg: 0.18 (4250 samples) ` If avg starts climbing, or if you see RedisConnectionFailureException in the gateway logs, your rate limiting policy is effectively taking down your API. In production, always wrap the rate limiter in a Resilience4J` circuit breaker so that if Redis dies, you allow traffic through (fail-open) rather than blocking the world.

Symptom: High Latency with "Route Predicate Mismatch"

When you have hundreds of routes, the gateway must iterate through the route list for every request to find a match. This is an $O(n)$ operation.

The Diagnostic Command: Check the execution time of the RoutePredicateHandlerMapping: ``bash grep "RoutePredicateHandlerMapping" logs/gateway.log | grep "Mapping [exchange] to Route{id='some-id'}" ``

The Optimization: Order your routes by frequency of use. Routes with a higher order property are checked later. If 80% of your traffic goes to the /api/v1/orders service, that route should have order: -1.

spring:
  cloud:
    gateway:
      routes:
        - id: order-service
          uri: lb://order-service
          order: -1
          predicates:
            - Path=/api/v1/orders/**

Also, avoid complex Regex predicates. A simple Path predicate is significantly faster than a Query or Header predicate that requires expensive pattern matching on every request.

Symptom: 504 Gateway Timeout During Deployments

This occurs when the Service Discovery (Eureka/Consul/K8s) thinks a service is up, but the pod is still initializing or has just been killed. Spring Cloud Gateway's LoadBalancerClientFilter relies on a local cache of instances.

The Diagnostic Command: Check the freshness of the service instance list: ``bash curl -s http://gateway-internal:8081/actuator/gateway/routes | jq '.[].uri' ``

The Fix: Configure the LoadBalancer to use a health-check-aware approach and reduce the cache expiration.

spring:
  cloud:
    loadbalancer:
      cache:
        ttl: 2s
        capacity: 1024

For Kubernetes, ensure your readinessProbe is configured correctly. The gateway won't stop sending traffic to a terminating pod until the Kubernetes API removes the endpoint and the gateway's DiscoveryClient cache expires. This often requires a "graceful shutdown" period in the downstream microservice to handle the 2-5 second propagation delay.

The Reactive Execution Trap

The single most common cause of "unexplained" performance degradation in Spring Cloud Gateway is the use of block() inside a filter.

// DO NOT DO THIS
public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
    User user = authService.getUserDetails().block(); // This kills performance
    return chain.filter(exchange);
}

When you call .block(), you seize one of the few available Netty event loop threads. Under load, all threads will eventually be blocked waiting for I/O, and the gateway will stop accepting new connections. You must use the reactive chain:

// DO THIS
public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
    return authService.getUserDetails()
        .flatMap(user -> {
            exchange.getAttributes().put("user", user);
            return chain.filter(exchange);
        });
}

The difference is structural. In the first example, the thread sits idle. In the second, the thread is released back to the pool to handle other requests while the user details are being fetched.

Tactical Observability Requirements

To manage this in production, you cannot rely on standard application logs. You need a correlation ID that spans from the gateway to the deepest microservice.

Enable the Observability features in Spring Boot 3.x/Spring Cloud 2022.x: 1. Every request must have a traceId. 2. Use micrometer-tracing-bridge-otel to export these to Jaeger or Tempo. 3. Add a GlobalFilter that injects the traceId into the response headers.

If a customer reports a slow request, you search by traceId in your log aggregator (Loki/ELK). If the gateway says the request took 2000ms but the downstream service says it took 100ms, you know the bottleneck is in the gateway's Netty thread pool or a GlobalFilter.

The sharp takeaway: Spring Cloud Gateway is not a "configure and forget" component. It is a high-performance proxy that demands you respect the non-blocking nature of its underlying engine. Treat your gateway and its connection pools as a specialized piece of infrastructure, separate from your business-logic microservices. Alignment of timeouts, avoidance of blockages, and aggressive monitoring of Netty's off-heap memory are the three pillars of a stable gateway deployment.

What this guide consolidates

The reactive/WebFlux-specific gateway page has been absorbed into this main Spring Cloud Gateway guide. The reactive thread model, predicate ordering and back-pressure notes are now a dedicated section here instead of a second competing URL.

Spring Cloud Gateway with WebFlux — A Reactive Approach

The Thread Model Trap: Why Servlets Fail at the Edge

If you are running a traditional Spring Boot application with spring-boot-starter-web, you are likely bound to the thread-per-request model. In a standard microservice, this is often acceptable; the database bottleneck usually hits long before you exhaust the Tomcat thread pool. However, at the Gateway level, this architecture is a recipe for cascading failure.

In a non-reactive gateway, every incoming request pins a worker thread for the entire duration of the downstream proxy call. If a downstream service starts experiencing a p99 spike of 5 seconds due to a database lock or a GC pause, your gateway threads back up instantly. With a default server.tomcat.threads.max=200, you can only handle 200 concurrent "slow" requests before the gateway itself becomes unresponsive, returning 504 Gateway Timeout or, worse, TCP connection resets because the accept queue is full.

Spring Cloud Gateway (SCG) bypasses this by building on Project Reactor and WebFlux. It uses an event loop (Netty) where a handful of threads—typically matching the number of CPU cores—handle thousands of concurrent connections. The "Reactive Approach" isn't about raw speed; it's about resilience under high concurrency. We traded the simplicity of ThreadLocal variables for the scalability of non-blocking I/O.

Anatomy of a Reactive Global Filter

To understand how Spring Cloud Gateway processes a request without blocking, we must dissect a GlobalFilter. This is the core interface where you implement cross-cutting concerns like JWT validation, rate limiting, or custom logging.

The following implementation is a non-blocking RequestidTracingFilter that extracts a correlation ID or generates one, ensuring it propagates through the reactive chain without losing context.

```java
package com.platform.gateway.filters;

import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.cloud.gateway.filter.GatewayFilterChain; import org.springframework.cloud.gateway.filter.GlobalFilter; import org.springframework.core.Ordered; import org.springframework.http.server.reactive.ServerHttpRequest; import org.springframework.stereotype.Component; import org.springframework.web.server.ServerWebExchange; import reactor.core.publisher.Mono;

import java.util.UUID;

@Component public class RequestIdFilter implements GlobalFilter, Ordered {

private static final Logger log = LoggerFactory.getLogger(RequestIdFilter.class); private static final String X_REQUEST_ID = "X-Request-Id";

@Override public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) { String requestId = exchange.getRequest().getHeaders().getFirst(X_REQUEST_ID); if (requestId == null || requestId.isBlank()) { requestId = UUID.randomUUID().toString(); }

// Mutation is required because ServerHttpRequest is immutable ServerHttpRequest mutatedRequest = exchange.getRequest().mutate() .header(X_REQUEST_ID, requestId) .build();

// mutate() creates a new exchange instance with our modified request ServerWebExchange mutatedExchange = exchange.mutate() .request(mutatedRequest) .build();

return chain.filter(mutatedExchange) .then(Mono.fromRunnable(() -> { // Post-filter logic: this runs when the downstream response returns exchange.getResponse().getHeaders().add(X_REQUEST_ID, requestId); log.debug("Path: {} | Status: {}", exchange.getRequest().getPath(), exchange.getResponse().getStatusCode()); })); }

@Override public int getOrder() { return Ordered.HIGHEST_PRECEDENCE; } } ```

Dissection of the Filter Flow

1. The ServerWebExchange abstraction Unlike HttpServletRequest, which is a blocking stream reader, ServerWebExchange is a container for both the request and response in a non-blocking context. Notice that we never call a .getBody() method that returns a String. Accessing the body in WebFlux requires subscribing to a Flux<DataBuffer>, which is an asynchronous operation. If you try to read the body in a filter naively, you will "drain" the request, and the downstream service will receive an empty payload.

2. Immutability and Mutation In the Servlet world, you can call request.setAttribute(). In SCG, the request and exchange objects are immutable. To add a header, you must use .mutate(). This is a deliberate design choice to prevent side effects in a highly concurrent environment. Every time you "change" the request, you are actually creating a new pointer in the reactive pipeline.

3. chain.filter(mutatedExchange) This is the most critical line. In a blocking filter, you would call chain.doFilter() and then the code below it would execute after the downstream call finishes. In WebFlux, chain.filter() returns a Mono<Void>. It doesn't mean the request is done; it means "here is a recipe to continue the request."

4. The .then() operator The code inside .then(Mono.fromRunnable(...)) is our "post-filter." It is scheduled to run only after the downstream service has responded. This is where we inject the correlation ID back into the response headers. This avoids blocking a thread while waiting for the upstream microservice to compute a result.

The Reactive Execution Pipeline

To visualize how SCG manages this without a thread pool per request, consider the Netty EventLoopGroup.

[ External Client ] 
       |
       v
[ Netty Boss Group ] (Accepts TCP connections)
       |
       v
[ Netty Worker Group ] (p1, p2, p3... threads matching CPU cores)
       |
       +--> [ Filter 1: RequestIdFilter ] (Calculates metadata)
       |
       +--> [ Filter 2: AuthFilter ] (Verifies JWT via Reactive Redis)
       |
       +--> [ Filter 3: LoadBalancerClientFilter ] (Selects downstream IP)
       |
       +--> [ Netty Routing Filter ] (Issues non-blocking HTTP request)
              |
              | (Gateway thread is now FREE to handle other clients)
              |
[ Downstream Service Response ]
              |
       +--> [ Netty Worker Group ] (Picks up the response event)
       |
       +--> [ Filter 1: .then() block ] (Applies post-processing)
       |
       v
[ Response returned to Client ]

In this model, if your downstream service takes 200ms to respond, the Gateway thread that initiated the request is not sitting idle. It is likely processing hundreds of other filter chains for other requests in those same 200ms.

Hard-Won Lesson: The "BlockHound" Reality Check

The biggest danger in Spring Cloud Gateway is accidentally introducing blocking code into the reactive pipeline. If a junior developer adds a restTemplate.getForObject() call or a legacy JDBC query inside a GlobalFilter, they will block one of the few Netty worker threads.

If you have 8 cores and 8 Netty threads, and you block all 8 with a synchronous database call, your entire gateway stops processing all traffic.

In my previous project, we saw p99 latency jump from 40ms to 2400ms after adding a simple "fetch user profile" filter. The culprit wasn't the database speed; it was that the filter used a synchronous Feign client. This "starved" the event loop. We resolved this by switching to WebClient and saw the p99 drop back to 45ms.

To prevent this in production, I recommend using BlockHound in your unit and integration tests. It is a Java agent that hooks into the JVM's threading logic and throws an error if a blocking call (like InputStream.read() or Thread.sleep()) is detected on a thread that is supposed to be non-blocking.

// Add this to your main method or a test listener
BlockHound.install();

Dissecting the `application.yml` for Reactive Performance

Configuration in SCG isn't just about routes; it’s about tuning the underlying Netty engine and the connection pool. A standard configuration usually overlooks the HttpClient settings, leading to "Connection pool exhausted" errors under load.

spring:
  cloud:
    gateway:
      httpclient:
        connect-timeout: 2000 # ms
        response-timeout: 5s
        pool:
          type: elastic
          max-connections: 500
          acquire-timeout: 20000
      routes:
        - id: customer_service
          uri: lb://customer-service
          predicates:
            - Path=/api/customers/**
          filters:
            - name: RequestRateLimiter
              args:
                redis-rate-limiter.replenishRate: 100
                redis-rate-limiter.burstCapacity: 200

Dissection of Gateway Metadata

1. httpclient.pool.type: elastic By default, SCG uses a fixed connection pool. While "fixed" is predictable, "elastic" allows the gateway to scale connections to downstream services during traffic spikes without rejecting requests at the gateway level. For high-throughput environments, a fixed pool is actually safer to prevent overwhelming downstream services, but it requires precise capacity planning.

2. response-timeout: 5s This is a global fail-fast mechanism. In a reactive world, "hanging" connections are the silent killer. Setting a strict response timeout ensures that the Mono chain is terminated and the Netty worker thread is returned to the pool if a downstream service ghosts the gateway.

3. RequestRateLimiter with Redis The RequestRateLimiter is implemented using Lua scripts within Redis. Because it's a reactive filter, it doesn't wait for Redis to return in a blocking fashion. It uses the ReactiveRedisTemplate, allowing the gateway to process other requests while the Redis RTT (Round Trip Time) is in progress.

4. lb:// Scheme This leverages Spring Cloud LoadBalancer. Instead of a hardcoded IP, it uses service discovery (Eureka, Consul, or k8s). The reactive LoadBalancer implementation is crucial here; it retrieves service instances without blocking the main request thread, often caching them in a local Flux that is refreshed periodically.

The Reactive Takeaway

Transitioning to Spring Cloud Gateway and WebFlux requires a mental shift from "imperative execution" to "declarative composition." You aren't writing a list of steps; you are building a pipeline of transformations.

The sharpest lesson is this: The performance of your gateway is inversely proportional to the amount of data you hold in memory. Reactive streams are designed to move data as chunks (DataBuffer). If you find yourself converting Flux<DataBuffer> into a String just to log the request body, you are killing the very performance benefits you sought. Keep your filters focused on metadata (headers and cookies) and let the body stream through untouched to the downstream service. This approach allows SCG to handle 10k+ concurrent requests on a single modest container, a feat a Servlet-based gateway could never achieve without thousands of idle threads.