Deploy FastAPI Applications to Kubernetes

The Ceiling of Single-Node Docker Compose

Moving FastAPI from a single-node VM running Docker Compose to Kubernetes isn't about following the hype; it’s about solving the "Gunicorn stall." In our legacy setup, we ran a standard Gunicorn/Uvicorn worker model on an AWS m5.large. When traffic spiked during our 9:00 AM API polling window, the single-instance scheduler would choke. We had no way to scale horizontally without manually provisioning an identical VM and hacking together an Nginx load balancer.

If your backend is still living in a docker-compose.yml on a standalone EC2 instance, you are managing state you shouldn't care about. The migration to Kubernetes transitions the responsibility of "liveness" from a bash script or a systemd unit to the Kubelet.

The first phase of this migration isn't about YAML; it's about the Dockerfile. Most FastAPI developers bloat their images with build-time dependencies like gcc or libpq-dev. In a K8s environment, image pull speed directly impacts your horizontal pod autoscaler (HPA) reaction time. We moved to a multi-stage build, shrinking our image from 840MB to 110MB.

```dockerfile
# Stage 1: Build dependencies
FROM python:3.11-slim-bookworm AS builder
ENV PYTHONUNBUFFERED=1 \
    PYTHONDONTWRITEBYTECODE=1
WORKDIR /build
RUN apt-get update && apt-get install -y --no-install-recommends gcc python3-dev
COPY requirements.txt .
RUN pip install --user --no-cache-dir -r requirements.txt

# Stage 2: Runtime FROM python:3.11-slim-bookworm WORKDIR /app COPY --from=builder /root/.local /root/.local COPY ./app /app/app ENV PATH=/root/.local/bin:$PATH # Use tini for signal forwarding; vital for K8s SIGTERM handling RUN apt-get update && apt-get install -y tini && rm -rf /var/lib/apt/lists/* ENTRYPOINT ["/usr/bin/tini", "--"] CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000", "--workers", "1"] ```

Rollback Criteria: If the image build exceeds 4 minutes or the transition to tini causes existing background tasks in Python to zombie, we revert the entrypoint. Note the --workers 1 setting. In Kubernetes, you scale by adding Pods, not by adding Gunicorn workers within a Pod. Vertical scaling inside a Pod hides resource consumption from the K8s scheduler.

Explicit Resource Scheduling and the OOMKill Trap

When we first pushed FastAPI to the cluster, we didn't set resource limits. The result was a "noisy neighbor" effect where a memory-intensive PDF generation endpoint ate all the RAM on a worker node, causing the Kubelet to kill critical system pods.

FastAPI is highly predictable with memory but spikey with CPU during Pydantic serialization. You must define requests and limits in your Deployment manifest.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: fastapi-api
  labels:
    app: fastapi
spec:
  replicas: 3
  selector:
    matchLabels:
      app: fastapi
  template:
    metadata:
      labels:
        app: fastapi
    spec:
      containers:
      - name: api
        image: registry.digitalocean.com/my-org/fastapi-app:v1.2.4
        ports:
        - containerPort: 8000
        resources:
          requests:
            cpu: "250m"
            memory: "512Mi"
          limits:
            cpu: "1000m"
            memory: "1Gi"
        livenessProbe:
          httpGet:
            path: /health
            port: 8000
          initialDelaySeconds: 5
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /ready
            port: 8000
          initialDelaySeconds: 5
          periodSeconds: 5

A hard-won lesson: Never set your CPU limit too low. If FastAPI hits its CPU limit, the throat-clearing of the CFS quota results in massive latency spikes (p99 going from 40ms to 800ms) without the pod actually dying. Keep CPU limits generous or omit them if you trust your node-level isolation, but always set strict memory limits to prevent runaway leaks.

Rollback Criteria: If kubectl get pods shows CrashLoopBackOff with an OOMKilled status, increase the memory limit by 2x. If the P99 latency increases by more than 25% post-migration, increase the CPU request to ensure the scheduler places the pod on a less-contended node.

Decoupling Environment Logic with ConfigMaps and Secrets

In our legacy setup, we used a .env file that was manually copied to the server via SCP. This is a security nightmare and an operational bottleneck. In Kubernetes, we utilize ConfigMap for non-sensitive data (like LOG_LEVEL or SENTRY_DSN) and Secret for sensitive data (like DATABASE_URL).

Do not inject these as a file. Inject them as environment variables so FastAPI's pydantic-settings can read them natively.

Cluster Topology:
[ Ingress-NGINX ] -> [ Service: ClusterIP ] -> [ Pods (FastAPI) ]
                                                   |
                                          [ ConfigMap / Secret ]
                                                   |
                                          [ Database (External) ]

When managing secrets, we adopted the "Immutable Secret" pattern. If you change a database password, you don't update the Secret in place; you create a new one and update the Deployment's envFrom field. This triggers a rolling update, ensuring all pods are using the new credentials simultaneously.

Rollback Criteria: If the application fails to start because of a ValidationError from Pydantic, the ConfigMap is likely missing a required key. Revert the Deployment to the previous revision using kubectl rollout undo deployment/fastapi-api.

The Ingress and Load Balancing Tier

FastAPI won't see the real client IP address by default in Kubernetes because of the Proxy Protocol and the multiple hops (Ingress -> Service -> Pod). You need to configure your Ingress controller to pass the X-Forwarded-For headers and set your FastAPI app to trust these proxies.

We utilize the NGINX Ingress Controller. The crucial configuration here is ensuring the service is of type ClusterIP, not LoadBalancer. You only want one entry point to your cluster.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: fastapi-ingress
  annotations:
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    nginx.ingress.kubernetes.io/proxy-body-size: "10m"
spec:
  ingressClassName: nginx
  rules:
  - host: api.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: fastapi-service
            port:
              number: 80

Rollback Criteria: If you encounter 502 Bad Gateway errors, verify that the service selector matches the pod labels exactly. Run kubectl get endpoints fastapi-service to ensure there are active IPs behind the service.

Reactive Scaling via Horizontal Pod Autoscaler (HPA)

The final phase of the migration was automating the response to traffic. We replaced our "hope-it-doesn't-crash" strategy with a metrics-driven HPA. We initially tried scaling based on CPU, but FastAPI is often I/O bound (waiting on DB queries), meaning CPU stays low while request queues build up.

We moved to scaling based on target concurrency per pod using the Prometheus Adapter, but for most, a safe starting point is a combination of 70% CPU and 80% Memory utilization.

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: fastapi-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: fastapi-api
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70

One specific, hard-won lesson: when we first enabled HPA, we saw "flapping"—the cluster would add a pod, the load would drop, the cluster would immediately remove the pod, and the load would spike again. We tuned the behavior field in the HPA spec to include a stabilizationWindowSeconds of 300 seconds for scale-down events. This prevents the cluster from being too aggressive in killing pods.

Rollback Criteria: If the pod count reaches maxReplicas and latency is still high, check for database connection pool exhaustion. Scaling the API does no good if the Postgres instance is at its max_connections limit.

Zero-Downtime Rollouts and Signal Handling

The most dangerous moment in the migration was the transition to rolling updates. If your FastAPI application doesn't handle SIGTERM correctly, Kubernetes will wait for the terminationGracePeriodSeconds (default 30s) and then SIGKILL your process, cutting off active database transactions and HTTP requests.

By using tini in our Dockerfile and ensuring Uvicorn is not running behind a shell (i.e., use CMD ["uvicorn", ...] not CMD uvicorn ...), signals are passed correctly.

We observed that our p99 dropped from 480ms during deployments to 90ms once we implemented proper readiness probes and graceful shutdowns. Without these, the Ingress controller would continue sending traffic to pods that were already in the "Terminating" state, leading to 502 errors for clients.

The sharp takeaway for this migration: Kubernetes is not a "fire and forget" platform for FastAPI. The magic is in the contract between the Readiness probe and the Ingress controller. If your application reports it is ready before the DB connection pool is initialized, you will drop traffic. If you don't handle SIGTERM, you will corrupt state. Moving to K8s is a shift from managing servers to managing the lifecycle of a process. Once that lifecycle is automated, the "Gunicorn stall" becomes a relic of your legacy infrastructure.