FastAPI Microservices Architecture Explained Step by Step

Day 0: Retiring the Monolith

The project started as a standard FastAPI SQLAlchemy monolith. It was clean, typed, and fast—until the background tasks started killing the API responsiveness. We were processing heavy image resizing and complex PDF generation within the same process space as the user-facing endpoints. When Pydantic hit a validation bottleneck on a 10MB JSON payload, or the PDF engine pinned a core for 4 seconds, the event loop starved.

The decision to move to microservices wasn't driven by "scaling teams," but by resource isolation. We needed to separate the I/O-bound API traffic from the CPU-bound processing tasks. We settled on a four-service skeleton: gateway-api, user-service, order-service, and worker-node, all communicating over a shared Redis backbone and distinct Postgres schemas.

[ Client ] -> [ API Gateway (Traefik/FastAPI) ]
                   |
        +----------+----------+
        |                     |
[ User Service ]      [ Order Service ]
(Postgres 1)          (Postgres 2)
        |                     |
        +----------+----------+
                   |
           [ Redis Streams ]
                   |
           [ Worker Service ]

Day 3: Defining the Contract and Shared Models

One of the first traps we hit was the "Shared Library" anti-pattern. Engineers love DRY (Don't Repeat Yourself), so the instinct was to create a common repo containing all Pydantic models. This is a mistake. It couples your services; a change in the Order model in the common repo forces a redeploy of every service, defeating the purpose of independent scaling.

We shifted to a "Contract First" approach. We used msgpack for internal service-to-service communication to save on serialization overhead, but kept external APIs as standard JSON. For internal consistency, we used a thin CLI tool to generate Pydantic schemas from a central Protobuf definition, then checked those into each service's local /models directory.

The Service Template

Every service follows this directory structure to ensure that any engineer can jump between repos without a cognitive load penalty:

.
├── app
│   ├── api          # Routes and dependencies
│   ├── core         # Config, logging, security
│   ├── db           # Session management, migrations
│   ├── models       # Pydantic schemas (Generated)
│   ├── services     # Logic layer (The "Fat Service" pattern)
│   └── main.py      # FastAPI entrypoint
├── alembic          # DB Migrations
├── docker-compose.yml
└── Dockerfile

Day 7: The API Gateway and JWT Passthrough

We realized early that we didn't want user-service to be hit for every single request just to validate a token. We implemented an "Edge Auth" pattern at the Gateway level. The Gateway validates the JWT's signature (using a shared RSA public key), Extracts the user_id and scopes, and then injects them into custom headers like X-User-ID.

This means the downstream microservices never need to touch the database to know who made the request. They simply trust the X-User-ID header because the services are isolated in a private Docker network and only the Gateway is exposed to the internet.

```python
# app/api/dependencies.py inside Order Service
from fastapi import Header, HTTPException

async def get_authenticated_user(x_user_id: str = Header(None)): if not x_user_id: # In internal network, this should only happen if # the Gateway is misconfigured or bypassed. raise HTTPException(status_code=403, detail="Gateway bypass blocked") return x_user_id

@router.post("/orders") async def create_order( order_data: OrderCreate, user_id: str = Depends(get_authenticated_user) ): # Business logic follows... pass ```

Day 12: Distributed Transactions and Redis Streams

By the second week, we hit the classic "Dual Write" problem. An order was created in order-service, but the message to notify the inventory-service failed because Redis was momentarily blipping. The database had the order, but the inventory was never deducted.

We stopped using simple Redis LPUSH/BRPOP queues and moved to Redis Streams. Streams provide consumer groups and acknowledgment mechanics, allowing for a "claimed but not finished" state.

We also implemented the Transactional Outbox Pattern. Instead of the order-service writing to Postgres and then trying to write to Redis, it writes both the Order and the "Pending Task" to the same Postgres transaction. A sidecar process (a small Python script running select * from outbox where status='pending') then pushes the data to Redis and marks the outbox entry as sent. This ensures at-least-once delivery without distributed locks.

Day 15: The P99 Crisis and Dependency Injection

Performance started degrading. Our p99 latency spiked from 40ms to 450ms. Profiling revealed that FastAPI's dependency injection system was being abused. We were creating a new Postgres connection and a new Redis client for *every* sub-dependency in a single request.

The fix was moving to asynchronous connection pooling and using lru_cache for configuration objects.

```python
# Fixed Postgres engine singleton
from sqlalchemy.ext.asyncio import create_async_engine, AsyncSession
from sqlalchemy.orm import sessionmaker

class DatabaseManager: def __init__(self, db_url: str): self.engine = create_async_engine( db_url, pool_size=20, max_overflow=0, pool_timeout=30, pool_recycle=1800, ) self.async_session = sessionmaker( self.engine, class_=AsyncSession, expire_on_commit=False )

db_manager = DatabaseManager("postgresql+asyncpg://user:pass@db/dbname")

async def get_db(): async with db_manager.async_session() as session: yield session ```

We also found that asyncpg was significantly faster than standard psycopg2 with thread pools. Switching the driver and tuning the pool size to match the number of worker processes (usually 2 * CPU_CORES + 1) dropped our p99 back down to sub-100ms levels.

Day 22: Observing the Chaos with OpenTelemetry

A microservices architecture is a black hole without tracing. When an order failed, we didn't know if it was the Gateway, the Order Service, or the database. We integrated opentelemetry-instrumentation-fastapi.

We configured a global Trace-ID that is generated at the Gateway and passed through the X-Trace-ID header. Every log line in every service now includes this ID. When a customer reports an error, we search the logs for that ID and see the entire lifecycle of the request across four different containers.

Crucially, we learned to stop logging everything. In a microservice environment, logging every SQL query in production generates gigabytes of noise. We scaled back to logging only errors and "key transition events" (e.g., OrderMovedToProcessing), which reduced our CloudWatch costs by 60%.

Day 30: The Docker Compose Reality Check

Local development became a nightmare as we hit 8 services. Running 8 FastAPI instances, 2 Postgres instances, Redis, and Traefik locally consumed 14GB of RAM.

We optimized the Dockerfile to use multi-stage builds. The development stage includes watchfiles for hot-reloading, while the production stage is a slim debian-slim image without build tools.

```dockerfile
# Dockerfile
FROM python:3.11-slim-bookworm as base
ENV PYTHONUNBUFFERED=1 \
    PYTHONDONTWRITEBYTECODE=1

WORKDIR /app

FROM base as builder RUN apt-get update && apt-get install -y gcc libpq-dev COPY requirements.txt . RUN pip install --prefix=/install -r requirements.txt

FROM base COPY --from=builder /install /usr/local COPY ./app /app/app COPY ./alembic /app/alembic COPY ./alembic.ini /app/alembic.ini

CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"] ```

By stripping out pip cache and build-essential tools from the final image, we shrunk the image size from 850MB to 120MB, which significantly sped up our CI/CD pipelines and deployment times in the cluster.

The Operational Reality of FastAPI Microservices

Moving from a monolith to this architecture took a simple stack and made it complex. However, the gains were concrete. In the monolith, a memory leak in the PDF generator would crash the entire API. In the microservice version, the worker-service simply hits its OOM (Out of Memory) limit, Docker restarts it, and the Redis Stream ensures no jobs are lost.

The most expensive lesson we learned was that latency is cumulative. If your Gateway calls Service A (30ms), which calls Service B (50ms), your total latency is 80ms + network overhead. If you can't run these calls in parallel using asyncio.gather(), you are building a distributed monolith that is slower than the original.

The sharp takeaway: FastAPI is the perfect glue for this because of its native async support, but the framework won't save you from a bad architectural split. If your services share a database, they aren't microservices—they're a distributed headache. Keep your schemas strictly separated and your communication asynchronous, or the complexity will swallow your velocity.