Scale from one box to a cluster
What has to move, and why
The gateway data plane is stateless — per-request virtual-key auth on /v1, dashboard sessions externalized — so only shared state needs a home every replica can reach:
| State | Single box | Cluster |
|---|---|---|
| Governance config + system of record | PostgreSQL | PostgreSQL (shared/managed) |
| Rate-limit counters, sessions, config broadcasts | in-JVM | Valkey |
| SIEM projection + audit store | OpenSearch (Pro) | OpenSearch (scaled) |
| Report artifacts | local directory | S3-compatible object store |
1. Externalize the object store
Switch aim.objectstore.backend to s3 and point it at any S3-compatible store (MinIO, SeaweedFS, AWS S3, R2), so every replica can serve a report download. Objects stay AES-256-GCM encrypted at rest regardless of backend.
2. Move counters and broadcasts to Valkey
Set aim.ratelimit.backend=valkey and aim.cluster.backend=valkey. Valkey holds the hot rate-limit counters (atomic increments on TTL'd minute buckets — the only backend that keeps an org-wide cap accurate across replicas) and carries config-change broadcasts so a change on one replica reloads on all.
3. Run replicas behind a load balancer
Single large host: the distributed compose file wires Postgres, Valkey, and OpenSearch for you:
cd deploy
docker compose --env-file .env -f compose/distributed.yml up -d --build --scale app=3
Multiple machines: use the Helm chart (deploy/helm/aimanager) — N replicas as a Deployment with HPA and Ingress, pointing at your managed PostgreSQL, an OpenSearch cluster, and HA Valkey.
4. Scheduled maintenance — already handled
Audit-partition upkeep and similar scheduled jobs become a cluster singleton automatically via a cluster lock, so they run once cluster-wide no matter how many replicas exist. OpenSearch retention and snapshots are scheduler-free by design.
Verify
- Generate a report on one replica and download it from another (object store shared).
- Hammer a scope's RPM cap and confirm the limit holds across replicas — the cap, not N × the cap (Valkey counters shared).
- Change a guardrail policy and confirm it takes effect on all replicas without restarts (broadcasts working).
Why it works
Statelessness was a design constraint, not an afterthought: nothing on the request path holds instance-local state that matters. Scaling out is therefore only a matter of re-pointing the three shared stores — the application itself is identical, byte for byte, to the single-box install.
Next
- Installation — Distributed — the install-time view of the same topology.
- System requirements — sizing the shared components.