Scale from one box to a cluster

Scale from one box to a cluster

What has to move, and why

The gateway data plane is stateless — per-request virtual-key auth on /v1, dashboard sessions externalized — so only shared state needs a home every replica can reach:

StateSingle boxCluster
Governance config + system of recordPostgreSQLPostgreSQL (shared/managed)
Rate-limit counters, sessions, config broadcastsin-JVMValkey
SIEM projection + audit storeOpenSearch (Pro)OpenSearch (scaled)
Report artifactslocal directoryS3-compatible object store

1. Externalize the object store

Switch aim.objectstore.backend to s3 and point it at any S3-compatible store (MinIO, SeaweedFS, AWS S3, R2), so every replica can serve a report download. Objects stay AES-256-GCM encrypted at rest regardless of backend.

2. Move counters and broadcasts to Valkey

Set aim.ratelimit.backend=valkey and aim.cluster.backend=valkey. Valkey holds the hot rate-limit counters (atomic increments on TTL'd minute buckets — the only backend that keeps an org-wide cap accurate across replicas) and carries config-change broadcasts so a change on one replica reloads on all.

3. Run replicas behind a load balancer

Single large host: the distributed compose file wires Postgres, Valkey, and OpenSearch for you:

Command
cd deploy
docker compose --env-file .env -f compose/distributed.yml up -d --build --scale app=3

Multiple machines: use the Helm chart (deploy/helm/aimanager) — N replicas as a Deployment with HPA and Ingress, pointing at your managed PostgreSQL, an OpenSearch cluster, and HA Valkey.

4. Scheduled maintenance — already handled

Audit-partition upkeep and similar scheduled jobs become a cluster singleton automatically via a cluster lock, so they run once cluster-wide no matter how many replicas exist. OpenSearch retention and snapshots are scheduler-free by design.

Verify

Cluster verification
  • Generate a report on one replica and download it from another (object store shared).
  • Hammer a scope's RPM cap and confirm the limit holds across replicas — the cap, not N × the cap (Valkey counters shared).
  • Change a guardrail policy and confirm it takes effect on all replicas without restarts (broadcasts working).

Why it works

Statelessness was a design constraint, not an afterthought: nothing on the request path holds instance-local state that matters. Scaling out is therefore only a matter of re-pointing the three shared stores — the application itself is identical, byte for byte, to the single-box install.

Next