Scaling
High availability
Keep Flowplan running when a server fails or is updated: active/passive with a standby server, or active/active in cluster mode – with a complete example at Hetzner.
11 min read
On this page
Two ways
| Active/passive | Active/active (cluster mode) | |
|---|---|---|
| What runs | two servers, Flowplan runs on only one at a time | two or more instances at once behind a load balancer |
| A server fails | you start Flowplan on the other one (minutes) | the other instances keep working (seconds) |
| Updates | a short interruption while it restarts | no interruption, one instance after the other |
| Database | SQLite with Litestream, or PostgreSQL | PostgreSQL |
| Other services | S3 (or a volume you can move) | PostgreSQL, Redis and S3 |
| More people | no, one instance carries everyone | yes, about 25–30 active people per instance (Sizing) |
For most teams active/passive is the better start: little new to run, and an outage costs minutes instead of hours. The cluster pays off when updates should not interrupt at all, or when one instance is no longer enough.
What both need
- Files on more than one server: an S3 bucket (Storage, S3 and backups). Required in a cluster, the simplest way for active/passive; otherwise a volume you can attach to the other server (for SQLite never a network share such as NFS).
- The same keys as variables on every server:
FLOWPLAN_ENCRYPTION_KEY(orFLOWPLAN_SECRETS_KEY) as well asWEB_PUSH_PUBLIC_KEYandWEB_PUSH_PRIVATE_KEY. Without them each instance creates its own in its data folder, and two-factor sign-in, stored secrets and push subscriptions only match one server. How to create them or take them over from an existing data folder: Cluster mode. - One address: a load balancer or a floating IP in front, the same
APP_URLeverywhere andFLOWPLAN_TRUSTED_PROXIESmatching the number of proxies. - The same version: the same image tag (
FLOWPLAN_VERSION) on every server.
Active/passive
Two servers with the same compose.yaml and the same .env. Flowplan runs on the active one; on the passive one the container is stopped. In front sits a load balancer that checks /api/health/ready and sends traffic only to the running server – then nothing about the address changes when you switch. With a floating IP instead of a load balancer, you move the IP to the other server when you switch.
Without cluster mode only one instance may run at a time. Two instances run timed jobs twice (reminders, digests, backups, workflows), and live changes do not reach the other one. With SQLite and Litestream, two running instances spoil the copy in the bucket.
With PostgreSQL
Both servers point DATABASE_URL at the same database and S3_* at the same bucket. If a second instance starts by mistake, Flowplan notices through a lock in the database: it still starts, logs a loud warning every ten minutes, and Administration → Monitoring shows on every instance, until only one runs: Since 0.10.0
"A second Flowplan instance is running against the same database without cluster mode (instance, since time). Timed jobs such as reminders, digests, backups and workflows then run twice, and live changes do not reach the other instance. Switch cluster mode on for every instance (FLOWPLAN_CLUSTER=1 with Redis and S3) or stop one of the instances. During a rolling restart this is normal for a moment."
Switching over when the active server fails:
- Make sure the old server is really down: stop the container or power the server off in your provider's console. A server that is merely unreachable may otherwise keep running.
- On the passive server,
docker compose up -d. - Wait until
/api/health/readyanswers200; the load balancer then sends traffic there by itself (with a floating IP: move the IP now). - Check under Administration → Monitoring that no warning about a second instance shows.
Switching back works the same way. A stopped server may start again only once the other one is stopped.
With SQLite and Litestream
The database is a file on the active server; Litestream copies every change into the bucket continuously (under <prefix>/db/, every second), and the files are there too. The passive server has the same S3_* variables and keys and an empty data folder. The warning about a second instance does not apply here – each instance would have a file of its own.
- Stop the old server – two Litestream processes must never write into the same folder of the bucket at once.
- No old database may be on the passive server: Litestream restores only when
flowplan.sqliteis missing. Empty a data folder from an earlier stint first (or take a new volume). docker compose up -d: Litestream fetches the database from the bucket, Flowplan the files on first access.- Switch the address as above; afterwards empty the old server's data folder before it becomes the passive server.
At most the changes of the last second before the failure, which Litestream had not copied yet, are lost. The steps match way C of moving an instance. Without S3 a block volume works too, which you detach from the failed server and attach to the passive one (at Hetzner: Volumes, within one location only) – never a network share.
Switching automatically (e.g. with keepalived) only with a safe way to power off the old server: after a network glitch both run otherwise.
Active/active: cluster mode
In cluster mode all instances run at once. They share live editing, presence, rate limits and the parallel AI answers through Redis, all data is in PostgreSQL, all files in the bucket; any instance answers imports and print views. No sticky sessions are needed. When an instance fails, the load balancer takes it out after the next health check, and another one takes over the timed jobs within seconds.
Two single points remain: PostgreSQL (see Failover and replicas and below) and Redis. When Redis fails, everyone keeps working; only live changes between instances arrive late. Set FLOWPLAN_READY_REQUIRE=database,objectStorage for that – otherwise every instance reports "not ready" and the load balancer has no target left.
Updates without downtime
One instance after the other: stop it, start the new image, wait until it is ready, then the next one. On SIGTERM an instance answers 503 on /api/health/ready at once, finishes open requests and hands on live connections and the lead (stop_grace_period: 30s). The new version only extends the schema, so old instances keep working with it.
So that browsers do not mix versions, every image carries a deployment id (v<version>-<commit>, fixed at build time). The app requests its files with ?dpl=<id>, so caches on the way serve nothing stale, and when a page of the old version reaches a new instance, the browser reloads the whole page at the next navigation. Should an instance still lack a file of the other version, Flowplan's own build check heals the page. Building the image yourself, set the id with FLOWPLAN_DEPLOYMENT_ID (--build-arg FLOWPLAN_DEPLOYMENT_ID=…) – the same for every instance of one image – or switch it off with off. Since 0.10.0
Example: Hetzner Cloud
A cluster with two Flowplan servers behind the Hetzner Load Balancer, Redis and PostgreSQL in a private network, and files in Hetzner Object Storage. The files for it are in the repository under deploy/compose/hetzner (opens in a new tab). Names, plans and menus change – the Hetzner Console is what counts.
Internet ── HTTPS ──▶ Load Balancer (public IP, certificate)
│ private network 10.0.0.0/16
┌───────────┴───────────┐
app-1 :3000 app-2 :3000 Flowplan (FLOWPLAN_CLUSTER=1)
└─────┬───────────┬─────┘
redis :6379 postgres :5432 private network only
Object Storage (S3, over HTTPS)Servers and network
All servers in the same location (e.g. Falkenstein, fsn1), with Ubuntu 24.04 and your SSH key.
| Server | Size | Note |
|---|---|---|
app-1, app-2 | 2 vCPU, 4 GB RAM each (the smallest type of that size, Arm or x86) | each carries about 25–30 active people; more app servers for more people |
redis | the smallest type | stores nothing permanently |
postgres | 2 vCPU, 4 GB RAM; from about 60 active people 4 vCPU, 8 GB, ideally with dedicated vCPUs | or one of the options below |
- Network: under Networks, create a private network (e.g.
10.0.0.0/16) and attach every server and the load balancer. Flowplan, Redis and PostgreSQL listen on their private address only. - Placement group: put
app-1andapp-2into a placement group of type spread, so they do not run on the same physical host. - Firewall: one Cloud Firewall for every server: inbound only TCP 22 from your address. The app servers need no public ports – the load balancer reaches them through the private network. The Cloud Firewall only applies to the public interface, so protect Redis and PostgreSQL with a password as well and bind them to the private address (Docker bypasses
ufw).
Object Storage
Under Object Storage, create a private bucket in the same location and create a key under S3 credentials (the secret is shown only once). Endpoint and region follow the location, e.g. S3_ENDPOINT=https://fsn1.your-objectstorage.com and S3_REGION=fsn1 – use the endpoint the console shows for your bucket. With FLOWPLAN_ENCRYPTION_KEY, files and backups are stored there encrypted.
PostgreSQL
| Option | The database server fails | Effort |
|---|---|---|
One PostgreSQL server with backups (postgres.yaml in the example) | Flowplan is down until the server runs again or is rebuilt from the backup | low: one server, backups as below |
| Patroni on three small servers with etcd, HAProxy locally on every app server | another server takes over within seconds | high: run and monitor Patroni, etcd and HAProxy |
| Managed service of a provider with a data centre nearby | the provider switches over | low, but the database sits outside the private network: only with TLS (sslmode=require) and access only from your servers' addresses |
If Hetzner offers no managed PostgreSQL in your location, pick a provider nearby – every query travels that distance. How Flowplan handles a failover and which address belongs in DATABASE_URL is under Failover and replicas. With Patroni, DATABASE_URL points to 127.0.0.1:5000, where HAProxy offers only the primary.
Redis
A small server with redis.yaml from the example: a password, no persistence, on the private address only. A managed Redis with a fixed address works too (rediss:// with TLS); Redis Sentinel and Redis Cluster do not. With FLOWPLAN_READY_REQUIRE=database,objectStorage (as in .env.example), Flowplan stays ready while Redis is briefly away.
Load balancer
Under Load Balancers, create a load balancer in the same location (the smallest type is enough) and attach it to the private network:
| Setting | Value |
|---|---|
| Targets | app-1 and app-2, through the private network |
| Service | HTTPS, port 443 → destination port 3000; a certificate managed by Hetzner (Let's Encrypt; the DNS zone must be at Hetzner for that) or one you upload |
| HTTP → HTTPS | switch the redirect on |
| Health check | HTTP, port 3000, path /api/health/ready, every 5–10 seconds, timeout 5 seconds, status 200 |
| Sticky sessions | off – not needed |
| Proxy protocol | off |
| Algorithm | round robin or least connections |
The load balancer sets X-Forwarded-For, which is why .env has FLOWPLAN_TRUSTED_PROXIES=1. Live connections are server-sent events: the load balancer has to pass them through unbuffered, and Flowplan sends a heartbeat regularly so they are not closed for silence. After setting up, check that a second person's changes appear at once and that a large import (up to 100 MB) goes through. Finally point your domain's A and AAAA records at the load balancer's addresses.
Setting up the app servers
- Install Docker following the official guide (opens in a new tab).
- Get the example files:
git clone https://github.com/yannicschuller/flowplan.gitand copy what is indeploy/compose/hetznerto/opt/flowplan. - Create the keys once –
openssl rand -base64 32forFLOWPLAN_ENCRYPTION_KEY,npx web-push generate-vapid-keysfor the push keys – and keep a copy apart from the backups. cp .env.example .envand fill it in: domain, database, Redis, bucket, keys, setup token..envis the same on both app servers except forPRIVATE_IPandFLOWPLAN_INSTANCE_ID.docker compose up -donapp-1first, once it is ready onapp-2; then create the first account athttps://<domain>/register?setup=<token>.
Backups
- Switch on the scheduled backup under Administration → Storage & backup: the leading instance writes a
pg_dumparchive into the bucket under<prefix>/backups/daily (or hourly), encrypted withFLOWPLAN_ENCRYPTION_KEY(Backups with pg_dump). - A second copy outside the bucket, e.g. via SFTP to a Hetzner Storage Box – a fault in the bucket then does not hit both.
- The database server additionally with Hetzner's backups (images of the disk); with Patroni, or to restore to any point in time, WAL archiving (e.g. pgBackRest) into a bucket of its own.
- The app servers hold nothing that is not in the bucket or the database – a new server with the same
.envreplaces an old one.
Updates
- Read the release notes and set
FLOWPLAN_VERSIONin.envon both servers to the new version. - On
app-1:docker compose pull && docker compose up -d. The old instance answers503, the load balancer takes it out, and the new one brings the schema up to date. - Wait until the load balancer shows
app-1as healthy again (orcurl http://<private IP>:3000/api/health/ready). - The same on
app-2.
To be on the safe side, remove the server as a target in the load balancer before step 2 and add it back afterwards. The deployment id makes sure browsers do not mix both versions in the meantime.
Next
- Cluster mode: every requirement and variable.
- PostgreSQL: failover, replicas and time limits.
- Sizing: how many instances and which hardware.
- Kubernetes: the same with the Helm chart.