Skip to content
flowplanDocumentationCommunityMarketplaceStart page

Scaling

Cluster mode

In cluster mode, several Flowplan instances serve one installation: one PostgreSQL database, one Redis, one S3 bucket and any number of instances behind a load balancer – for more people active at the same time and updates without downtime.

12 min read

On this page
  1. Do you need it?
  2. Requirements
  3. What the instances share
  4. Health, readiness and shutdown
  5. Schema before the start
  6. Docker Compose
  7. Kubernetes (Helm)
  8. Administration → Monitoring → System
  9. More variables
  10. Workflows in a cluster
  11. Known limits
  12. Next

Do you need it?

Most teams do not: a single container carries about 20 simultaneously active people with SQLite and about 30 with PostgreSQL (see Sizing). Cluster mode pays off when more people regularly work at the same time, or when Flowplan should keep running through updates and the loss of a server. In return it brings three services you have to run (or rent) and back up.

If Flowplan only has to survive the loss of a server, active/passive without cluster mode is often enough – see High availability, with an example at Hetzner. Since 0.10.0

Requirements

Switch it on with FLOWPLAN_CLUSTER=1. Then these are required:

WhatVariableFor
PostgreSQL 16+DATABASE_URL=postgres://…All data. SQLite is a file on one host and does not work. See PostgreSQL.
Redis (tested with 7)REDIS_URL=redis://… or rediss://… (TLS)Live editing, presence and rate limits between the instances. Redis does not need to persist any of it.
S3 storageS3_BUCKET, S3_ENDPOINT, S3_ACCESS_KEY_ID, S3_SECRET_ACCESS_KEYAll files; the local uploads/ folder is only a cache. See Storage, S3 and backups.

If one is missing, an instance does not start and lists every problem at once in the log. REDIS_URL without cluster mode is ignored with a warning.

Also required – here too an instance refuses to start without them: keys that are identical on every instance. A single container creates some keys itself in its data folder – in a cluster every instance has its own folder, and each would create different ones. So set the same values everywhere:

  • FLOWPLAN_ENCRYPTION_KEY or FLOWPLAN_SECRETS_KEY (or their …_FILE variants, e.g. for Docker secrets) – otherwise an instance cannot read stored secrets and signatures of the others (two-factor sign-in, AI keys, unlocked share links). See Stored secrets.
  • WEB_PUSH_PUBLIC_KEY and WEB_PUSH_PRIVATE_KEY (a VAPID key pair, e.g. from npx web-push generate-vapid-keys) – otherwise push subscriptions only match one instance. See Push notifications.

Coming from a single container without these variables, take the values from its data folder so stored secrets and push subscriptions stay valid: the content of secrets.key as FLOWPLAN_SECRETS_KEY (in addition to a FLOWPLAN_ENCRYPTION_KEY; both are read), and publicKey/privateKey from web-push-keys.json as WEB_PUSH_PUBLIC_KEY/WEB_PUSH_PRIVATE_KEY.

What the instances share

  • Live editing and presence: changes to documents, records and whiteboards, cursors and "who is here" reach every instance through Redis. Two people on different instances work together as on one. The data itself is in PostgreSQL; if a message is lost, the browser catches up on its next sync. No sticky sessions are needed.
  • Rate limits: limits such as those for AI requests count across all instances (through Redis). Limits already kept in the database (failed sign-ins, demos, MCP) applied to all of them anyway. So do the at most three parallel AI answers per person. Since 0.10.0
  • Imports and print views: their state is in the database, so any instance answers the progress of a background import and serves a print view. Since 0.10.0
  • Leader-only jobs: exactly one instance leads at any time (elected through a lock in PostgreSQL). Only it runs the timed jobs: reminders, automations, page reviews, the search indexer, text extraction from files, cleanup and retention, digests, scheduled backups and Markdown exports, LDAP, calendar and issue sync, inbound e-mail, Telegram, AI background work and the monitoring alerts. If it fails, another one takes over within seconds. Every round is guarded: taking the lead raises a counter in the database, a round starts only while it is still its own, and each job holds a lock of its own while its round runs – so a round of an instance that lost its connection never runs twice beside the new leader's. Since 0.10.0
  • Queues: every instance sends e-mails, push messages, webhooks and chat-app messages; each message is taken by exactly one instance.
  • Files: the bucket is the truth. A request that stores a file answers only once the file is in the bucket; every instance reads through its cache and fetches from the bucket what it lacks. Uploads in several requests (long transcriptions, recordings of meetings and notebooks, an import between its preview and the import) keep their parts in the bucket under <prefix>/tmp/, so each request may reach another instance. Since 0.10.0

Health, readiness and shutdown

AddressAnswersMeant for
/api/health/live200 as long as the process answers – no database check, since a restart does not fix a database outageLiveness probe
/api/health/ready200 when the schema and the data steps are done Since 0.10.0 and the database, Redis and the bucket answer; 503 otherwise and while shutting downReadiness probe, load balancer
/api/healthas before, see Health checkDocker health check, apps

Without a token, /api/health/ready only answers ok/error per check; with FLOWPLAN_METRICS_TOKEN as a bearer token, response times and error messages are added.

FLOWPLAN_READY_REQUIRE decides which checks count for readiness – a list of database, redis and objectStorage, by default all three. The database always counts. If Redis fails, the default turns every instance "not ready" at once and the load balancer has nothing left. With FLOWPLAN_READY_REQUIRE=database,objectStorage they stay ready: editing and saving keep working, only live changes between instances arrive on the next sync, and each instance counts rate limits on its own. The Helm chart sets it so. Since 0.10.0

Shutdown: on SIGTERM the instance immediately answers 503 on /api/health/ready, closes its live connections (browsers reconnect through the load balancer to another instance), finishes open requests, hands over the lead and signs off from the others. Give it 30 seconds for this (stop_grace_period, terminationGracePeriodSeconds). Running jobs and workflow steps still finish, uploads under way reach the bucket, then it hands back leases and the lead – all within FLOWPLAN_SHUTDOWN_SECONDS (20). Since 0.10.0

Schema before the start

Each instance brings the database schema up to its version on start (under a lock; they wait for each other). Cleaner is a separate step before the instances start, which fails loudly:

DATABASE_URL=postgres://… node /app/scripts/db-schema.mjs

The Docker Compose example and the Helm chart work this way. The instances still check the schema at their start under the same lock (nothing to do then) and are ready only once it is current; FLOWPLAN_PG_SCHEMA_AUTO=off leaves it to the separate step alone. Since 0.10.0 The schema is only extended, never shrunk: during a rolling update, old and new instances run side by side.

So that browsers do not mix versions meanwhile, every image carries a deployment id. Since 0.10.0

Docker Compose

A complete example lives in deploy/compose/cluster (opens in a new tab): PostgreSQL 16, Redis 7 with a password and without persistence, MinIO as S3 including bucket creation, the schema step, two Flowplan instances and Caddy as load balancer. It is meant for trying things out on one host and as a starting point; in production, run the database, Redis and storage as separate, backed-up services.

git clone https://github.com/yannicschuller/flowplan.git
cd flowplan/deploy/compose/cluster
cp .env.example .env    # replace every "change-me" value
docker compose up -d

In it:

  • .env holds APP_URL, FLOWPLAN_SETUP_TOKEN and the passwords for PostgreSQL (POSTGRES_PASSWORD and POSTGRES_ADMIN_PASSWORD), Redis and MinIO – placeholders only, to be replaced. FLOWPLAN_ENCRYPTION_KEY, WEB_PUSH_PUBLIC_KEY and WEB_PUSH_PRIVATE_KEY are required too – empty in .env.example; without them docker compose up stops with a hint (see Requirements).
  • The schema service runs once before the instances (node /app/scripts/db-schema.mjs); the instances have FLOWPLAN_PG_SCHEMA_AUTO=off.
  • flowplan-1 and flowplan-2 each have their own volume as a cache, FLOWPLAN_PG_POOL=20 and a health check on /api/health/ready. More instances: copy the flowplan-2 service and add it to the Caddyfile.
  • Caddy distributes round robin over ready instances (health check on /api/health/ready) and passes live connections through unbuffered (flush_interval -1). For HTTPS, replace :8080 in the Caddyfile with your host name.
  • PostgreSQL runs with max_connections=200. Flowplan connects as the role flowplan, which owns the database but is no superuser: POSTGRES_PASSWORD is its password, POSTGRES_ADMIN_PASSWORD that of the image's superuser postgres, for administration only. On the database's first start postgres-init.sh creates the role and installs pg_trgm (and pgvector when the image has it) into flowplan_ext. A volume made with an older version of the example keeps its roles – there flowplan is still a superuser until you demote it.

Another load balancer (nginx, Traefik, HAProxy) needs the same: health check on /api/health/ready, no buffering for server-sent events (/api/documents/live, /api/live and the cursor channels), long read timeouts, uploads up to 110 MB and FLOWPLAN_TRUSTED_PROXIES matching the number of proxies.

Kubernetes (Helm)

Installation, moving from a SQLite container, updates, backups and troubleshooting are on the page Kubernetes. Since 0.10.0

The chart in deploy/helm/flowplan (opens in a new tab) has cluster mode built in (cluster.enabled=true). It does not create PostgreSQL, Redis or S3 – use managed services or their own charts. The connection addresses live only in Secrets you create yourself:

kubectl create namespace flowplan
kubectl -n flowplan create secret generic flowplan-cluster \
  --from-literal=DATABASE_URL='postgres://flowplan:<password>@postgres:5432/flowplan' \
  --from-literal=REDIS_URL='redis://:<password>@redis-master:6379/0'
kubectl -n flowplan create secret generic flowplan-s3 \
  --from-literal=S3_ACCESS_KEY_ID='<key-id>' \
  --from-literal=S3_SECRET_ACCESS_KEY='<secret>'
# the keys every pod shares (push keys: npx web-push generate-vapid-keys)
kubectl -n flowplan create secret generic flowplan-keys \
  --from-literal=FLOWPLAN_ENCRYPTION_KEY="$(openssl rand -base64 32)" \
  --from-literal=WEB_PUSH_PUBLIC_KEY='<public VAPID key>' \
  --from-literal=WEB_PUSH_PRIVATE_KEY='<private VAPID key>'

helm install flowplan deploy/helm/flowplan -n flowplan \
  --set appUrl=https://flowplan.example.org \
  --set cluster.enabled=true \
  --set cluster.replicas=3 \
  --set cluster.database.existingSecret=flowplan-cluster \
  --set cluster.redis.existingSecret=flowplan-cluster \
  --set cluster.keys.existingSecret=flowplan-keys \
  --set s3.bucket=flowplan --set s3.endpoint=https://s3.example.org \
  --set s3.existingSecret=flowplan-s3

The example values-ha.yaml (opens in a new tab) holds the same with ingress, SSO, metrics and network policy. The chart takes secrets only as references to Secrets (s3.existingSecret, existingSecret), not as values. Since 0.10.0

What the chart does in cluster mode:

  • Schema job as a pre-install/pre-upgrade hook (node /app/scripts/db-schema.mjs) before pods of a new version start. Without it (cluster.migration.enabled=false) the pods do it on start.
  • Rolling updates without downtime (maxUnavailable: 0), a PodDisruptionBudget (cluster.podDisruptionBudget.minAvailable, default 1) and pods spread over nodes where possible.
  • Probes: liveness on /api/health/live, readiness on /api/health/ready – without Redis (cluster.readyRequire: database,objectStorage) Since 0.10.0; before stopping, a pod waits cluster.preStopSeconds (5) so the load balancer takes it out.
  • No volume: /app/data is an emptyDir per pod (cluster.cacheSizeLimit, 5 Gi) that keeps itself below 70 % of its limit (FLOWPLAN_CACHE_MAX_MB) Since 0.10.0; files live in the bucket, backups come from PostgreSQL (pg_dump).
  • Database role: the role in DATABASE_URL should own the database but not be a superuser. Many PostgreSQL charts make their user a superuser – create a role of its own then, see Role without superuser rights.
  • Connections: cluster.poolSize (10) per pod plus three of its own Since 0.10.0; replicas × (poolSize + 3) must stay below the server's max_connections.
  • Shared keys: every pod gets FLOWPLAN_ENCRYPTION_KEY, WEB_PUSH_PUBLIC_KEY and WEB_PUSH_PRIVATE_KEY from the Secret in cluster.keys.existingSecret. If the keys are named differently in your Secret, name them with cluster.keys.encryptionKey, cluster.keys.webPushPublicKey and cluster.keys.webPushPrivateKey. Keep a copy of the encryption key apart from the backups.
  • The chart refuses to install if the database, Redis, the bucket or cluster.keys.existingSecret is missing.

For ingress-nginx, the chart sets the annotations for live connections and uploads itself (no buffering, timeouts of an hour, ingress.bodySize); the chart's README describes other controllers and explains every value. Since 0.10.0

Administration → Monitoring → System

The System section under Administration → Monitoring shows how the installation runs:

  • Mode (single container, or cluster with the number of instances), Database with version and reachability,
  • Connections of this instance (open, idle, waiting) and the Timeouts,
  • Redis with version and messages sent, received and failed, Object storage with bucket and reachability,
  • the Leader and since when it leads,
  • the Shared queues (files, mail, push, webhooks, chat apps),
  • in a cluster, a table of all Instances with their state (leader, ready, stopping), busy and waiting connections, memory and last heartbeat. An instance silent for 30 seconds drops off the list.

"Waiting" connections mean requests are waiting for a free database connection – raise FLOWPLAN_PG_POOL or add instances.

A second instance without cluster mode: when two instances run against the same PostgreSQL database without FLOWPLAN_CLUSTER=1, the log and Monitoring warn: "A second Flowplan instance is running against the same database without cluster mode …" – timed jobs then run twice. Since 0.10.0

More variables

VariableDefaultMeaning
FLOWPLAN_CLUSTERempty1 (or true, on) switches cluster mode on. Equivalent: FLOWPLAN_MODE=cluster.
REDIS_URLemptyredis://:<password>@host:6379/0, with TLS rediss://…. User, password and database number go into the address.
FLOWPLAN_READY_REQUIREdatabase,redis,objectStorageChecks that decide /api/health/ready (above).
FLOWPLAN_INSTANCE_IDhost name + randomName of the instance in the overview and in the application_name of its database connections. The Helm chart uses the pod name.
FLOWPLAN_REDIS_PREFIXflowplan:Prefix for keys and channels when several installations share one Redis.
FLOWPLAN_LEADER_INTERVAL5000How often (ms) instances check the lead; decides how fast a failed leader is replaced.
FLOWPLAN_SHUTDOWN_SECONDS20How long stopping on SIGTERM may take (running jobs, uploads, handing over the lead) – a few seconds below the platform's grace period. Applies to the single container too. The Helm chart sets terminationGracePeriodSeconds less the pause less 5. Since 0.10.0
FLOWPLAN_CACHE_MAX_MBemptySize (MB) above which an instance in a cluster removes the longest unused files from its cache; they stay in the bucket. Empty: no limit. The Helm chart sets 70 % of cluster.cacheSizeLimit. Since 0.10.0

Workflows in a cluster

The steps of workflows wait in a queue in the database. Every instance claims steps that are due with a lease (2 minutes); a fencing token makes sure an instance whose lease has run out can no longer commit. If an instance fails, another takes over its steps once the lease has expired. New steps wake the other instances through Redis; without the bus they check on a timer. Schedules and date triggers run only on the leader and exactly once per slot. FLOWPLAN_WORKFLOW_WORKERS sets how many steps an instance runs at once.

Known limits

  • Files: if the bucket does not answer, uploads fail with 503 – a file counts as stored only once it is in the bucket. Since 0.10.0
  • Restores and instance moves with one instance. An uploaded backup sits in the cache of the instance that received it and is only applied once that instance leads. Applying it replaces every table in one transaction; requests from other instances wait meanwhile and fail after FLOWPLAN_PG_LOCK_TIMEOUT. On Kubernetes, an emptyDir is lost on restart. So scale down to one instance first (cluster.replicas=1, or only flowplan-1), restore, then scale up again. The same goes for the move tool.
  • Redis is a single point: if it fails, editing and saving keep working, but live changes between instances only arrive on the next sync, each instance counts rate limits on its own, and with the default FLOWPLAN_READY_REQUIRE every instance turns "not ready" (above; the Helm chart leaves Redis out Since 0.10.0).
  • No Redis Sentinel and no Redis Cluster: Flowplan talks to exactly one Redis address. A managed Redis with a stable address works.
  • Per instance remain the progress displays of restores and moves and the search indexer's backlog figures.

Next

  • Sizing: how many instances and what hardware.
  • Switching the database: bring an existing instance to PostgreSQL. For a cluster, first turn on S3 in the single container and wait until every file is in the bucket (Administration → Storage & backup, or objectStorage.pending at 0 in /api/health).
Was this helpful?
YesThanks!
No – tell us what's missing

Found a mistake or something missing?Edit on GitHub