Scale your self-hosted deployment infrastructure
Learn how to scale your Retool deployment and infrastructure.
Retool's self-hosted Docker image consists of several containers required to run your deployment. Before you continue, you should review Retool's self-hosted architecture to understand which services need scaling as usage increases.
Scale your Retool deployment
It's recommend to self-host Retool on a container orchestration service, such as Kubernetes. Retool provides Helm charts and Terraform modules for you to easily get started. See the deployment overview for a full list of supported deployment options.
Retool is packaged as a single stateless Docker container. The only dependency it has is a PostgreSQL database, which is used to store data such as user information, organization settings, audit logs, and applications.
To scale a Retool instance, you follow these high-level steps:
- Host the PostgreSQL database on an external system, e.g., AWS RDS. The separation of database and application allows you to independently scale and manage each service.
- Start multiple Retool containers that use the same Postgres database. The number of containers you deploy depends on your traffic and resource requirements. We recommend scaling the
api
container. You can do this by updating the replica count in your container orchestration service of choice. - Use a load balancer to route traffic between the Retool server containers to ensure high availability.
You should only run one replica of the jobs-runner
in each Retool environment because it runs database migrations and other background tasks that should only operate as a singleton. You can scale up as many replicas as necessary of the containers/pods that run the other service types.
Near-zero downtime upgrades
All MAJOR.MINOR
release upgrades, such upgrading from 3.148 to 3.196, perform database migrations. Patch release upgrades within the same MAJOR.MINOR
release, such as 3.196.1 to 3.196.2, generally do not.
General upgrade strategy
Each container or pod attempts to perform database migrations at startup after an upgrade. In multi-container setups, upgrades must be coordinated carefully to prevent container deadlocks and minimize downtime.
Database migrations are backward-compatible so older containers or pods can continue serving traffic while these migrations run.
- Docker-based
- Kubernetes + Helm
- Kubernetes (without Helm)
- Stop all containers except
jobs-runner
. - Start a new
jobs-runner
container using the latestMAJOR.MINOR.PATCH
image tag, such as3.196.1-stable
. - Wait for the database migrations to complete.
- Stop the old
jobs-runner
container. - Start the remaining containers using the new image.
- Verify all containers are healthy before routing traffic.
Kubernetes + Helm deployments support rolling upgrades that simplify the process. First, use helm upgrade
with the latest release tag. For example:
helm upgrade retool retool/retool \
--namespace retool \
--set image.tag=3.196.1-stable
During the upgrade process:
- A new
jobs-runner
pod is created and starts running migrations. - All other new pods start but wait for migrations to complete.
- Once all new pods are verified as healthy, the old pods are terminated automatically.
You can monitor the progress of database migrations for the job-runner
pod using kubectl
:
kubectl logs deployment/jobs-runner -n retool
For Kubernetes deployments without Helm, perform a manual rolling update.
- Patch the
jobs-runner
deployment to use the new image tag. - Wait for the new pod to start and complete database migrations.
- Patch the remaining deployments to use the new image.
- Monitor rollout status and ensure all pods are healthy:
You can monitor the progress of database migrations for the job-runner
pod using kubectl
:
kubectl rollout status deployment/<deployment-name> -n retool