Self-hosted Retool at scale

This guide provides a technical overview of how to deploy, manage, and scale Retool for enterprise use cases.

Retool can be self-hosted in any containerized environment: locally, on a virtual machine, or through container orchestration services like Kubernetes. Even though a single virtual machine can support users and Retool apps, we recommend you deploy Retool instances through a container orchestration service. This method of deployment makes it easier to manage, upgrade, and scale.

We provide templates and starting points for the following orchestration services:

• Amazon Elastic Container Service and Fargate
• Helm
• Kubernetes
• Terraform

Retool’s self-hosted architecture

Retool's self-hosted Docker image comes with containers that are responsible for running the web app, storing instance-level data like users and permissions, and running jobs like database migrations. The docker-compose file for Retool Self-hosted defines three main containers: api, postgres, and jobs-runner.

api

This is the primary container that serves the Retool web application. It handles most of the logic for interacting with the app.

postgres

Retool uses a Postgres container to store apps, users, permissions, and other necessary information. We cannot access this information on self-hosted deployments.

We recommend externalizing this container (e.g., to Amazon RDS) so the Retool application can be completely stateless, and to make it easier to take snapshots and monitor performance. If you deploy using one of the orchestration services above, our templates provided specify the inputs needed to host the database separately.

jobs-runner

The jobs-runner container is responsible for database migrations, syncing protected apps, and other background tasks. This container also contains logs related to these tasks.

jobs-runner is dependent on the Postgres container as it runs operations on the Retool Postgres database.

🚧

There should only ever be one jobs-runner container instance. It is responsible for syncing changes between any api containers and the postgres instance.

When to deploy multiple Retool instances

As you begin Retool app development, we recommend spinning up one or two instances if most of the initial users will be building apps. As teams get comfortable with their development workflows and end-users are spending more time using Retool apps, it’s important to get a pulse and understand whether it makes sense to scale out new instances.

Separate Retool development and production environments

You might have development endpoints where editors can safely build and test apps on non-production data. If so, it's recommended to mirror a typical Software Development Life Cycle (SDLC):

• Deploy a development instance of Retool in the development VPC.
• Deploy a production instance of Retool in the production VPC.

This ensures your development Retool instance can only point to development endpoints and helps organize different user types across your organization.

Our Source Control feature lets you safely promote application changes between instances via a Git-based workflow. You can only have one Git repository associated with an instance—all pull requests must go through a single group of reviewers.

Isolate team development workflows

We recommend you split out Retool instances for each team if:

• You operate under a full-stack or Spotify model and have many autonomous teams building tooling on their own.
• You don't have shared resources or endpoints across teams, but teams want to manage their own development workflows independently.

Legal and compliance purposes

If you are launching or operate into a market that requires network traffic to be isolated to specific regions, you may need to isolate your Retool instance as well. In other cases, a production instance might need to be locked down to be read-only (i.e., no editors can build apps).

Testing new features and upgrades

If you want to safely test out a new beta feature or instance-wide setting (e.g., a new SSO configuration), you can spin up a separate Retool instance for this purpose.

When to use a single Retool instance

In general, you should only deploy the instances you need. If you’re starting out with a small, defined set of use cases for one team, or if you don’t need to isolate Retool instances by VPC, you can start with a single instance.

Retool allows you to define multiple environments within a single instance so that you can safely build, run, and test apps for your end-users.

Scale your Retool instance

If you start to see performance issues when many users (typically 100+) are using a single Retool instance, we recommend scaling the api container. You can do this by updating the replica count in your container orchestration service of choice.