Skip to main content

Scale your self-hosted deployment infrastructure

Learn how to scale your Retool deployment and infrastructure.

Retool's self-hosted Docker image consists of several containers required to run your deployment. Before you continue, you should review Retool's self-hosted architecture to understand which services need scaling as usage increases.

Scale your Retool deployment​

Retool requires you to self-host production deployments on Kubernetes with Helm. For new deployments, Retool provides Terraform blueprints to provision all required infrastructure and deploy automatically. For non-production testing and development, you can use Docker.

Retool is packaged as a single stateless Docker container. The only dependency is PostgreSQL (version 13 or later) for the platform database which is used to store data such as user information, organization settings, audit logs, and applications.

To scale a Retool instance, you follow these high-level steps:

  1. Host the PostgreSQL database on an external system, e.g., AWS RDS. The separation of database and application allows you to independently scale and manage each service.
  2. Start multiple Retool containers that use the same Postgres database. The number of containers you deploy depends on your traffic and resource requirements. We recommend scaling the api container. You can do this by updating the replica count in your container orchestration service of choice.
  3. Use a load balancer to route traffic between the Retool server containers to ensure high availability.
note

You should only run one replica of the jobs-runner in each Retool environment because it runs database migrations and other background tasks that should only operate as a singleton. You can scale up as many replicas as necessary of the containers/pods that run the other service types.

Configure shared Redis for replay protection​

Retool prevents SAML responses from being replayed by tracking a one-time-use marker (the response's unique identifier and expiry) for each response in its cache. Without Redis, each worker process keeps its own set of markers, so the protection applies per worker rather than across your whole deployment.

Most deployments run more than one worker process: each api container runs one worker per CPU (up to three) unless you set NUM_WORKERS, and you typically run multiple containers when scaling. To enforce replay protection consistently across every worker, configure a shared Redis cache using the backend cache variables: REDIS_HOST, REDIS_PORT, REDIS_PASSWORD, REDIS_DB, and REDIS_TLS. These are separate from the REDIS_USER_CACHE_* variables, which configure caching for workflow and resource queries.

A single-worker deployment (one container with NUM_WORKERS=1) can rely on the in-memory cache and does not require Redis for replay protection.

note

Requires Retool self-hosted 4.0.0 or later.

Agent sandbox pool​

The agent sandbox is the only Retool service that autoscales without manual configuration. The agent-sandbox-controller provisions and recycles short-lived agent-sandbox-job pods as editor sessions open and close.

How it scales​

Each active editing session requires approximately one agent-sandbox-job pod. When a builder opens an app, the controller assigns a pod from the prewarm pool or provisions a new one. When the session ends, the pod is recycled.

ComponentCPUMemoryReplicas
agent-sandbox-controller¼ core256 MiB1
agent-sandbox-proxy¼ core256 MiB1
agent-sandbox-job pods1 core2 GiB~1 per active editor
r2-worker1 core2 GiB1
js-executor2 cores6 GiB1 (up to 10 with autoscaling enabled)

The controller, proxy, and r2-worker do not need to be replicated. r2-worker and js-executor are separate rr.* components (not part of the sandbox pool itself), but both are enabled by the same rr.enabled master switch and their default resource requests are fixed regardless of sandbox activity — factor them into capacity planning alongside the sandbox components.

The Git server (rr.gitServer) runs in-process on the main backend pod by default and adds no separate fixed workload. It only becomes its own deployment, with its own resource footprint, if you explicitly set rr.gitServer.separate.enabled: true — and even then it falls back to the same resource defaults as the main backend rather than a sandbox-specific fixed cost.

Prewarm pool​

A pool of idle pods is kept ready to eliminate cold-start latency when builders open new sessions. The default is 5 pods:

values.yaml
rr:
  agentSandbox:
    controller:
      scaling:
        prewarmPoolSize: 5

Set prewarmPoolSize to 0 if startup latency is acceptable and reducing idle resource consumption is a priority. Builders will still get a sandbox session, but they'll wait for a pod to provision, which can take up to a minute or more depending on your environment.

Node autoscaling​

The sandbox controller scales the pod pool automatically, but it can only provision pods up to your cluster's current node capacity. For the sandbox to scale beyond existing node resources, your cluster needs node autoscaling enabled:

  • Amazon EKS: Cluster Autoscaler or Karpenter. Karpenter requires additional configuration; refer to the cloud-specific notes in the Kubernetes deployment guide.
  • GKE: GKE cluster autoscaler or Autopilot.
  • AKS: AKS cluster autoscaler.

Without node autoscaling, the sandbox pool is limited by your current cluster capacity. If that ceiling is reached, new editing sessions won't get a working sandbox until capacity is freed or additional nodes are provisioned.

Troubleshooting capacity exhaustion​

When the sandbox pool needs more pods than your cluster can schedule, a builder's sandbox session may silently fail to start. You can identify the cause by checking the pending pod's events for a scheduling failure and whether your node autoscaler attempted to add capacity.

First, review the events log for the pending pod and look for a scheduling failure.

kubectl describe pod <agent-sandbox-job-pod-name> -n retool

Look for a FailedScheduling event such as 0/N nodes are available: N Insufficient cpu.

If you're running the open-source Cluster Autoscaler as a Deployment in your cluster (the typical setup on EKS), check its logs to determine whether it attempted to add capacity:

kubectl logs deployment/cluster-autoscaler -n <autoscaler-namespace>

A NotTriggerScaleUp: max node group size reached entry means Cluster Autoscaler recognized the pending pod but your node group is already at its configured maximum, so no new node was added. Raise the node group's maximum size (or add capacity manually) to let the sandbox pool grow.

Karpenter and the managed autoscalers on GKE and AKS don't expose this reasoning through a kubectl logs-accessible deployment. For those, use the diagnostic tooling documented by your cloud provider to confirm whether a scale-up was attempted and why it didn't succeed.

Capacity planning​

Each sandbox session backs one app building thread in the app builder, and each session runs in its own pod. prewarmPoolSize, maxTotalJobs, and the idle timeout are the settings that manage cluster resources against your peak number of concurrent sessions. To estimate sandbox resource requirements at peak usage:

  1. Count the maximum number of builders you expect to be simultaneously active.
  2. Multiply by the maximum concurrent agent threads a single user can hold open (perUserSandboxLimit, default: 5) to get the maximum possible sandbox pod count if every user opens the maximum number of threads at once. Each thread holds its own sandbox session, not just one pod per active editor. Actual usage is often lower than this maximum; monitor real pod counts and adjust as needed rather than always provisioning for it.
  3. Add prewarmPoolSize (default: 5) for idle pool overhead.
  4. Multiply the pod total by per-pod resources: 1 CPU core and 2 GiB memory.
  5. Add fixed overhead for the agent-sandbox-controller, agent-sandbox-proxy (¼ core and 256 MiB each) and r2-worker (1 core and 2 GiB) — these run regardless of sandbox pod count.
  6. Add overhead for js-executor: 2 cores and 6 GiB per replica. This is 1 replica by default, but if you've enabled rr.jsExecutor.autoscaling (disabled by default), size for its configured maxReplicas (default: 10) instead, since the HPA can scale up to that ceiling under load. The Git server adds no separate fixed cost unless you've explicitly enabled rr.gitServer.separate.enabled.
  7. Convert the CPU and memory totals to a node count using your nodes' allocatable capacity. Refer to Convert pods to nodes.

For example, 20 concurrent builders at the default 5-thread limit is a maximum of 100 sandbox pods. With the default prewarm pool size, that's capacity for 105 pods (105 CPU cores and 210 GiB memory), plus 1.5 CPU cores and 2.5 GiB for the fixed controller, proxy, and r2-worker, plus 2 to 20 CPU cores and 6 to 60 GiB for js-executor depending on whether autoscaling is enabled.

The default maximum total sandbox job pods is 50 (rr.agentSandbox.controller.scaling.maxTotalJobs). Set it at or above the pod total from step 3. If that total exceeds the default, raise the cap explicitly; otherwise the pool will silently stop growing and editors beyond the cap won't get a working sandbox session:

values.yaml
rr:
  agentSandbox:
    controller:
      scaling:
        maxTotalJobs: 120  # raise from default 50 as needed
note

On Docker Compose, the sandbox pool is limited by the host VM's total memory and CPU. There is no node autoscaling on a single VM. If you expect more than a small number of concurrent builders, plan to size the VM accordingly or migrate to a Kubernetes deployment.

Convert pods to nodes​

Each sandbox pod reserves 1 CPU core and 2 GiB of memory, and only a node's allocatable capacity counts. Calculate allocatable capacity as:

Total for the node - system overhead

System overhead includes the requests of DaemonSet pods running on every node, such as log shippers and monitoring agents.

For example, a node with 4 vCPUs might have about 3.9 allocatable cores, which fits 3 sandbox pods per node.

Because each sandbox pod reserves a whole core, the remaining 0.9 cores can't fit another sandbox pod, though pods with smaller CPU requests can use that capacity. Larger nodes leave a smaller share of capacity that sandbox pods can't use.

Retool doesn't recommend a specific instance type. Use the steps in Capacity planning with the allocatable capacity of your own instance family.

Sizing examples​

The following table uses the default perUserSandboxLimit of 5 and prewarmPoolSize of 5. Node counts cover sandbox pods only, using the 3 sandbox pods per node from the previous example. Add capacity for the fixed components in steps 5 and 6 above.

Concurrent buildersPeak sandbox podsRaise maxTotalJobs above 50?Nodes at 3 pods each
530No10
1055Yes19
20105Yes35
40205Yes69

These figures are maximums that assume every builder opens the maximum number of threads. Monitor real pod counts and adjust.

Concurrency limits​

Each agent thread runs in a dedicated sandbox session (pod). A user can have up to 5 concurrent threads open by default, which means up to 5 sandbox sessions held simultaneously. This per-user limit is separate from the deployment-wide pod cap set by rr.agentSandbox.controller.scaling.maxTotalJobs in values.yaml.

When all of a user's sessions are active and they open a new thread, the thread fails to start rather than evicting an existing live session. If at least one session is idle, the system automatically reclaims the oldest idle session and assigns it to the new thread. The reclamation happens in the background and does not interrupt active work.

To raise or lower the per-user limit, set perUserSandboxLimit under rr.agentSandbox.controller.scaling:

values.yaml
rr:
  agentSandbox:
    controller:
      scaling:
        perUserSandboxLimit: 8  # raise from default 5 as needed

Idle timeout​

An unassigned sandbox shuts down automatically after a configurable idle period. The default is 10 minutes. When a session times out, Retool releases the reserved CPU and memory back to the pool.

In high-traffic deployments where usage patterns are unpredictable, reducing the idle timeout helps reclaim resources faster. A shorter timeout doesn't lower your peak, so size for the peak and use the timeout to shorten how long your cluster operates near it. Set sandboxIdleTimeoutMs under rr.agentSandbox.sandbox:

values.yaml
rr:
  agentSandbox:
    sandbox:
      sandboxIdleTimeoutMs: 300000  # 5 minutes, in milliseconds

Separately, sandboxGlobalLifetimeMs under the same rr.agentSandbox.sandbox block sets a hard ceiling on total sandbox lifetime regardless of activity (default is 2.5 hours). When this limit is reached, the sandbox is destroyed once the current agent loop ends, even if the session is still active.

Set sandboxGlobalLifetimeMs under rr.agentSandbox.sandbox:

values.yaml
rr:
  agentSandbox:
    sandbox:
      sandboxGlobalLifetimeMs: 3600000  # 1 hour, in milliseconds

Frontend caching with a service worker​

Retool installs a service worker in the browser that caches the application frontend and serves it on subsequent page loads. The service worker uses a stale-while-revalidate strategy: it returns the cached version of index.html immediately, then requests the latest version from Retool in the background. If the cached and latest versions differ, Retool automatically refreshes the page so users see the current build.

The cache is version-aware and automatically invalidates when you upgrade Retool. After an upgrade, users land on the previously cached version once, then receive the new version on the next request without needing to clear their browser cache. Users can bypass the cache with a hard refresh (Cmd+Shift+R / Ctrl+Shift+R).

The service worker is enabled by default in self-hosted Retool 3.334 and later. Admins can disable it from Settings > Beta if needed to diagnose page load issues.

Near-zero downtime upgrades​

All MAJOR.MINOR release upgrades, such upgrading from 4.34 to 4.86, perform database migrations. Patch release upgrades within the same MAJOR.MINOR release, such as 4.86.1 to 4.86.2, generally do not.

General upgrade strategy​

Each container or pod attempts to perform database migrations at startup after an upgrade. In multi-container setups, upgrades must be coordinated carefully to prevent container deadlocks and minimize downtime.

Best practice

Database migrations are backward-compatible so older containers or pods can continue serving traffic while these migrations run.

  1. Stop all containers except jobs-runner.
  2. Start a new jobs-runner container using the latest MAJOR.MINOR.PATCH image tag, such as 4.86.1-stable.
  3. Wait for the database migrations to complete.
  4. Stop the old jobs-runner container.
  5. Start the remaining containers using the new image.
  6. Verify all containers are healthy before routing traffic.