Skip to main content

Upgrade from Retool 3.x to 4.0 (Docker Compose)

Upgrade an existing self-hosted Retool 3.x Docker Compose deployment to Retool 4.0.

Additional requirements introduced in Retool 4.0 include more services and resources. If you have an existing Docker Compose-based staging deployment of self-hosted Retool running 3.x, this guide explains how to upgrade and add the necessary services.

The upgrade will likely require a larger VM. Plan for a maintenance window and read the whole guide before you start.

This guide is for non-production deployments only. For production, migrate to a Kubernetes deployment using Terraform blueprints.

note

Retool's officially supported deployment configurations are Helm on Kubernetes and the Docker Compose blueprints. This guide is provided for convenience, but Retool does not provide support guarantees for deployments outside of those configurations. If you are running Docker Compose in production, contact your account team about migrating to a Kubernetes deployment.

Reference configuration files live in the tryretool/retool-onpremise repository.

What changed from Retool 3.x​

ChangeDetails
New servicesThe reference compose.yaml adds services that Retool 3.x doesn't have, including js-executor, agent-sandbox-controller, agent-sandbox-proxy, r2-agent-worker, and mcp. They start automatically after the upgrade.
Blob storage is requiredObject storage (S3, Azure Blob, or GCS) is a platform requirement. The reference config includes a bundled MinIO container for non-production use, but production deployments should use a cloud object store.
VM size increase requiredThe new services require significantly more CPU and memory. Plan for at least 8 vCPUs and 64 GiB on the VM.
Temporal required for agentsThe agent Temporal worker requires a Temporal cluster. If you do not have one, configure Temporal Cloud after deployment.
MCP server requires manual configuration on upgradeThe mcp service needs three docker.env variables that install.sh only writes on a fresh install. Add them manually (refer to MCP server below) or MCP clients can't authenticate.

Before you start​

Use this checklist as you prepare the upgrade to ensure you have everything ready.

Follow the standard upgrade best practices in addition to the steps here. Perform the migration during off-peak hours.

Choose your target version​

Keeping your deployment up-to-date ensures your organization has access to newer features and security updates. Running a deployment that is multiple releases behind the current one makes each upgrade riskier. Changelogs accumulate more breaking changes and deprecations, incremental migration becomes harder to test, and a large version jump can skip over a release with a required migration step.

If your deployment is more than one stable release behind, upgrade through each stable release in sequence rather than jumping directly to your target version. For example, to upgrade from 3.300 to 4.34:

3.300 -> 3.334 -> 4.0 -> 4.34

Verify each installation before proceeding to the next release. Review the Stable releases or Edge releases page and read all changelogs between your current version and your target version before starting.

Upgrading sequentially through each release is another reason to keep your deployment current: the fewer releases you're behind, the fewer intermediate upgrades you need to perform.

Services overview​

This upgrade adds several services to your stack. Some work out of the box; others require a decision or external infrastructure before you start.

ServiceIncluded in reference configAction required
JS ExecutorYes, added by defaultNone; works out of the box
Agent SandboxYes, added by defaultNone; works out of the box
Blob storageYes (bundled MinIO)Recommended: switch to a cloud object store for non-trivial use
MCP serverYes, added by defaultRequired: add three docker.env variables manually (refer to MCP server below)
Temporal + agent Temporal workerDepends on existing setupChoose a Temporal path (refer to Temporal below)

For a full description of each service, refer to the architecture overview.

Blob storage​

The reference configuration includes a MinIO container that provides working blob storage with no external dependencies. For development and POC environments, no additional configuration is needed. For any meaningful use, switch to a cloud object store (S3, Azure Blob, or GCS). The bundled MinIO container uses local VM disk, which will eventually run out of space.

Temporal​

The agent Temporal worker requires a Temporal cluster:

  • You already have a Temporal cluster: Use your existing cluster, even if it runs locally in your stack. Point the new worker at the same cluster using the Temporal environment variables in Configure the new environment variables.
  • You do not have Temporal: Retool recommends Temporal Cloud. Configure it via the in-product setup after deployment (click the Workflows nav item). Temporal Cloud requires outbound egress on ports 443 and 7233. Refer to Temporal egress.

CPU and memory requirements​

These services consume meaningful additional CPU and memory. You will most likely need to resize your VM before starting.

For a Retool 4.0 deployment, plan for at least 8 vCPUs and 64 GiB memory. If your current VM is smaller than this, resize it before proceeding.

Pull the latest configuration files​

The updated configuration adds the new services to your Compose stack. How you pull it depends on how you manage the repository.

Before you pull, note the images your postgres and retooldb-postgres services run. You need them to preserve your customizations:

sudo docker compose ps postgres retooldb-postgres

If you cloned the repository​

git stash
git fetch origin
git checkout master
git pull --rebase origin master

# Review and re-apply still necessary changes
git stash show -p

git stash sets aside your local edits, such as the version in your Dockerfile, because git pull --rebase doesn't run with uncommitted changes. git stash show -p lists what it set aside, so you can re-apply what you still need in the following steps.

If you did not clone the repository​

Download the latest directly:

curl -L https://github.com/tryretool/retool-onpremise/archive/refs/heads/master.zip \
  -o retool.zip
unzip retool.zip

Preserve your customizations​

Do not overwrite your existing docker.env, Dockerfile, or Compose file wholesale. Reconcile the reference files you pulled against your current setup to keep your license key, SSL settings, and any custom environment variables.

The reference Compose file is compose.yaml. Older instances might use docker-compose.yml instead. If your directory has both files, Compose uses compose.yaml, so move your customizations into it and delete docker-compose.yml.

PostgreSQL can't start on data from an older major version. If the reference compose.yaml runs postgres or retooldb-postgres on a different PostgreSQL major version than the image you noted, set that service's image back to the one you noted. If your PostgreSQL version is older than the minimum Retool supports, upgrade PostgreSQL before you upgrade Retool.

Update your version​

Set the new Retool version in your Dockerfile. The stable release is 4.34.15-stable:

ARG VERSION=X.Y.Z-stable  # use the tag shown above
ARG CLASSIC_VERSION=X.Y.Z-stable  # same version, without the -hardened suffix

FROM tryretool/agent-sandbox-service:${CLASSIC_VERSION} AS agent-sandbox
FROM tryretool/code-executor-service:${VERSION} AS code-executor
FROM tryretool/js-executor-service:${CLASSIC_VERSION} AS js-executor
FROM tryretool/backend:${VERSION}

If you use hardened images, set VERSION to a hardened tag (for example, X.Y.Z-stable-hardened) for backend and code-executor-service. Hardened variants aren't available for agent-sandbox-service or js-executor-service, so keep CLASSIC_VERSION on the non-hardened tag for those two.

Replace X.Y.Z-stable with the stable version from the release notes. If you use a separate CodeExecutor.Dockerfile, update it to the same version.

Configure the new environment variables​

install.sh is only for fresh installs. When docker.env already exists, it stops without changing it, so it can't add the new variables below to an existing instance. Add them to your existing docker.env manually, alongside your current values, including ENCRYPTION_KEY and LICENSE_KEY.

Review docker.env and confirm the following are set correctly.

Blob storage​

Set the provider and credentials for your chosen object store:

RR_BLOB_STORAGE_PROVIDER=s3
RR_DEFAULT_S3_BUCKET=<your-bucket>
RR_DEFAULT_S3_ACCESS_KEY_ID=<your-key-id>
RR_DEFAULT_S3_SECRET_ACCESS_KEY=<your-secret-key>
RR_DEFAULT_S3_REGION=<your-region>
RR_DEFAULT_S3_ENDPOINT=             # optional; for S3-compatible stores

If you are using the bundled MinIO for development, no additional blob storage configuration is needed.

MCP server​

The updated configuration adds an mcp service that serves Retool's MCP server at /mcp. It requires HTTPS and three variables in docker.env that install.sh only writes on a fresh install:

# Public host serving OAuth metadata and /api/oauth2/* (your Retool domain, no scheme)
OAUTH_MAIN_DOMAIN=<your-domain>
# Public base URL the MCP server is reachable on, used for import upload URLs
MCP_SERVICE_EXTERNAL_URL=https://<your-domain>
# Shared secret the MCP server presents to the backend's /api/oauth2/introspect.
# Read by both the api and mcp services, so it must match on both.
OAUTH_INTROSPECTION_AUTH_TOKEN=<generate-a-random-value>

Without these variables, the mcp container still starts, but MCP clients fail to authenticate. If you don't want to expose an MCP server, remove the mcp service block from your Compose file instead.

Temporal​

The reference compose.yaml includes temporal.yaml by default, which runs a local Temporal cluster alongside Retool. If your instance already uses a Temporal cluster, keep using it. Otherwise, Retool recommends Temporal Cloud.

  • Temporal Cloud (recommended): Comment out the include block in your Compose file and any WORKFLOW_TEMPORAL_* variables in docker.env. If you haven't set up Temporal Cloud yet, do it in the Retool UI after deployment.

  • Existing local cluster: Keep the include block, and confirm docker.env points to the local cluster:

    WORKFLOW_TEMPORAL_CLUSTER_FRONTEND_HOST=temporal
    WORKFLOW_TEMPORAL_CLUSTER_FRONTEND_PORT=7233

  • Existing self-managed cluster: Comment out the include block in your Compose file, and confirm the connection variables are present in docker.env:

    WORKFLOW_TEMPORAL_CLUSTER_FRONTEND_HOST=<host>
    WORKFLOW_TEMPORAL_CLUSTER_FRONTEND_PORT=<port>
    WORKFLOW_TEMPORAL_CLUSTER_NAMESPACE=<namespace>

    If your cluster runs on your own Temporal Cloud account, also set the WORKFLOW_TEMPORAL_TLS_* variables for mTLS.

Agent sandbox​

Without these variables, the agent-sandbox-controller and agent-sandbox-proxy services fail to start.

The agent-sandbox-controller and agent-sandbox-proxy services share state (sandbox assignments, the prewarm pool, and the leader lease) through a Postgres-backed store that uses the existing postgres service instead of requiring a separate Redis deployment. Add these variables to docker.env to configure Postgres usage:

STATE_BACKEND=postgres
RR_AGENT_PUBSUB_BACKEND=postgres
AGENT_EXECUTOR_POSTGRES_URL=postgres://<user>:<password>@<postgres-host>:<postgres-port>/<database>
AGENT_EXECUTOR_POSTGRES_SCHEMA=agent_executor
AGENT_EXECUTOR_ENCRYPTION_KEY=<64-character-hexadecimal-value>
AGENT_EXECUTOR_JWT_PRIVATE_KEY=<your-ES256-private-key-pem>
AGENT_EXECUTOR_JWT_PUBLIC_KEY=<your-ES256-public-key-base64>

  • Use the same Postgres host, port, credentials, and database as your existing POSTGRES_* variables in docker.env. If you're using the bundled postgres container, the host is postgres and the port is 5432; if you've configured an external database, use its POSTGRES_HOST and POSTGRES_PORT values instead. The user must have CREATEDB privilege so the agent sandbox controller can initialize its required database schema.

  • AGENT_EXECUTOR_ENCRYPTION_KEY must be a 64-character hexadecimal value (e.g., 0a1b2c3d4e5f...). Generate one with openssl rand -hex 32.

  • Preserve the encryption key. If you need to restore or recreate the instance, use the same key or the sandbox state becomes unreadable.

  • Generate AGENT_EXECUTOR_JWT_PRIVATE_KEY/AGENT_EXECUTOR_JWT_PUBLIC_KEY as an ES256 (EC P-256) PEM pair:

    openssl genpkey -algorithm EC -pkeyopt ec_paramgen_curve:P-256 -out ae_private.pem
    openssl ec -in ae_private.pem -pubout -out ae_public.pem

  • AGENT_EXECUTOR_JWT_PRIVATE_KEY must be the raw PEM contents, single-quoted so docker.env preserves the line breaks:

    AGENT_EXECUTOR_JWT_PRIVATE_KEY='-----BEGIN EC PRIVATE KEY-----
    ...
    -----END EC PRIVATE KEY-----'

  • AGENT_EXECUTOR_JWT_PUBLIC_KEY must be the PEM contents base64-encoded to a single line. Run:

    base64 -i ae_public.pem | tr -d '\n'

    Then set docker.env to the resulting string, for example AGENT_EXECUTOR_JWT_PUBLIC_KEY=LS0tLS1CRUdJTi....

Apply the AppArmor profiles​

On Linux hosts with AppArmor enabled, such as Ubuntu, the agent sandbox needs Retool's docker-default AppArmor profile. Without it, Docker's built-in version of that profile stops the agent sandbox from starting, so the app builder can't open apps.

Run the setup script from your retool-onpremise directory to install it:

./scripts/setup-docker-apparmor.sh

The script uses sudo if you aren't running as root. Hosts without AppArmor skip this step automatically.

On Ubuntu 24.04 and later, Code Executor also needs the usr.bin.nsjail profile. If your instance was installed before install.sh added it, follow the nsjail steps in the AppArmor README, which also explains what each profile changes.

Start the updated deployment​

Rebuild the images from your updated Dockerfile and bring the stack up:

sudo docker compose up -d --build

Without --build, Compose keeps using the images it built for your current version. The initial start can take several minutes as Compose builds the new service images and first-time setup runs.

Verify all services are healthy:

sudo docker compose ps

Confirm these services (JS Executor, Agent Sandbox, the agent Temporal workers agent-worker and r2-agent-worker, and MinIO if used) are running alongside your existing containers. If any service is restarting repeatedly or being OOM-killed, the VM likely needs more memory (refer to CPU and memory requirements).

To investigate a failing service:

sudo docker compose logs <service-name>

Complete setup in the Retool UI​

Once the deployment is healthy, an admin user should complete the following in the Retool interface:

  1. Enable Temporal, if you're setting up Temporal Cloud. Navigate to Workflows and follow the in-product setup steps. A valid license key is required.
  2. Configure your organization to use the new app builder.
  3. Enable Retool AI features. Refer to Enable Retool AI features for the required settings and AI resource prerequisite.

Rollback​

If the migration does not go as planned:

  1. Run sudo docker compose down to stop the stack.

  2. Remove the agent sandbox containers. They keep running after the stack stops, and nothing in Retool 3.x removes them:

    sudo docker ps -aq --filter label=com.retool.agent-executor.role=sandbox | xargs -r sudo docker rm -f

  3. Restore your Dockerfile to the previous version, and restore your backed-up docker.env and Compose file. If you restore docker-compose.yml, delete compose.yaml, because Compose uses compose.yaml when both files exist.

  4. Restore your PostgreSQL snapshot if the database was modified.

  5. Bring the previous deployment back up: sudo docker compose up -d --build.