Docker Kubernetes Manifest Installation

Retool does not recommend deploying to physical machines for team development and deployments. Consider deploying to managed Kubernetes services such as Amazon EKS, Azure Kubernetes Service, or Google Kubernetes Engine.

The following example focuses on using Kubernetes on Docker Desktop for education purposes.

Requirements

To deploy Retool using Docker, you need:

A Retool license key, which you can obtain from my.retool.com or your Retool account manager.
A working installation of Docker desktop.
Kubernetes enabled on Docker Desktop.
Computer with 16 GB of RAM.

Self-hosted deployments also require the following and are configured within Docker Desktop Settings/Resources:

OS Supported by Docker Desktop configured with:
- 12GiB memory
- 6 vCPUs
- 60GiB storage.

To confirm you have sufficient resources, open up Docker Desktop and go to Settings > Resources. This console will allow you to set the values it can access.

Docker Desktop Resources

In addition confirm that you have Kubernetes enabled by going to Settings > Kubernetes.

Docker Desktop Kubernetes

This example uses a Mac M1 Pro processor, 16 GB of RAM with Docker Desktop for Mac version 4.37.0 with Kubernetes version 1.30.5.

Confirm Setup

Once you have performed the previous steps, please confirm that Docker Desktop Kubernetes is enabled. Execute the following command:

kubectl get nodes

The response should be something similar to:

NAME             STATUS   ROLES           AGE   VERSION
docker-desktop   Ready    control-plane   22d   v1.30.5

If this doesn't return something similar you may be pointing at another Kubernetes installation. Execute the following command to see what context you are using:

kubectl config get-contexts

This should show docker-desktop selected with a star symbol. If it doesn't you can redirect your context by using the following command:

kubectl config use-context docker-desktop

Steps

Run the following command to download the installation and Kubernetes manifests.

curl -L -O https://github.com/tryretool/retool-onpremise/archive/master.zip && unzip master.zip
cd retool-onpremise-master/kubernetes

Once the command is run, the directory should have the following contents:

% ls -la
total 56
drwxr-xr-x   9 user  staff   288 Dec  3 15:48 .
drwxr-xr-x  21 user  staff   672 Dec  3 15:48 ..
-rw-r--r--   1 user  staff  1586 Dec  3 15:48 retool-code-executor-container.yaml
-rw-r--r--   1 user  staff  2811 Dec  3 15:48 retool-container.yaml
-rw-r--r--   1 user  staff  2062 Dec  3 15:48 retool-jobs-runner.yaml
-rw-r--r--   1 user  staff  1429 Dec  3 15:48 retool-postgres.yaml
-rw-r--r--   1 user  staff   546 Dec  3 15:48 retool-secrets.template.yaml
-rw-r--r--   1 user  staff  2523 Dec  3 15:48 retool-workflows-backend-container.yaml
-rw-r--r--   1 user  staff  2587 Dec  3 15:48 retool-workflows-worker-container.yaml

Secret Updates

Copy the retool-secrets.template.yaml to retool-secrets.yaml.

cp retool-secrets.template.yaml retool-secrets.yaml

Edit retool-secrets.yaml to enter in the Retool license key. Opening the yaml document will appear as follows:

apiVersion: v1
kind: Secret
metadata:
  name: retoolsecrets
type: Opaque
data:
  jwt_secret: {{ random base64 encoded string to sign jwt tokens }}
  encryption_key: {{ random base64 encoded string to encrypt database credentials }}
  postgres_password: {{ random base64 encoded string to set as the internal retool db password }}
  license_key: {{ base64 encoded string of the license key Retool will provide you }}
  google_client_id: {{ google client id encoded in base64 }}
  google_client_secret: {{ google client secret encoded in base64 }}

Use the following commands to generate random unique base64 entries for jwt_secret, encryption_key.

openssl rand -base64 16 //take result and put into jwt_secret line
openssl rand -base64 16 //take result and put into encryption_key line

Take the Retool license key and use the following command to generate a base64 encoded string:

echo -n "ENTER RETOOL LICENSE KEY HERE" | openssl base64

Create base64 encoded value for the Postgres DB Password, use the following command:

echo -n "ENTER POSTGRES PASSWORD HERE" | openssl base64

NOTE: The following values are not required for this POC, just place random text here and update the retool-secrets.yaml. These entries are examined on startup by the jobs-runner and api pods as part of their environment settings.

Create base64 encoded value for the Google Client ID, use the following command:

echo -n "Google Client ID HERE" | openssl base64

Create base64 encoded value for the Google Client Secret, use the following command:

echo -n "Google Client Secret HERE" | openssl base64

Image updates to Stable or Edge Release

Edit the retool-jobs-runner.yaml and retool-container.yaml to have an image id specified. Determine the release you wish to use for functionality. For example we will use the Stable 3.114 release available at Docker Hub:

image: tryretool/backend:3.75.17-stable

Resource Limit updates

It is important that you are aware that the manifests have CPU/Memory limits defined and this could cause OOMKilled (Out of Memory) errors when deploying on a local laptop in their default settings. For this example, we need to reduce the default memory limits in the manifests (YAML files) by adjusting the resources sections:

resources:
  limits:
    cpu: 2
    memory: 2Gi
  requests:
    cpu: 1
    memory: 1Gi

Here is a recommended list of resource limits for this example that has been tested:

manifest	resources limits
retool-code-executor-container.yaml	limits / cpu & memory = 2/2Gi, requests / cpu & memory 1/1Gi
retool-container.yaml	limits / cpu & memory = 1/5Gi, requests / cpu & memory = 1/1Gi
retool-jobs-runner.yaml	limits / cpu & memory = 2/2Gi, requests / cpu & memory 1/1Gi
retool-workflows-backend-container.yaml	limits / cpu & memory = 2/2Gi, requests / cpu & memory 1/1Gi
retool-workflows-worker-container.yaml	limits / cpu & memory = 2/2Gi, requests / cpu & memory 1/1Gi

Retool Secrets

Deploy the secrets manifest using the following command:

kubectl apply -f retool-secrets.yaml

This returns:

secret/retoolsecrets created

Check to make the sure the secrets were applied using the following command:

kubectl get secrets

This returns:

NAME            TYPE     DATA   AGE
retoolsecrets   Opaque   4      32s

postgres Pod

Deploy the postgres manifest using the following command:

kubectl apply -f retool-postgres.yaml

This returns:

persistentvolumeclaim/postgres-pvc created
deployment.apps/postgres created
service/postgres configured

Check to make sure the postgres deployment / pod have successfully deployed using the following command:

kubectl get pods

This returns:

NAME                        READY   STATUS    RESTARTS   AGE
postgres-69898447dd-9zs2v   1/1     Running   0          31s

Also check the logs to make sure that the postgres pod is accepting connections.

kubectl logs <postgres pod name>

This returns:

The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
...
2024-12-19 20:47:27.117 UTC [1] LOG:  database system is ready to accept connections

jobs-runner Pod

Deploy the jobs-runner manifest using the following command:

kubectl apply -f retool-jobs-runner.yaml

This returns:

deployment.apps/jobs-runner created

Check to make sure the jobs-runner deployment / pod have successfully deployed using the following command:

kubectl get pods

This returns:

NAME                           READY   STATUS    RESTARTS   AGE
jobs-runner-6fd55db6dc-dr989   1/1     Running   0          17s
postgres-775c447bdc-l97sg      1/1     Running   0          5m4s

Also check the logs to make sure that the jobs-runner pod is executing migrations.

kubectl logs <jobs-runner pod name>

This returns:

wait-for-it.sh: waiting for postgres:5432 without a timeout
wait-for-it.sh: postgres:5432 is available after 0 seconds
not untarring the bundle
sed: can't read ./dist/mobile/*.js: No such file or directory
{"level":"info","message":"[process service types] JOBS_RUNNER","timestamp":"2024-12-19T20:52:17.345Z"}
Acquiring lock to run migrations...
Acquired lock
Running database migrations...
...
{"level":"info","message":"Jobs runner checking for changes.","pid":17,"source":"JOBS_RUNNER","timestamp":"2024-12-19T20:52:54.809Z"}
{"level":"info","message":"No organizations found, sleeping...","pid":17,"source":"SOURCE_CONTROL_DEPLOYMENT","timestamp":"2024-12-19T20:52:54.820Z"}

code-executor Pod

Deploy the code-executor manifest using the following command:

kubectl apply -f retool-code-executor-container.yaml

This returns:

deployment.apps/code-executor created
service/code-executor created

Check to make sure the jobs-runner deployment / pod have successfully deployed using the following command:

kubectl get pods

This returns:

NAME                           READY   STATUS    RESTARTS   AGE
code-executor-d576d4d6-44tlv   1/1     Running   0          54s
jobs-runner-6fd55db6dc-dr989   1/1     Running   0          3m25s
postgres-775c447bdc-l97sg      1/1     Running   0          8m12s

Also check the logs to make sure that the code-executor pod is running.

kubectl logs <code-executor pod name>

This returns:

Running code executor container in privileged mode.
iptables: Disallowing link-local addresses
{"level":"info","message":"Not configuring StatsD...","timestamp":"2025-01-09T13:07:52.358Z"}
{"level":"info","message":"Installing request logging middleware...","timestamp":"2025-01-09T13:07:52.378Z"}
{"level":"info","message":"Checking if nsjail can be used","timestamp":"2025-01-09T13:07:52.382Z"}
{"level":"info","message":"Populating environment cache","timestamp":"2025-01-09T13:07:52.386Z"}
...

api Pod

Deploy the container manifest using the following command:

kubectl apply -f retool-container.yaml

This returns:

deployment.apps/api created
service/api unchanged
persistentvolumeclaim/retool-pvc created

Check to make sure the api deployment / pod have successfully deployed using the following command:

kubectl get pods

This returns:

NAME                           READY   STATUS    RESTARTS   AGE
api-f4cd96689-7r8qj            1/1     Running   0          21s
code-executor-d576d4d6-44tlv   1/1     Running   0          5m14s
jobs-runner-6fd55db6dc-dr989   1/1     Running   0          7m45s
postgres-775c447bdc-l97sg      1/1     Running   0          12m

Also check the logs to make sure that the api pod is receiving health checks with a HTTP 200 status code from K8S. It will take several minutes for the K8S health check to display.

kubectl logs <api pod name>

This returns:

wait-for-it.sh: waiting for postgres:5432 without a timeout
wait-for-it.sh: postgres:5432 is available after 0 seconds
not untarring the bundle
sed: can't read ./dist/mobile/*.js: No such file or directory
{"level":"info","message":"[process service types] MAIN_BACKEND, DB_CONNECTOR, DB_SSH_CONNECTOR","timestamp":"2025-01-09T13:12:56.415Z"}
Database migrations are up to date.
{"level":"info","message":"Not configuring StatsD...","timestamp":"2025-01-09T13:13:02.014Z"}
{"level":"info","message":"Not configuring StatsD...","timestamp":"2025-01-09T13:13:02.017Z"}
{"level":"info","message":"Not configuring Sentry...","timestamp":"2025-01-09T13:13:02.054Z"}
{"level":"info","message":"Not configuring StatsD...","timestamp":"2025-01-09T13:13:02.055Z"}
{"level":"info","message":"Running node v18.18.2","timestamp":"2025-01-09T13:13:02.071Z"}
{"0":"--openssl-legacy-provider","1":"--no-experimental-fetch","level":"info","message":"ARGV:","timestamp":"2025-01-09T13:13:02.074Z"}
...
{"level":"info","message":"license check http response code: 200","timestamp":"2025-01-09T13:13:07.973Z"}
...

workflows-api Pod

Deploy the retool-workflows-backend-container.yaml manifest using the following command:

kubectl apply -f retool-workflows-backend-container.yaml

This returns:

deployment.apps/workflows-api created
service/workflows-api created

Check to make sure the workflows-api deployment / pod have successfully deployed using the following command:

kubectl get pods

This returns:

NAME                             READY   STATUS    RESTARTS   AGE
api-f4cd96689-7r8qj              1/1     Running   0          5m16s
code-executor-d576d4d6-44tlv     1/1     Running   0          10m
jobs-runner-6fd55db6dc-dr989     1/1     Running   0          12m
postgres-775c447bdc-l97sg        1/1     Running   0          17m
workflows-api-5fd56cbb44-pn4lb   1/1     Running   0          40s

Also check the logs to make sure that the workflows-api pod is running.

kubectl logs <workflows-api pod name>

This returns:

wait-for-it.sh: waiting for postgres:5432 without a timeout
wait-for-it.sh: postgres:5432 is available after 0 seconds
not untarring the bundle
sed: can't read ./dist/mobile/*.js: No such file or directory
{"level":"info","message":"[process service types] MAIN_BACKEND, DB_CONNECTOR, DB_SSH_CONNECTOR","timestamp":"2025-01-09T13:18:50.257Z"}
Database migrations are up to date.
{"level":"info","message":"Not configuring StatsD...","timestamp":"2025-01-09T13:18:54.255Z"}
{"level":"info","message":"Not configuring StatsD...","timestamp":"2025-01-09T13:18:54.257Z"}
{"level":"info","message":"Not configuring Sentry...","timestamp":"2025-01-09T13:18:54.305Z"}
{"level":"info","message":"Not configuring StatsD...","timestamp":"2025-01-09T13:18:54.306Z"}
{"level":"info","message":"Running node v18.18.2","timestamp":"2025-01-09T13:18:54.316Z"}
{"0":"--openssl-legacy-provider","1":"--no-experimental-fetch","level":"info","message":"ARGV:","timestamp":"2025-01-09T13:18:54.317Z"}
{"level":"info","message":"Node.js heap size limit: 5168 MiB","timestamp":"2025-01-09T13:18:54.318Z"}
{"level":"info","message":"Initialized general rate limiter: 60 attempts every 60 seconds","timestamp":"2025-01-09T13:18:55.563Z"}
...
{"level":"info","message":"license check http response code: 200","timestamp":"2025-01-09T13:18:57.870Z"}
{"level":"info","message":"License key feature flag overrides: {\"allowAppEditorAccessWhileVersionControlLocked\":true,\"emailSupportOverride\":true,\"fastTableWidget\":true,\"retoolMobile\":true,\"statusWidget\":true,\"tableV2\":true,\"usageAnalytics\":true,\"uuidAppNavigation\":true,\"workflowsAsyncTask\":true,\"workflowsBetaAccess\":true,\"workflowsEnableTemporal\":true,\"workflowsPythonBlocks\":true,\"workflowsRemoveBillableRunsFreeTierCapForCommitted\":true,\"workflowsUnlimitedActiveWorkflows\":true,\"workflowsWebhookReturns\":true}","timestamp":"2025-01-09T13:18:57.882Z"}
{"level":"info","message":"Updated license status from licensing server","timestamp":"2025-01-09T13:18:57.899Z"}
{"level":"info","message":"[Master] Detected 6 cpus, starting 3 workers","timestamp":"2025-01-09T13:18:57.902Z"}
...

workflow-worker Pod

Deploy the workflow-worker manifest using the following command:

kubectl apply -f retool-workflows-worker-container.yaml

This returns:

deployment.apps/workflow-worker created

Check to make sure the workflow-worker deployment / pod have successfully deployed using the following command:

kubectl get pods

This returns:

NAME                               READY   STATUS    RESTARTS   AGE
api-f4cd96689-7r8qj                1/1     Running   0          10m
code-executor-d576d4d6-44tlv       1/1     Running   0          15m
jobs-runner-6fd55db6dc-dr989       1/1     Running   0          18m
postgres-775c447bdc-l97sg          1/1     Running   0          22m
workflow-worker-5587bff55f-kkl9m   1/1     Running   0          56s
workflows-api-5fd56cbb44-pn4lb     1/1     Running   0          6m12

Also check the logs to make sure that the workflow-worker pod is running.

kubectl logs <workflow-worker pod name>

This returns:

wait-for-it.sh: waiting for postgres:5432 without a timeout
wait-for-it.sh: postgres:5432 is available after 0 seconds
not untarring the bundle
sed: can't read ./dist/mobile/*.js: No such file or directory
{"level":"info","message":"[process service types] WORKFLOW_TEMPORAL_WORKER","timestamp":"2025-01-09T13:20:54.126Z"}
{"level":"info","message":"Not configuring StatsD...","timestamp":"2025-01-09T13:20:57.579Z"}
{"level":"info","message":"Not configuring StatsD...","timestamp":"2025-01-09T13:20:57.581Z"}
(node:17) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
{"level":"info","message":"installing temporal runtime","timestamp":"2025-01-09T13:20:57.661Z"}
{"level":"info","message":"Rechecking license status...","timestamp":"2025-01-09T13:20:57.721Z"}
{"level":"info","message":"license check http response code: 200","timestamp":"2025-01-09T13:20:58.544Z"}

When the container pod is deployed, it creates a service of type LoadBalancer. To determine the Kubernetes service values for the api service use the following command:

kubectl get svc

This returns:

NAME            TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
api             LoadBalancer   10.111.123.145   localhost     3000:32692/TCP   120m
code-executor   ClusterIP      10.96.67.5       <none>        80/TCP           17m
kubernetes      ClusterIP      10.96.0.1        <none>        443/TCP          2d6h
postgres        ClusterIP      None             <none>        55555/TCP        122m
workflows-api   LoadBalancer   10.108.50.3      localhost     80:30997/TCP     7m54s

The api service is the service a user will access to get into Retool. Using the following URL, Retool login should appear:

http://localhost:3000

Retool Login

Installation Walkthrough

The following video performs the full installation to help in executing the lab.

Cleanup of Retool Installation

The following steps can be used to cleanup the Retool Platform K8S artifacts.

Delete the Deployments

kubectl delete deployment api code-executor jobs-runner postgres workflow-worker workflows-api

Delete the Secret

kubectl delete secret retoolsecrets

Delete the Persistent Volume Claim

kubectl delete pvc retool-pvc postgres-pvc

Requirements​

Confirm Setup​

Steps​

Secret Updates​

Image updates to Stable or Edge Release​

Resource Limit updates​

Retool Secrets​

postgres Pod​

jobs-runner Pod​

code-executor Pod​

api Pod​

workflows-api Pod​

workflow-worker Pod​

Login to Retool​

Installation Walkthrough​

Cleanup of Retool Installation​

Delete the Deployments​

Delete the Secret​

Delete the Persistent Volume Claim​

Requirements

Confirm Setup

Steps

Secret Updates

Image updates to Stable or Edge Release

Resource Limit updates

Retool Secrets

postgres Pod

jobs-runner Pod

code-executor Pod

api Pod

workflows-api Pod

workflow-worker Pod

Login to Retool

Installation Walkthrough

Cleanup of Retool Installation

Delete the Deployments

Delete the Secret

Delete the Persistent Volume Claim