Skip to main content

Deploy Retool on Kubernetes with Helm

Learn how to deploy Retool on Kubernetes with the Helm package manager.

You can deploy Self-hosted Retool on Kubernetes with Helm 3.3.1 or later.

Requirements​

To deploy Self-hosted Retool with Workflows, you need:

Temporal​

Temporal is a distributed system used to schedule and run asynchronous tasks for Retool Workflows. A Self-hosted Retool instance uses a Temporal cluster to facilitate the execution of each workflow amongst a pool of self-hosted workers that make queries and execute code in your VPC. Temporal manages the queueing, scheduling, and orchestration of workflows to guarantee that each workflow block executes in the correct order of the control flow. It does not store any block results by default.

You can use a Retool-managed cluster on Temporal Cloud, which is recommended for most use cases. You can also use an existing self-managed cluster that is hosted on Temporal Cloud or in your own infrastructure. Alternatively, you can spin up a new self-hosted cluster alongside your Self-hosted Retool instance.

Recommended

You should use a Retool-managed cluster if:

  • You are on a version greater than 3.6.14.
  • Your organization is on the Enterprise plan.
  • You don't have an existing cluster which you prefer to use.
  • Your cluster only needs to be used for a Self-hosted Retool deployment.
  • You don't want to manage the cluster directly.
  • You have a single or multi-instance Retool deployment, where each instance requires its own namespace.

Retool admins can enable Retool-managed Temporal. To get started, navigate to the Retool Workflows page and click Enroll now. Once you update your configuration, return to the page and click Complete setup.

It can take a few minutes to initialize a namespace in Retool-managed Temporal.

Retool-managed Temporal clusters are hosted on Temporal Cloud. Your Self-hosted Retool deployment communicates with the cluster when building, deploying, and executing Retool Workflows. All orchestration data to Temporal is fully encrypted and uses the private encryption key set for your deployment.

Cluster size​

The cluster must have at least one node with 8x vCPUs and 16 GB of memory. Use the following command to retrieve the capacity of your nodes.

$ kubectl describe nodes

In the Capacity section, verify the cpu and memory values meet the above requirements.

Capacity:
  attachable-volumes-aws-ebs:            25
  cpu:                                   8
  ephemeral-storage:                     83873772Ki
  hugepages-1Gi:                         0
  hugepages-2Mi:                         0
  memory:                                7931556Ki
  pods:                                  29

Cluster storage class​

If you want to mount volumes, ensure the volume supplied by your cloud provider can be mounted to multiple nodes. To identify your cluster's storage class, run the command

$ kubectl get storageclass

Reference your cloud provider's documentation to verify that this storage class supports the ReadWriteMany access mode.

1. Add the Retool Helm chart repository​

Use the following command to add the Retool Helm repository.

Add Retool chart
$ helm repo add retool https://charts.retool.com

Run helm search repo retool/retool to confirm you can access the Retool chart.

Verify Retool chart
NAME            	CHART VERSION	APP VERSION	DESCRIPTION
retool/retool   	6.0.3        	           	A Helm chart for Kubernetes

2. Download Helm configuration file​

Retool's Helm chart is configured using a values.yaml file. Download a copy of values.yaml to your local machine, using the command below. Open values.yaml in a text editor or IDE to follow along the steps below.

curl -L -o values.yaml https://raw.githubusercontent.com/tryretool/retool-helm/main/values.yaml

3. Update Helm configuration​

In Kubernetes, you can store configuration options in plain text or use Kubernetes secrets. The following example sets config.licenseKey as plain text.

config:
  licenseKey: "XXX-XXX-XXX"

A Kubernetes secret is an object that contains multiple key-value pairs. You need both the secret name and the key to configure the values.yaml file. The example below uses a value stored in the license-key-secret under the key license-key.

config:
  licenseKeySecretName: license-key-secret
  licenseKeySecretKey:  license-key

Retool recommends storing sensitive data—for example, passwords and credentials—as Kubernetes secrets, especially if you commit values.yaml to source control.

Set the following values in values.yaml.

To generate 36-character random strings for config.encryptionKey and config.jwtSecret, run the command openssl rand -base64 36.

SettingDescription
config.licenseKeyLicense key, in plain text or as a secret value.
config.encryptionKeyKey used to encrypt the database. Generate a random string with openssl.
config.jwtSecretSecret used to sign authentication requests from Retool's server. Generate a random string with openssl.
image.tagVersion of Retool to install, in the format X.Y.Z. Self-hosted Retool with Workflows requires 2.108.4 or later when using a local Temporal Cluster, or 3.6.14 or later for Retool-managed Temporal.
config.useInsecureCookiesWhether to allow insecure cookies. Set to true if you have not configured SSL. Set to false if you use HTTPS to connect to the instance.
workflows.enabledWhether to enable Retool Workflows. Set to true. Defaults to true for Retool version 3.6.14 or later.
codeExecutor.enabledWhether to enable the Code Executor service. Set to true. Defaults to true for Retool version 3.20.15 or later, and automatically sets the image tag for tryretool/code-executor-service corresponding to tryretool/backend.

Each workflow worker can process approximately 10 queries per second (QPS). Increase the workflow replicaCount if this is not high enough for your needs. You should be able to scale this to approximately 40 QPS, using four workflow workers, before you need to make infrastructure changes.

Configure Temporal​

Recommended

Allow your deployment to connect to Temporal​

Open up egress to the public internet on ports 443 and 7233 to allow outbound-only connections to Temporal Cloud from your deployment. This is so services can enqueue work to, and poll work out, of Temporal.

Temporal Cloud does not have a static IP range to allow list. If more specificity is required, egress is required on ports on the following domains:

PortDomains
443*.retool.com, *.tryretool.com, *.temporal.io
7233*.tmprl.cloud

Kubernetes pods are non-isolated for egress by default thereby allowing all outbound connections. If the Retool backend or workers cannot connect to Temporal Cloud, check your egress NetworkPolicy for any issues.

Configure Helm for Temporal cluster​

Update the values.yaml configuration file to specify whether to use a Retool-managed cluster or a self-managed one.

VariableDescription
.Values.workflows.enabledWhether to enable workers and workflow backend pods. Set to true.
.Values.codeExecutor.enabledWhether to enable code executor pods. Set to true.
.Values.retool-temporal-services-helm.enabledWhether to use a local Retool-deployed Temporal cluster. Set to false.
.Values.workflows.temporal.enabledWhether to use a self-managed Temporal cluster. Set to false.

4. Install Self-hosted Retool​

After updating the configuration, install Self-hosted Retool.

helm install my-retool retool/retool -f values.yaml

After installing Retool, run kubectl get pods to verify you have pods for the main service and jobs-runner. If you use the PostgreSQL subchart, there is also a postgresql pod. If you have enabled Workflows, there are also workflow-worker and workflow-backend pods.

Verify pods
my-retool-7898474bbd-pr8n6               1/1     Running   1 (8h ago)   8h
my-retool-jobs-runner-74796ddd99-dd856   1/1     Running   0            8h
my-retool-postgresql-0                   1/1     Running   0            8h

Once the main service is running, verify the installation by port forwarding to localhost.

kubectl port-forward my-retool-7898474bbd-69zjt 3000:3000

You can then access Retool at http://localhost:3000/.

Additional configuration​

The following configuration steps are optional but strongly recommended for using Retool in a production environment.

Whenever you run helm upgrade, use the --version flag to specify the chart's version number. Otherwise, Helm upgrades to the latest chart version, which may cause compatibility issues. You can check the release version of your deployment with the command helm list.

Externalize database​

By default, the Retool Helm chart uses the PostgreSQL subchart to create a containerized instance of PostgreSQL. This is not suitable for production use cases, and the Retool storage database should be hosted on an external, managed database. Managed databases are more maintainable, scalable, and reliable than containerized PostgreSQL instances. These instructions explain how to set up Retool with an external database.

1. Export data​

If you have already populated the PostgreSQL pod, export its data.

kubectl exec -it <POSTGRES-POD-NAME> -- bash -c 'pg_dump hammerhead_production --no-acl --no-owner --clean -U postgres' > retool_db_dump.sql

2. Disable PostgreSQL chart​

In values.yaml, set postgresql.enabled to false to disable the included PostgreSQL chart. This prevents the containerized PostgreSQL from starting.

3. Update PostgreSQL configuration​

In values.yaml, set the config.postgresql properties with settings for your external database. This specifies the externalized database to back up your Retool instance.

4. Upgrade Helm chart version​

Use helm to perform the upgrade and include the Helm chart version number. Retool requires version 6.1.1 or later.

helm upgrade -f values.yaml my-retool retool/retool --version 6.1.1

Add environment variables​

Environment variables provide ways to configure a Retool instance. The values.yaml file has three locations to add environment variables.

ObjectType
envPlain text key-value pairs.
environmentSecretsPlain text or Kubernetes secrets.
environmentVariablesPlain text or Kubernetes secrets.

Do not store sensitive information, such as access tokens, in env. Use environmentSecrets or environmentVariables as they can populate environment variables from Kubernetes secrets.

Mount volumes​

There are several use cases which require the use of volumes. For example, when configuring a gRPC resource, you need to mount a volume containing the protos files to the Retool deployment. Follow these instructions to create a persistent volume and copy files from your local machine to the volume.

1. Enable PersistentVolumeClaim​

The Helm chart defines a PersistentVolumeClaim (PVC) which is automatically mounted to the Retool pods, enabling Retool to access files within this volume. The PVC is disabled by default. To enable the persistentVolumeClaim, modify your values.yaml file:

persistentVolumeClaim:
  enabled: true
  existingClaim: ""

If you have an existing PVC in your Kubernetes cluster to use, you can specify its name in existingClaim. Otherwise, leave existingClaim blank.

2. Set security context​

In a later step, you use kubectl cp to copy files from your local machine to the Kubernetes cluster, which requires the pod to run with root privileges. Modify your deployment so the pods run as root by changing the securityContext in your values.yaml file:

securityContext:
  enabled: true
  runAsUser: 0

Use helm to perform the upgrade and include the Helm chart version number. Retool requires version 6.1.1 or later.

helm upgrade -f values.yaml my-retool retool/retool --version 6.1.1

3. Verify pods​

Run kubetcl get pods to verify pods are running.

my-retool-7898474bbd-pr8n6               1/1     Running   1 (8h ago)   8h
my-retool-jobs-runner-74796ddd99-dd856   1/1     Running   0            8h
my-retool-postgresql-0                   1/1     Running   0            8h

4. Copy files​

Next, copy the protos files from your local machine to the PVC. Note from kubectl get pods the three pods in the deployment: the main, jobs-runner, and postgresql containers. Identify the name of the main container.

Ensure you local machine has a folder named protos and run the following command, and replacing my-retool-7c4c89798-fqbh7 with the name of your Retool container.

kubectl cp protos/ my-retool-7c4c89798-fqbh7:/retool_backend/pv-data/protos

4. Set env​

If you're configuring gRPC, you need to specify the location of the protos directory. In values.yaml, set the PROTO_DIRECTORY_PATH environment variable.

env:
  PROTO_DIRECTORY_PATH: "/retool_backend/pv-data/protos"

5. Reset security context​

Revert the security context of your deployment back to a disabled state.

securityContext:
  enabled: false
  runAsUser: 1000

Use helm to perform the upgrade and include the Helm chart version number. Retool requires version 6.1.1 or later.

helm upgrade -f values.yaml my-retool retool/retool --version 6.1.1

Configure SSL​

When configuring SSL, you can use Let's Encrypt to provision a certificate, or provide your own. See Configure SSL and custom certificates for more detail on certificates.

1. Install cert-manager​

First, add the jetstack Helm repository if you haven't already.

helm repo add jetstack https://charts.jetstack.io

Next, run the following command to install cert-manager.

helm install \
cert-manager jetstack/cert-manager \
--namespace cert-manager \
--create-namespace \
--version v1.11.0 \
--set installCRDs=true --set ingressShim.defaultIssuerName=letsencrypt-prod \
--set ingressShim.defaultIssuerKind=ClusterIssuer \
--set ingressShim.defaultIssuerGroup=cert-manager.io

2. Configure certificate issuer​

Create a file called production-issuer.yml. Copy the following configuration, replace the example email with your email, and paste it into the new file.

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: example@example.com
    privateKeySecretRef:
      name: letsencrypt-prod
    solvers:
      - http01:
          ingress:
            class: nginx

3. Create the certificate manager​

First, use kubectl to create the certificate manager as ClusterIssuer.

kubectl apply -f production-issuer.yml

4. Verify pod​

Run kubectl get clusterissuer to verify that the ClusterIssuer pod is ready.

NAME               READY   AGE
letsencrypt-prod   True    10m

5. Update ingress configuration​

Add the annotations section to your ingress and modify the host and hosts placeholders accordingly.

ingress:

  ...

  annotations:
    kubernetes.io/tls-acme: "true"
    certmanager.io/cluster-issuer: letsencrypt-prod
  hosts:
  - host: example.example.com
    paths:
      - path: /
  tls:
  - secretName: letsencrypt-prod
    hosts:
      - example.example.com

6. Apply changes​

After the pods restart, you can access the page in your browser using TLS.

Update your Kubernetes instance​

Follow these instructions to update your Retool instance to a newer release version.

1. Back up your database​

If you use a managed database service, your database provider may have a feature to take snapshots or otherwise back up your database. If you use the PostgreSQL subchart, run the following command to export data from the PostgreSQL pod to a .sql file.

kubectl exec -it <POSTGRES-POD-NAME> -- bash -c 'pg_dump hammerhead_production --no-acl --no-owner --clean -U postgres' > retool_db_dump.sql

2. Select a new version​

Update the image.tag value in values.yaml to the Docker tag for the version of Retool to install, such as tryretool/backend:3.114.3-stable.

3. Upgrade instance and apply changes​

Always upgrade your deployment to use the latest version of Retool's Helm chart. The latest version is .

Run helm search repo retool/retool to check the current version of Retool's Helm chart that is installed. Use helm upgradeto then upgrade the Helm chart version, if required.

Upgrade Retool Helm chart version
helm upgrade -f values.yaml my-retool retool/retool --version

4. Verify instance​

Run kubectl get pods to verify that the update has completed.

my-retool-7898474bbd-pr8n6               1/1     Running   1 (8h ago)   8h
my-retool-jobs-runner-74796ddd99-dd856   1/1     Running   0            8h
my-retool-postgresql-0                   1/1     Running   0            8h

You should see additional services for workflows, such as workflows-worker, workflows-backend, and code-executor.