Deploy Retool with Kubernetes

You can deploy self-hosted Retool on Kubernetes following the instructions in this guide.

Requirements

To deploy Retool on Kubernetes, you need:

A Retool license key, which you can obtain from my.retool.com.
A domain you own, to which you can add a DNS record.
A Kubernetes cluster. To create a cluster, see documentation on Google Cloud Platform, AWS, and Azure.
A working installation of kubectl. To install kubectl, see documentation on Google Cloud Platform, AWS, and Azure.

Cluster size

The cluster must have at least one node with 2x vCPUs and 8 GB of memory. Use the following command to retrieve the capacity of your nodes.

$ kubectl describe nodes

In the Capacity section, verify the cpu and memory values meet the above requirements.

Capacity:
  attachable-volumes-aws-ebs:  25
  cpu:                                   2
  ephemeral-storage:                     83873772Ki
  hugepages-1Gi:                         0
  hugepages-2Mi:                         0
  memory:                                7931556Ki
  pods:                                  29

Cluster storage class

If you want to mount volumes, ensure the volume supplied by your cloud provider can be mounted to multiple nodes. To identify your cluster's storage class, run the command

$ kubectl get storageclass

Reference your cloud provider's documentation to verify that this storage class supports the ReadWriteMany access mode.

1. Clone manifests

Retool's Kubernetes deployment is configured using a set of manifests. To retrieve a copy of the manifests, clone the retool-onpremise repository to your local machine. Open the kubernetes directory in an IDE to follow along the steps below.

git clone https://github.com/tryretool/retool-onpremise.git

2. Configure version

In retool-container.yaml and retool-jobs-runner.yaml, change the image tag to indicate the version of Retool to install. The following example specifies the image tag to install version 2.115.2.

image: tryretool/backend:2.115.2

3. Configure secrets

Copy the retool-secrets.template.yaml file to a new file named retool-secrets.yaml. This file sets the configuration options for your deployment, and stores them as Kubernetes secrets.

cp retool-secrets.template.yaml retool-secrets.yaml

Set the configuration options for your instance. Note that values in this file need to be encoded in base64.

📘
Use openssl for base64 encoded string generation
To generate 36-character random strings, encoded in base64, run the command $ openssl rand -base64 36.

Set the following values in retool-secrets.yaml:

Setting	Description
`data.jwt_secret`	Secret used to sign authentication requests from Retool's server. Generate a base64 encoded string with `openssl`.
`data.encryption_key`	Key used to encrypt the database. Generate a base64 encoded string with `openssl`.
`data.postgres_password`	Password for Retool's internal database. Generate a base64 encoded string with `openssl`.
`data.license_key`	License key. Encode your license key using the command `echo -n <licensekey> \\| base64`.

4. Install Retool

After updating the configuration, install Retool. Run the following commands in sequence.

kubectl apply -f ./retool-secrets.yaml
kubectl apply -f ./retool-postgres.yaml
kubectl apply -f ./retool-container.yaml
kubectl apply -f ./retool-jobs-runner.yaml

After installing Retool, verify you have pods for the api service, jobs-runner and postgres.

$ kubectl get pods
NAME                                     READY   STATUS    RESTARTS      AGE
api-76464f5576-vc5f4                     1/1     Running   1 (8h ago)   8h
jobs-runner-5cfb79cbfd-b49rd             1/1     Running   0            8h
postgres-69c485649c-lkjgc                1/1     Running   0            8h

Once all three pods are running, verify the installation by port forwarding to localhost.

kubectl port-forward api-bb4848bc6-sb5rn 3000:3000

You can then access Retool at http://localhost:3000/.

Additional steps

The following steps are optional. On production instances, Retool strongly recommends you externalize your database, configure SSL, and keep up-to-date with the latest version of Retool. Setting environment variables is often necessary to configure SSO, source control, and other self-hosted features.

Externalize database

By default, the Retool Kubernetes installation uses a PostgreSQL pod to create a containerized instance of PostgreSQL. This is not suitable for production use cases, and the Retool storage database should be hosted on an external, managed database. Managed databases are more maintainable, scalable, and reliable than containerized PostgreSQL instances. These instructions explain how to set up Retool with an external database.

If you have already populated the PostgreSQL pod, export its data.

kubectl exec -it <POSTGRES-POD-NAME> -- bash -c 'pg_dump hammerhead_production --no-acl --no-owner --clean -U postgres' > retool_db_dump.sql

Encrypt your database password.

echo -n <password> | base64

In retool-secrets.yaml, set the value of postgres_password as the base64 encoded password.
In retool-container.yaml, set the PostgreSQL credentials to the credentials of your managed PostgreSQL instance. You do not need to specify POSTGRES_PASSWORD, since that value is pulled from the secret you already configured.

- name: POSTGRES_HOST
  value: <postgres-host>
- name: POSTGRES_PORT
  value: "5432"
- name: POSTGRES_USER
  value: <postgres-user>

In retool-jobs-runner.yaml, set the PostgreSQL credentials to the credentials of your managed PostgreSQL instance. The values are the same as in the previous step.
Apply changes to the three manifests.

kubectl apply -f retool-secrets.yaml
kubectl apply -f retool-container.yaml
kubectl apply -f retool-jobs-runner.yaml

Update Retool

Back up your database. If you use a managed database service, your database provider may have a feature to take snapshots or otherwise back up your database. If you use the PostgreSQL subchart, run the following command to export data from the PostgreSQL pod to a .sql file.

kubectl exec -it <POSTGRES-POD-NAME> -- bash -c 'pg_dump hammerhead_production --no-acl --no-owner --clean -U postgres' > retool_db_dump.sql

Identify the appropriate release version on Docker Hub. See Retool's self-hosted release notes to learn about version-specific features.
In retool-container.yaml and retool-jobs-runner.yaml, change the image tag to indicate the version of Retool to install. The following example specifies the image tag to install version 2.115.2.

image: tryretool/backend:2.115.2

Apply changes to the two manifests.

kubectl apply -f retool-container.yaml
kubectl apply -f retool-jobs-runner.yaml

Verify that your pods are running.

$ kubectl get pods
NAME                                     READY   STATUS    RESTARTS      AGE
api-76464f5576-vc5f4                     1/1     Running   1 (8h ago)   8h
jobs-runner-5cfb79cbfd-b49rd             1/1     Running   0            8h
postgres-69c485649c-lkjgc                1/1     Running   0            8h

Add environment variables

Environment variables provide ways to configure a Retool instance.

Add the environment variable to both retool-container.yaml and retool-jobs-runner.yaml. This example sets the USE_SHORT_SESSIONS variable, but this pattern applies to other environment variables as well.

env:
- name: USE_SHORT_SESSIONS
  value: true

Apply the changes to the two manifests.

kubectl apply -f retool-container.yaml
kubectl apply -f retool-jobs-runner.yaml

Verify that your pods are running.

$ kubectl get pods
NAME                                     READY   STATUS    RESTARTS      AGE
api-76464f5576-vc5f4                     1/1     Running   1 (8h ago)   8h
jobs-runner-5cfb79cbfd-b49rd             1/1     Running   0            8h
postgres-69c485649c-lkjgc                1/1     Running   0            8h

Mount volumes

There are several use cases which require the use of volumes. For example, when configuring a gRPC resource, you need to mount a volume containing the protos files to the Retool deployment. Follow these instructions to create a persistent volume and copy files from your local machine to the volume.

1. Set security context

In a later step, you use kubectl cp to copy files from your local machine to the Kubernetes cluster, which requires the pod to run with root privileges. Modify your deployment so the pods run as root by adding the securityContext in your retool-container.yaml file:

spec:
      securityContext:
        runAsUser: 0
        fsGroup: 2000

Apply changes to the manifest.

kubectl apply -f retool-container.yaml

Verify that your pods are in a ready state before continuing.

$ kubectl get pods
NAME                                     READY   STATUS    RESTARTS      AGE
api-76464f5576-vc5f4                     1/1     Running   1 (8h ago)   8h
jobs-runner-5cfb79cbfd-b49rd             1/1     Running   0            8h
postgres-69c485649c-lkjgc                1/1     Running   0            8h

2. Copy files

Next, copy the protos files from your local machine to the PVC. Ensure you local machine has a folder named protos and run the following command. Replace api-76464f5576-vc5f4 with the name of your main Retool container, retrieved from kubectl get pods.

kubectl cp protos/ api-76464f5576-vc5f4:/retool_backend/pv-data/protos

3. Set directory path

If you're configuring gRPC, specify the location of the protos directory. In retool-container.yaml, set the PROTO_DIRECTORY_PATH environment variable.

env:
- name: PROTO_DIRECTORY_PATH
  value: "/retool_backend/pv-data/protos"

4. Reset security context

Reset the security context of your deployment by removing the securityContext field, or by defining a non-root user.

Apply changes to the manifest.

kubectl apply -f retool-container.yaml

Configure SSL

Run the following command to install cert-manager.

kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.11.0/cert-manager.yaml

Create a file called production-issuer.yml. Copy the following configuration, replace the example email with your email, and paste it into the new file.

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: [email protected]
    privateKeySecretRef:
      name: letsencrypt-prod
    solvers:
      - http01:
          ingress:
            class: nginx

Create the cert manager as ClusterIssuer.

kubectl apply -f production-issuer.yml

Verify that the pod is ready.

$ kubectl get clusterissuer

NAME               READY   AGE
letsencrypt-prod   True    10m

Add the annotations section to your ingress and modify the host and hosts placeholders accordingly.

ingress:

  ...

  annotations:
    kubernetes.io/tls-acme: "true"
    certmanager.io/cluster-issuer: letsencrypt-prod
  hosts:
  - host: example.example.com
    paths:
      - path: /
  tls:
  - secretName: letsencrypt-prod
    hosts:
      - example.example.com

Apply the changes to your manifests. After the pods restart, you can access the page in your browser using TLS.

Requirements

Cluster size

Cluster storage class

1. Clone manifests

2. Configure version

3. Configure secrets

📘Use openssl for base64 encoded string generation

4. Install Retool

Additional steps

Externalize database

Update Retool

Add environment variables

Mount volumes

1. Set security context

2. Copy files

3. Set directory path

4. Reset security context

Configure SSL

📘
Use `openssl` for base64 encoded string generation