Skip to main content

Self-hosted Retool architecture

Learn about Retool's self-hosted architecture.

Organizations that have on-premise requirements can deploy the Retool platform on their own infrastructure. Each deployment instance uses a distributed set of containers with services for different functions. These work together to securely run the platform within your VPN or VPC.

You can deploy self-hosted Retool on almost any Linux-based VM cloud provider using Docker Compose, such as Amazon EC2, Azure Virtual Machines, and Google Compute Engine. Self-hosted Retool is also available for Kubernetes clusters and AWS ECS managed containers.

Containers and services

A standard deployment instance uses the following containers and services that interact with each other. The SERVICE_TYPE environment variable values determine what services a container runs.

ContainerImageRepositoryServices
apibackendtryretool/backendMAIN_BACKEND
DB_CONNECTOR_SERVICE
DB_SSH_CONNECTOR_SERVICE
jobs-runnerbackendtryretool/backendJOBS_RUNNER
workflows-workerbackendtryretool/backendWORKFLOW_TEMPORAL_WORKER
workflows-backendbackendtryretool/backendWORKFLOW_BACKEND
DB_CONNECTOR_SERVICE
DB_SSH_CONNECTOR_SERVICE
code-executorcode-executor-servicetryretool/code-executor-serviceNo service type.

api

The api container manages the core functionality for a Retool deployment instance, such as:

  • Frontend interactions (e.g., building apps and workflows).
  • Hosting apps and workflows.
  • Managing users and permissions.

api runs the following services:

MAIN_BACKEND

The core service for a Retool deployment instance. It handles most logic for frontend interactions.

DB_CONNECTOR_SERVICE

Handles query requests to resources (databases, APIs, etc.).

DB_SSH_CONNECTOR_SERVICE

Manages connections for DB_CONNECTOR_SERVICE.

Network ingress and egress

api requires ingress from client interactions, such as loading an app in a browser. These are typically handled by a load balancer (e.g., nginix) which proxies the requests.

The following table displays ingress and egress requirements for api.

ContainerNetwork ingressNetwork egress
code-executor
postgres
retooldb-postgres
Temporal
Resources

Replicas

You can replicate api as you scale to manage higher volumes of workflow traffic.

jobs-runner

Thee jobs-runner container manages background tasks, runs database migrations for Retool version upgrades, and Source Control. It runs the following service:

JOBS_RUNNER

Performs database migrations and manages Source Control.

Network ingress and egress

The following table displays ingress and egress requirements for jobs-runner.

ContainerNetwork ingressNetwork egress
postgres

Replicas

You must not replicate jobs-runner. It performs tasks and migrations that must be performed as a single container.

workflows-worker

The workflows-worker container continuously polls the Temporal cluster for tasks required to either start or execute blocks within a workflow. It makes requests to code-executor to execute blocks and process results, then reports back to Temporal to continue or complete workflow execution.

workflows-worker runs the following service.

WORKFLOW_TEMPORAL_WORKER

Polls the Temporal cluster for tasks required to start or execute workflow blocks.

Network ingress and egress

The following table displays ingress and egress requirements for workflows-worker.

ContainerNetwork ingressNetwork egress
code-executor
postgres
retooldb-postgres
Temporal
Resources

Replicas

You can replicate workflows-worker as you scale to manage higher volumes of workflow traffic.

workflows-backend

The workflows-backend container receives and processes requests from code-executor to process workflows. It also handles the upload of workflows logs, block results, and run status updates.

WORKFLOW_BACKEND

Handles query, retry, and asynchronous logic.

DB_CONNECTOR_SERVICE

Handles query requests to resources (databases, APIs, etc.).

DB_SSH_CONNECTOR_SERVICE

Manages connections for DB_CONNECTOR_SERVICE.

Network ingress and egress

The following table displays ingress and egress requirements for workflows-backend.

ContainerNetwork ingressNetwork egress
code-executor
postgres
retooldb-postgres
Temporal
Resources

Replicas

You can replicate workflows-backend as you scale to manage higher volumes of workflow traffic.

code-executor

The code-executor container executes single block runs from the Workflows IDE, multiple blocks as part of a Workflow execution, and any arbitrary user code written in a Workflow. It executes JavaScript and Python code directly, or makes requests to workflows-backend to run resource queries against databases, APIs, etc.

By default, the code-executor uses nsjail to sandbox code execution. nsjail requires privileged container access. If your deployment does not support privileged access, the code executor automatically detects this and runs code without sandboxing. Without sandboxing, use of custom JS libraries and custom Python libraries is not allowed.

Network ingress and egress

The following table displays ingress and egress requirements for code-executor.

ContainerNetwork ingressNetwork egress
api
workflows-backend
workflows-worker

Replicas

You can replicate code-executor as you scale to manage higher volumes of workflow traffic.

postgres

postgres manages the internal PostgreSQL database in which an instance stores its apps, resources, users, permissions, and internal data. All deployment instances contain a default postgres container for testing purposes. You must externalize this database before using the deployment instance in production.

retooldb-postgres

You can optionally configure your deployment to use Retool Database with a separate PostgreSQL database. You can use retooldb-postgres to get started but Retool recommends you migrate to an externally hosted database for use in production.

Other dependencies

Self-hosted deployment instances have additional dependencies.

Resources

Each data source you use with Retool is represented by a resource, such as APIs and external databases. Your deployment instance must have access to any of its resources to interact with data.

Temporal

Temporal is a distributed system used to schedule and run asynchronous tasks for Retool Workflows. A Self-hosted Retool instance uses a Temporal cluster to facilitate the execution of each workflow amongst a pool of self-hosted workers that make queries and execute code in your VPC. Temporal manages the queueing, scheduling, and orchestration of workflows to guarantee that each workflow block executes in the correct order of the control flow. It does not store any block results by default.

You can use a Retool-managed cluster on Temporal Cloud, which is recommended for most use cases. You can also use an existing self-managed cluster that is hosted on Temporal Cloud or in your own infrastructure. Alternatively, you can spin up a new self-hosted cluster alongside your Self-hosted Retool instance.

Docker images

Retool uses Docker images that configure the containers and their respective services. Each container runs one or more services to distribute functionality. This approach allows for efficient scaling of a Retool deployment instance as needs grow.

Retool maintains two image repositories on Docker Hub:

  • The backend image is used for most containers and services.
  • The code-executor-service image is used to execute blocks of code in Retool Workflows.

Each Docker Hub repository tag corresponds to a particular version of Retool. You specify the version to use when deploying or upgrading a release. For instance, the Docker Hub tags for Retool 3.33.10 Stable are: