System architecture of self-hosted Retool deployments

Learn about Retool's self-hosted architecture.

Self-hosted Retool Availability
Self-hosted Edge	Generally Available	Enterprise
Self-hosted Stable	Generally Available	Enterprise

Self-hosted Retool is available on an Enterprise plan only.

Organizations that have on-premise requirements can deploy the Retool platform on their own infrastructure. Each deployment instance uses a distributed set of containers with services for different functions. These work together to securely run the platform within your VPN or VPC.

Containers and services

A standard deployment instance uses the following containers and services that interact with each other. The SERVICE_TYPE environment variable values determine what services a container runs.

Container	Image	Repository	Services
api	backend	tryretool/backend	MAIN_BACKEND DB_CONNECTOR DB_SSH_CONNECTOR
jobs-runner	backend	tryretool/backend	JOBS_RUNNER
workflows-worker	backend	tryretool/backend	WORKFLOW_TEMPORAL_WORKER (for workflows)
workflows-backend	backend	tryretool/backend	WORKFLOW_BACKEND DB_CONNECTOR DB_SSH_CONNECTOR
agent-worker	backend	tryretool/backend	WORKFLOW_TEMPORAL_WORKER (for agents)
agent-eval-worker	backend	tryretool/backend	AGENT_EVAL_TEMPORAL_WORKER
code-executor	code-executor-service	tryretool/code-executor-service	No service type.

Loading diagram...

api

The api container manages the core functionality for a Retool deployment instance, such as:

• Frontend interactions (e.g., building apps and workflows).
• Hosting apps and workflows.
• Managing users and permissions.

api runs the following services:

MAIN_BACKEND

The core service for a Retool deployment instance. It handles most logic for frontend interactions.

DB_CONNECTOR

Handles query requests to resources (databases, APIs, etc.).

DB_SSH_CONNECTOR

Manages connections for DB_CONNECTOR.

Network ingress and egress

api requires ingress from client interactions, such as loading an app in a browser. These are typically handled by a load balancer (e.g., nginx) which proxies the requests.

The following table displays ingress and egress requirements for api.

Container	Network ingress	Network egress
code-executor
postgres (platform database)
retooldb-postgres
Temporal
Resources

Replicas

You can replicate api as you scale to manage higher volumes of workflow traffic.

Resource query trace

The following diagram illustrates a typical resource query trace as it processes through a self-hosted deployment.

Loading diagram...

jobs-runner

The jobs-runner container manages background tasks, runs database migrations for Retool version upgrades, and Source Control. It runs the following service:

JOBS_RUNNER

Performs database migrations and manages Source Control.

Network ingress and egress

The following table displays ingress and egress requirements for jobs-runner.

Container	Network ingress	Network egress
postgres

Replicas

You must not replicate jobs-runner. It performs tasks and migrations that must be performed as a single container.

workflows-worker

The workflows-worker container continuously polls the Temporal Cluster for tasks required to either start or execute blocks within a workflow. It makes requests to code-executor to execute blocks and process results, then reports back to Temporal to continue or complete workflow execution.

workflows-worker runs the following service.

WORKFLOW_TEMPORAL_WORKER

Polls the Temporal Cluster for tasks required to start or execute workflow blocks.

Network ingress and egress

The following table displays ingress and egress requirements for workflows-worker.

Container	Network ingress	Network egress
code-executor
postgres (platform database)
retooldb-postgres
Temporal
Resources

Replicas

You can replicate workflows-worker as you scale to manage higher volumes of workflow traffic.

workflows-backend

The workflows-backend container receives and processes requests from code-executor to process workflows. It also handles the upload of workflows logs, block results, and run status updates.

WORKFLOW_BACKEND

Handles query, retry, and asynchronous logic.

DB_CONNECTOR

Handles query requests to resources (databases, APIs, etc.).

DB_SSH_CONNECTOR

Manages connections for DB_CONNECTOR.

Network ingress and egress

The following table displays ingress and egress requirements for workflows-backend.

Container	Network ingress	Network egress
code-executor
postgres (platform database)
retooldb-postgres
Temporal
Resources

Replicas

You can replicate workflows-backend as you scale to manage higher volumes of workflow traffic.

agent-worker

The agent-worker container continuously polls the Temporal cluster for pending agent runs to execute. It makes requests to code-executor to execute the agent logic, call tools and process results, and then it reports back to Temporal to continue or complete agent execution.

Network ingress and egress

The following table displays ingress and egress requirements for agent-worker.

Container	Network ingress	Network egress
code-executor
postgres (platform database)
retooldb-postgres
Temporal
Resources

Replicas

You can replicate agent-worker as you scale to manage higher volumes of workflow traffic.

agent-eval-worker

The agent-eval-worker continuously polls the Temporal Cluster for pending agent eval runs. When an eval is running, the agent-eval-worker makes calls from Postgres to the LLM provider configured in the test case, and compares the output to the expected value.

Network ingress and egress

The following table displays ingress and egress requirements for agent-eval-worker.

Container	Network ingress	Network egress
code-executor
postgres (platform database)
Temporal
Resources

Replicas

You can replicate agent-eval-worker as you scale to manage higher volumes of workflow traffic.

code-executor

The code-executor container executes single block runs from the Workflows IDE, multiple blocks as part of a Workflow execution, and any arbitrary user code written in a Workflow. It executes JavaScript and Python code directly, or makes requests to workflows-backend to run resource queries against databases, APIs, etc.

By default, the code-executor uses nsjail to sandbox code execution. nsjail requires privileged container access. If your deployment does not support privileged access, you can run the code executor in unprivileged mode. However, Retool strongly recommends running NsJail with privileged mode, as it provides sandboxing and remains the most secure option.

Network ingress and egress

The following table displays ingress and egress requirements for code-executor.

Container	Network ingress	Network egress
api
workflows-backend
workflows-worker

Replicas

You can replicate code-executor as you scale to manage higher volumes of workflow traffic.

postgres

postgres manages the platform database, which stores your instance's apps, resources, users, permissions, and internal data. All deployment instances contain a default postgres container for testing purposes. You must externalize this database before using the deployment instance in production.

When configuring an external database, you must:

• Enable the uuid-ossp module.
• Use the Read Committed isolation level.

Retool creates a number of tables in the database, such as:

Table	Description
app_observability_provider_configs	Observability setup and streaming details.
audit_trail_events	Events that feed audit logs on user instances with the /audit endpoint.
grid_managed_cluster_resources	Retool Database when linked to an external PostgreSQL database.
grid_managed_clusters	Retool Database when linked to an external PostgreSQL database.
organizations	Spaces within the organization.
page_saves	Changes made to specific apps since they were created.
resources	Resources, including full setup and configuration information.
users	All enabled and disabled users.
user_task_instance	Workflows that include user task blocks.

Do not directly modify Retool table data. Doing so can damage the instance.

All storage database tables

access_control_list_members
access_control_lists
access_levels
api_keys
app_metadata
app_themes
approval_task_executions
approval_task_items
approval_task_votes
appstore_tags
async_jobs
bad_passwords
block_saves
blocks
blueprints_appstore_tags
blueprints
branches
commits
component_metadata
config_var_values
config_vars
custom_component_collection_revision_files
custom_component_collection_revisions
custom_component_collections
custom_domains
dg_activity
dg_bulk_edit
dg_grid
dg_single_edit
email_verification_tokens
embeds
environment_config_vars
environments
event_workflows
experiment_audiences
experiment_strategies
experiments
external_embed_sessions
external_users
features
flow_input_schemas
flow_queries
flow_stages
flow_task_histories
flow_task_inputs
flow_tasks
flows
folder_favorites
folders
form_fields
forms
grid_field
grid_group_access
grid_table_group_access
grid_table_user_access
grid_user_access
grid_view
group_folder_defaults
group_pages
group_resource_folder_defaults
group_resources
group_workflows
groups
iam_credentials
instrumentation_integrations
language_configuration_save
language_configuration
mobile_settings
notification_applications
notification_subscribed_devices
notification_topic_subscriptions
org_image_blobs
organization_email_domains
organization_user_attributes
page_docs
page_favorites
page_onboarding_state
page_save_playground_query_saves
page_user_heartbeats
pages
partially_registered_users
personal_access_tokens
plan_features
plans
playground_queries
playground_query_saves
query_metadata
recently_visited_apps
resource_folders
resource_preview_hints
retool_databases
retool_db_migrations
retool_db_provision
retool_files
retool_managed_note_comment
retool_managed_note
retool_rules
retool_table_events
retool_tables
role_pages_members
role_pages
secrets_manager_configs
SequelizeMeta
sessions
source_control_deployment_settings
source_control_deployments
source_control_protection_status
source_control_provider_configs
source_control_relationships
source_control_repo_migration_logs
source_control_repo_migrations
source_control_settings
source_control_user_info
source_control_uuid_mappings
ssh_keys
startup_programs
storage_blobs
tags
temporal_cloud_settings
temporal_cloud_tls_configs
themes
tracked_property_usages
translations
user_groups
user_invite_groups
user_invite_suggestions
user_invites
user_login_ip_addresses
user_session_states
user_viewed_features
vectors
vscode_sessions
vscode_types
workflow_aggregate_usage
workflow_block_result_location_enum
workflow_block_results
workflow_block_runs
workflow_compression_scheme_enum
workflow_custom_url_path
workflow_release
workflow_run_logs
workflow_run
workflow_save
workflow_tracked_property_usages
workflow_trigger
workflow
workspaces

retooldb-postgres

You can optionally configure your deployment to use Retool Database with a separate PostgreSQL database. You can use retooldb-postgres to get started but Retool recommends you migrate to an externally hosted database for use in production.

Other dependencies

Self-hosted deployment instances have additional dependencies.

Resources

Each data source you use with Retool is represented by a resource, such as APIs and external databases. Your deployment instance must have access to any of its resources to interact with data.

Temporal

Temporal is a distributed system used to schedule and run asynchronous tasks for Retool Workflows and Retool Agents. A Self-hosted Retool instance uses a Temporal cluster to facilitate the execution of each workflow amongst a pool of self-hosted workers that make queries and execute code in your VPC. Temporal manages the queueing, scheduling, and orchestration of workflows to guarantee that each workflow block executes in the correct order of the control flow. It does not store any block results by default.

You can use a Retool-managed cluster on Temporal Cloud, which is recommended for most use cases. You can also use an existing self-managed cluster that is hosted on Temporal Cloud or in your own infrastructure. Alternatively, you can spin up a new self-hosted cluster alongside your Self-hosted Retool instance.

Docker images

Retool uses Docker images that configure the containers and their respective services. Each container runs one or more services to distribute functionality. This approach allows for efficient scaling of a Retool deployment instance as needs grow.

Retool maintains two image repositories on Docker Hub:

• The backend image is used for most containers and services.
• The code-executor-service image is used to execute blocks of code in Retool Workflows.

Each Docker Hub repository tag corresponds to a particular version of Retool. You specify the version to use when deploying or upgrading a release. For instance:

• tryretool/backend: X.Y.Z-stable

• tryretool/code-executor-service: X.Y.Z-stable

For information about hardened variants of these images, including arm64 support and security changes, see Hardened images (beta).