Skip to main content

Execute Python with the Code block

Learn how to use Python code in a workflow.

Use Code blocks to write custom Python code that can transform data and perform complex logic. You can also use popular Python libraries to further extend the functionality of workflows.

To use Python code blocks on self-hosted deployments, you must have the code-executor container configured.

Add a Python Code block​

To use Python in a workflow, add a Code block to the canvas and select Python.

Write and execute Python code​

The Python code editor has much of the same features as the JavaScript editor, such as autocomplete and syntax highlighting. For example, you can transform an array of records in a similar manner to map() with JavaScript:

data = query1.data
return [{
   "fullName": customer['name'],
   "emailAddress": customer['email']
} for customer in data]
Using Python in a workflow

Python limitations​

JavaScript is the primary method for manipulating and transforming data in Retool. To maintain interoperability with other blocks, Python does not support:

  • Function block calls or triggering queries.
  • Usage within Loop and Filter blocks.
  • Data output in types other than serialized JSON.
  • User-imported Python libraries.

Use Python libraries​

Workflows includes built-in support for many popular libraries. This enables you to extend the functionality of workflows beyond data transformation.

Built-in libraries​

To use a built-in Python library:

  1. Open the Libraries tab click +.
  2. Search for the libraries you want to include and then toggle the checkbox to make it available.

You can browse through all built-in libraries below. Some libraries are only available on self-hosted deployments.

Demo

Add custom libraries​

Retool uses NsJail to sandbox the creation of execution environments for custom libraries. NsJail requires privileged container access.

Privileged mode is not supported by ECS/Fargate deployments. To make use of custom libraries, you must either:

  • Migrate to an ECS/EC2 deployment or another supported deployment framework.
  • Host the code-executor service in a separate "jailed" cluster that can run containers in privileged mode on ECS/EC2 or another supported deployment framework. You then configure the CODE_EXECUTOR_INGRESS_DOMAIN environment variable to communicate with the code-executor service over http or https.

Both Cloud and self-hosted organizations can import public packages from PyPI. Self-hosted organizations can also import private packages from PyPI.

Add a public PyPI repository​

As with typical Python development, you provide a list of libraries to use in a requirements.txt file. Retool makes this available through the Workflow editor. To populate it, navigate to the Libraries tab, click +, then select Modify requirements.txt.

requirements.txt
bson==0.5.10
numpy==1.24.1
pandas==1.5.2

Add a private PyPI respository​

Private PyPI repositories are only available on self-hosted deployments that have a configured code-executor service.

Configuring a private PyPI repository requires setting of two environment variables in the code executor deployment: PYPI_REPOSITORIES and TRUSTED_HOSTS.

  • PYPI_REPOSITORIES is a comma-separated list of domains, and each entry is the same format as index-url flag in pip. The default value is https://pypi.org/simple. Some configurations are below:
  • TRUSTED_HOSTS is a comma-separated list of domains, and each entry is a domain name, similar to the trusted-host flag in pip. This allows for authentication through pip and may not be required in most circumstances.
PYPI_REPOSITORIES=https://pypi.org/simple,https://repository.example.com/simple
TRUSTED_HOSTS=pypi.org,repository.example.com

Once configured, you can install packages through the Modify requirements.txt option in the Workflows editor.

requirements.txt
my_internal_library==0.1.0

Configure block settings​

You can configure the following settings for Code blocks.

Timeout after (ms)

The duration to wait before the block times out. Default is 10000.

Retry count

The maximum number of attempts to retry the query if an error is returned.

Interval

The minimum length of time to wait between query retries. This is useful if a query is triggering a rate limit.

Exponential

Whether to increase the interval between retries, also known as exponential backoff. This is useful for handling rate limits.

Finally

Whether to stop the workflow and return an error if the block fails, or continue to allow an error handler to run.

Coefficient

When exponential backoff is enabled, the constant factor by which the time delay between retry attempts increases after each failure.

Max interval

When exponential backoff is enabled, the maximum time to delay between retries.