Git Syncing

🚧

Use Protected Apps

Git Syncing should only be used if your organization cannot use Github or Github Enterprise. Otherwise, we strongly recommend upgrading to v2.69.17 (or greater) and setting up Protected Apps with Github or Github Enterprise for a simpler and more powerful source control integration.

Retool apps can be synced with a Git repository, letting you customize how you develop and deploy Retool apps across multiple environments and helping your team adhere to application development lifecycle best practices.

The most common approach is to have multiple instances of Retool representing multiple environments: development, staging, and production. Code promotion across environments is controlled via Git, so that every change must be reviewed as a pull request before being promoted to a higher environment.

With this approach, engineers can safely build Retool apps on the development instance as well as run tests & perform QA on the staging instance; end-users can access the applications on the production instance.

Note: you must be on version 2.41.13 or later to use Git syncing.

🚧

Enterprise only

Git syncing is only available for the Enterprise version of Retool. You can get in touch with sales to start an Enterprise trial here.

📘

Resources are not synced

Only application code is synced between Retool and the Git repository. You will need to create resources manually in your non-development environment(s) for apps to work once promoted.

When recreating resources in non-development environment(s), it is important that you use the same resource name to ensure that queries in synced apps work as expected. See our document on secret management using environment variables to learn how to dynamically pass connection details into your resources.

It is also important that you use the same ENCRYPTION_KEY environment variable value across multiple git synced instances.

Git syncing overview

In the below diagram, the development instance is configured to continually push changes to a dev branch. The staging and production instances are configured to automatically watch and deploy code from the staging and master branches, respectively.

In this setup, engineers will only have access to build and edit applications on the development instance of Retool. The apps on the staging and production instances are locked from changes in the Retool user interface (i.e. the instance is read-only).

At your own cadence, you can merge changes from dev to staging to production.

Setting up Retool Git syncing with GitHub

🚧

Other Git-repository managers

We also support using AWS CodeCommit, BitBucket, and Gitlab. If you have any questions about getting these set up, don't hesitate to reach out to us directly!

As you set up Git syncing, note that this is a unidirectional process. Any individual Retool instance can either push changes to your Git repository or pull changes from it.

A typical setup will include:

  • A single development instance pushing changes to the dev branch of your GitHub repository
  • One or more additional instances reading from different branches of the same repository (e.g. a staging instance reading from a staging branch and a production instance reading from the main branch)

See more details & diagrams on our Git workflows page.

Configure a Retool instance to push changes to GitHub

1. Create an empty Git repository

In order to set up Git syncing, we'll need to initialize an empty repository following these steps:

Go to GitHub and create a new repository.

Create a new local directory and clone the repository:

git clone https://github.com/GITHUB_USERNAME/REPO_NAME.git

Navigate into the repository:

cd REPO_NAME

Create an empty commit using the --allow-empty flag:

git commit -m 'Initial commit' --allow-empty

Push this up to GitHub:

git push

🚧

Make sure the repo is empty!

Note: you cannot have any files in the repository. It must be empty! You should now have an empty GitHub repository that looks like this:

2. Get your public key and connect to your branch

As an admin, login to Retool and go to Settings > Advanced. Toward the bottom of the page, there is a section for Git Syncing URL and Branch.

Enter your GitHub URL and branch name and click "Save". Make sure you enter the SSH URL, not the HTTPS URL. The SSH URL should mirror this format:

[email protected]:USERNAME/REPO_NAME.git

Your Retool instance provides a public key to authorize Git syncing. Click the blue link to Download Retool's public key. Open the retool.pub file and copy the entire key to your clipboard.

3. Configure the Deploy Key in GitHub

Navigate back to your GitHub repository and click on the Settings tab, then click Deploy Keys on the left sidebar. Click Add Deploy Key, give it a title, and paste the Retool public key from your clipboard.

Make sure you check the box for Allow Write Access so that Retool can commit to this repository.

Add Retool's public key to your Git repository. Make sure you grant write access.Add Retool's public key to your Git repository. Make sure you grant write access.

Add Retool's public key to your Git repository. Make sure you grant write access.

4. Test that syncing works

Create a new app or make a change to an existing app in Retool. After about 10 seconds, you should should see the changes syncing to your repository.

Configure a read-only Retool instance

A common setup is to have your production Retool instance be read-only. By configuring this feature, you will disable making any changes to production Retool apps from the browser and force all changes to be read from the Git repository.

Before configuring a read-only instance of Retool, make sure you've already configured a separate instance to push changes to an existing repository.

🚧

This process will overwrite existing changes

When you set up a Retool instance to be read-only from a branch of a GitHub repository, the Git syncing process will replace all existing applications in the Retool Postgres database with those found in the repository. That means you will lose any existing apps that are not already in the repository.

1. Lock the instance from in-app changes

To lock your instance from in-app changes, define the following environment variable in your docker.env file:

VERSION_CONTROL_LOCKED=true

2. Create a new branch in your GitHub repository

From the command line or GitHub UI, create a new branch for this instance to follow. Give it a name that makes sense for this instance (e.g. staging for the staging instance).

3. Get your public key and connect to your branch

As an admin, login to Retool and go to Settings > Advanced. Toward the bottom of the page, there is a section for Git Syncing URL and Branch.

Enter your GitHub URL and branch name and click "Save". Make sure you enter the SSH URL, not the HTTPS URL. The SSH URL should mirror this format:

[email protected]:USERNAME/REPO_NAME.git

Your Retool instance provides a public key to authorize Git syncing. Click the blue link to Download Retool's public key. Open the retool.pub file and copy the entire key to your clipboard.

4. Configure the Deploy Key in GitHub

Navigate back to your GitHub repository and click on the Settings tab, then click Deploy Keys on the left sidebar. Click Add Deploy Key, give it a title, and paste the Retool public key from your clipboard.

Adding the deploy key of the read-only instance to the repository. Leave the option for "Allow write access" unchecked.Adding the deploy key of the read-only instance to the repository. Leave the option for "Allow write access" unchecked.

Adding the deploy key of the read-only instance to the repository. Leave the option for "Allow write access" unchecked.

🚧

Should I enable write access?

Since this is a read-only instance, you do not need to enable write access. In fact, we recommend that you leave this box unchecked, just to prevent any unexpected behavior.

5. Test that syncing works

Wait a few moments and then refresh the Retool page in the browser. You should now see the same apps from your GitHub repository appear in Retool.

Retool application DSL

Retool encodes data for each Retool application internally using an extension of JSON. While effective, it is a format that makes it difficult to visualize change history, so instead of directly storing this data in version control, Retool reformats the data into a YAML file.

A simple page that with a table showing data from a query might be represented as follows:

version: 2.8.1
components:
  - id: table1
    position:
      top: 1
      right: 1
      width: 8
      height: 5
    template:
      data: "{{ query1.data }}"
      selectedIndex: ''
      columns: 
        - id
        - name
        - salary
      columnWidths: 
        - 10
        - 100
        - 50
      columnColors:
        - white
        - white
        - "{{ self > 60 : 'green' : 'red' }}"
      pageSize: 10
      alwaysShowPaginator: true
      onRowSelect: ''
      serverPaginated: false
      totalRowCount: ''
      paginationOffset: 0
      sort: null
      sortedColumn: ''
      sortedDesc: false
      allowMultiRowSelect: false
queries:
  - id: query1
    template:
      query: "select * from users"
      runWhenPageLoads: false
      runWhenModelUpdates: true
      requireConfirmation: false
      confirmationMessage: ""
      queryDisabled: "",
      triggersOnSuccess: []
      triggersOnFailure: []
      privateParams: []
      queryRefreshTime: ''
      queryThrottleTime: '750'
      queryTimeout: '10000'
      showSuccessToaster: true,
      successMessage: ''

Every save in Retool will automatically trigger a new commit message in the repository

How to cycle the deploy key

You can manually regenerate Retool's SSH keys in the event that you need to rotate the deploy key on your GitHub repo. This is useful if your security team has a credential rotation policy or if you accidentally changed your encryption key and now Retool cannot decode the existing SSH key.

In order to do this, you need access to the Retool Postgres database.

First check the ssh_keys table to ensure we only have one record:

select * from ssh_keys

Next, remove this record:

truncate ssh_keys

Then regenerate and download the SSH keys.

  • As an admin, go to the Settings > Advanced page in Retool
  • Click the blue link to Download Retool's public key. Open the retool.pub file and copy the entire key to your clipboard.
  • Clicking this link will generate a new record in the ssh_key table as well as download the public key to your local machine.

Next, navigate back to your GitHub repository and click on the Settings tab, then click Deploy Keys on the left sidebar. Click Add Deploy Key, give it a title, and paste the Retool public key from your clipboard.

🚧

Should I enable write access for the deploy key?

If this is a read-only instance, you should leave the option for Allow Write Access unchecked.

If this is a development instance that writes to GitHub, check this option to enable Retool to commit to this repository.


Did this page help you?