Configuring S3

Connecting to S3 requires the following to be completed in AWS.

Create an S3 Bucket

Navigate to S3 > Buckets > Create bucket. Here you can give your bucket a name and pick a region for it to be hosted in. If this is your first time setting up the resource, skip the additional settings for now, you can update them after this initial setup.

After creating your bucket, keep a note of its ARN to use later.

Set up CORS on your bucket

Navigate to the Permissions tab for your bucket. There you should find a section near the bottom for Cross-origin resource sharing (CORS). Edit it to allow the domain where your Retool instance is hosted.

The following should grant access to most Cloud-hosted orgs but you can replace the wildcard * with your own subdomain if needed. Self-hosted users will need to enter their own domain.

[
  {
    "AllowedOrigins": ["https://*.retool.com"],
    "AllowedMethods": ["PUT", "POST", "DELETE"],
    "AllowedHeaders": ["*"]
  },
  {
    "AllowedOrigins": ["*"],
    "AllowedMethods": ["GET"]
  }
]

🚧

On-premise instances

If you deployed Retool on-premise, you need to adjust your AllowedOrigins to reflect the domain on which your Retool instance is served on.

Get the Access key ID and Secret access key associated with your IAM User

Create an IAM User

Navigate to IAM, and make a new user. Programmatic access needs to be enabled in order to generate the access key ID and secret access key Retool needs to connect to S3.

You can attach policies to your user immediately if you’d like, but you can also add them later.

Keep your Access key ID and Secret access key—you'll need them on Retool's S3 resource setup page.

Create a new Access key ID and Secret access key for an existing user

First, find your user in IAM > User

If you no longer have your Access key ID and Secret access key for your user, you can create a new access key instead.

Create a Policy that describes permissions on your S3 bucket

To create a new policy, navigate to IAM > Policies > Create policy. The policy can be created using the GUI or by copying one of the JSON files shown below.

📘

IAM permissions best practices

While the simplest way to get Retool working with S3 is to give Retool full S3 access, the best practice is to restrict access to buckets on an as-needed basis.

You'll need to change the BUCKET_NAME variable to the name of your bucket. Make sure to keep both the BUCKET_NAME/\* and BUCKET_NAME resources - they're both required.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "VisualEditor0",
      "Effect": "Allow",
      "Action": ["s3:PutObject", "s3:GetObject", "s3:GetBucketLocation"],
      "Resource": ["arn:aws:s3:::BUCKET_NAME", "arn:aws:s3:::BUCKET_NAME/*"]
    }
  ]
}

For more robust permissions you can use the following:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket",
        "s3:GetBucketAcl",
        "s3:GetBucketCORS",
        "s3:GetBucketLocation",
        "s3:GetBucketLogging",
        "s3:GetBucketNotification",
        "s3:GetBucketPolicy",
        "s3:GetBucketWebsite",
        "s3:GetObject",
        "s3:GetObjectAcl",
        "s3:GetObjectVersion",
        "s3:GetObjectVersionAcl",
        "s3:PutObject",
        "s3:PutObjectAcl",
        "s3:PutObjectTagging",
        "s3:PutObjectVersionAcl",
        "s3:PutObjectVersionTagging"
      ],
      "Resource": ["arn:aws:s3:::BUCKET_NAME", "arn:aws:s3:::BUCKET_NAME/*"]
    }
  ]
}

You can always go back and edit your policy once it has been created.

Attach the policy to your IAM user

You can attach the policy directly to your IAM user, to the group the user is a part of, or to a role the user can assume.

Directly to the IAM User

Navigate to Users > YOUR_USER > Permissions > Add permissions

From there, you can choose to Attach existing policies directly and select the policy that contains permissions for your S3 bucket

To a group the user is a part of

Select the group you’d like to add the policy to in IAM > User groups, then select Add permissions > Attach policies and attach the policy containing permissions for your S3 bucket.

After adding the policy, you can navigate to the settings page of your user and add them to the group.

This automatically attaches all policies associated with the group to the user itself.

To a role the user can assume

If you want to avoid having permanent permissions for your S3 bucket granted to your IAM user, you can instead have your user assume a role that has sufficient permissions. To do so, navigate to IAM > Roles > Create role.

During setup, or after the role has been created, add a trust relationship that gives the IAM user permission to assume that role.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Statement1",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::<AWS_ACCOUNT_ID>:user/<IAM_USER_NAME>"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

This can be done on the role itself:

Additionally, attach the policy you’ve created that contains permissions for your S3 bucket to the role:

Add to Retool

Create a new resource in Retool and select Amazon S3 as the resource type. Enter the access key and secret generated for your IAM user, as well as the S3 bucket name and a default ACL. Click Create resource. At minimum you need to include your Access key ID and Secret access key ID.

If you’re granting permissions to your user using an assumed role, include the Role to assume (ARN) as well:

Uploading Files to S3

See here for docs on how to upload files to S3

Searching and downloading files from S3

Besides the S3 Button, you can also use the configured S3 resource as a datasource.

Filtering for files in a S3 bucket

Choose the S3 resource you configured previously in the Query Editor. By default it will fetch all files from your bucket. You can configure it to also filter files by a prefix. Below is an example of filtering files via a textinput, and then rendering the list of files in a Table.

Downloading files from S3

Let's extend our previous example to allow users to select a file and then click a button to download the selected file. Here's how that might look:

Now when a user clicks the "Download S3 File" button, it will run the buttonTrigger query which will fetch the file from S3 and download it.

Generating a Signed URL to Download Files

What if instead of just downloading the file, you want to generate a link that let's someone download a file in your S3 Bucket, but you want the link to expire after 60 seconds.

To do that, we'll use the Generate a signed url functionality that Retool offers. Try configuring the query like below. The Expires: 60 means that the URL will expire after 60 seconds -- you can change this to be as high as you'd like!

Press save, and then create a new text component to display the URL we generated using the above query.

Great! Now whenever we select a file in our table, our app will generate a signed url that we can use to download the file.

Connecting to S3 compatible endpoints (DigitalOcean Spaces, Wasabi, etc.)

Retool's S3 integration also works for S3-compatible object storage APIs like DigitalOcean Spaces. We'll talk through how to connect to a Space here, but the same concepts apply to other object storage solutions.

To get started, create a new resource (/resources/new) and choose the "S3" option.

• Name: we’ll call it “DigitalOcean Spaces” (creative, I know)

• Base URL: this is going to be <your_data_center>.digitaloceanspaces.com. Our Spaces is in DigitalOcean’s nyc3 data center, so the URL is nyc3.digitaloceanspaces.com. If you’re not sure which data center your Space is in, you can find this URL in your Space’s settings.

• Access keys: follow the tutorial here to generate an API key for Spaces. For the Access key ID, put in the first value. For the Secret access Key ID, put in the secret key.

After all of this not-so-tedious credential wrangling, this is what your resource form in Retool should look like (give or take the red boxes):

Just click “create resource” at the bottom right, and you should be ready to go.

🚧

Choosing a Space to connect to

If you only want to connect to one space, add the Space's name in the "S3 bucket name" field when connecting your resource. If you plan on building an app that connects to multiple Spaces, you can add that information in later on a query-by-query basis.

CORS for DigitalOcean Spaces

If you're looking to write to your Spaces, you'll need to add a CORS configuration to allow PUT and POST operations. Head to your Space's settings and add a config that allows requests from Retool:

In the “Origin” field, we’ve put in a wildcard to let in traffic from any Retool subdomain. You can limit traffic to your specific subdomain if you’d like, too (probably a good idea).