Configuring S3

Making a new S3 IAM user

Head over to IAM, make a new user, and call it retool-s3-uploader. Only enable "programmatic access".

Hit "next" to grant the account permissions. The easiest is granting it full S3 permissions, but if you want, you can further restrict the permissions. You'll need to create a new policy, then attach the policy to the new user.

Hit "next" to add optional tags.

Hit "next" to review your user details and permissions summary.

Success! Keep your Access key and Secret access key—you'll need them in Retool's S3 resource setup page.

📘

IAM Permissions: best practices

While the simplest way to get Retool working with S3 is to give Retool full S3 access, the best practice is to restrict access to buckets on an as-needed basis.

Here's an example JSON IAM policy that works. You'll need to change the BUCKET_NAME variable to the name of your bucket. Keep both the BUCKET_NAME/* and BUCKET_NAME resources - they're both necessary!

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetBucketAcl",
                "s3:GetBucketCORS",
                "s3:GetBucketLocation",
                "s3:GetBucketLogging",
                "s3:GetBucketNotification",
                "s3:GetBucketPolicy",
                "s3:GetBucketWebsite",
                "s3:GetObject",
                "s3:GetObjectAcl",
                "s3:GetObjectVersion",
                "s3:GetObjectVersionAcl",
                "s3:PutObject",
                "s3:PutObjectAcl",
                "s3:PutObjectTagging",
                "s3:PutObjectVersionAcl",
                "s3:PutObjectVersionTagging"
            ],
            "Resource": [
                "arn:aws:s3:::BUCKET_NAME",
                "arn:aws:s3:::BUCKET_NAME/*"
            ]
        }
    ]
}

📘

Minimum Grants for Uploading Files

The following actions are required to at least upload files to S3 with Retool.

            "s3:GetBucketLocation"
            "s3:PutObject"

Configuring CORS

Since we upload directly from your browser, you'll need to configure CORS (cross origin resource sharing). Open up the S3 bucket, click the Permissions tab, and then click CORS configuration, and paste in the following JSON, which lets Retool upload directly in to your S3 bucket from the browser.

🚧

Configuring CORS in the S3 console

If you are configuring CORS in the S3 console, you'll need to use JSON to create a CORS configuration. The new S3 console does not support XML CORS configurations.

[
  {
    "AllowedOrigins": [
    "https://*.retool.com"
    ],
    "AllowedMethods": [
    "PUT",
    "POST",
    "DELETE"
    ],
    "AllowedHeaders": ["*"]
  },
  {
    "AllowedOrigins": ["*"],
    "AllowedMethods": ["GET"]
  }
]

<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<CORSRule>
    <AllowedOrigin>https://*.retool.com</AllowedOrigin>
    <AllowedMethod>PUT</AllowedMethod>
    <AllowedMethod>POST</AllowedMethod>
    <AllowedMethod>DELETE</AllowedMethod>
    <AllowedHeader>*</AllowedHeader>
</CORSRule>
<CORSRule>
  <AllowedOrigin>*</AllowedOrigin>
  <AllowedMethod>GET</AllowedMethod>
</CORSRule>
</CORSConfiguration>

That's it! you should be good to go.

Add to Retool

Create a new resource in Retool and select Amazon S3 as the resource type. Enter the access key and secret generated for your IAM user, as well as the S3 bucket name and a default ACL. Press Create resource.

Uploading Files to S3

See here for docs on how to upload files to S3

Searching and downloading files from S3

Besides the S3 Button, you can also use the configured S3 resource as a datasource.

Filtering for files in a S3 bucket

Choose the S3 resource you configured previously in the Query Editor. By default it will fetch all files from your bucket. You can configure it to also filter files by a prefix. Below is an example of filtering files via a textinput, and then rendering the list of files in a Table.

Downloading files from S3

Let's extend our previous example to allow users to select a file and then click a button to download the selected file. Here's how that might look:

Now when a user clicks the "Download S3 File" button, it will run the buttonTrigger query which will fetch the file from S3 and download it.

Generating a Signed URL to Download Files

What if instead of just downloading the file, you want to generate a link that let's someone download a file in your S3 Bucket, but you want the link to expire after 60 seconds.

To do that, we'll use the Generate a signed url functionality that Retool offers. Try configuring the query like below. The Expires: 60 means that the URL will expire after 60 seconds -- you can change this to be as high as you'd like!

Press save, and then create a new text component to display the URL we generated using the above query.

Great! Now whenever we select a file in our table, our app will generate a signed url that we can use to download the file.

Connecting to S3 compatible endpoints (DigitalOcean Spaces, Wasabi, etc.)

Retool's S3 integration also works for S3-compatible object storage APIs like DigitalOcean Spaces. We'll talk through how to connect to a Space here, but the same concepts apply to other object storage solutions.

To get started, create a new resource (/resources/new) and choose the "S3" option.

• Name: we’ll call it “DigitalOcean Spaces” (creative, I know)

• Base URL: this is going to be <your_data_center>.digitaloceanspaces.com. Our Spaces is in DigitalOcean’s nyc3 data center, so the URL is nyc3.digitaloceanspaces.com. If you’re not sure which data center your Space is in, you can find this URL in your Space’s settings.

• Access keys: follow the tutorial here to generate an API key for Spaces. For the Access key ID, put in the first value. For the Secret access Key ID, put in the secret key.

After all of this not-so-tedious credential wrangling, this is what your resource form in Retool should look like (give or take the red boxes):

Just click “create resource” at the bottom right, and you should be ready to go.

🚧

Choosing a Space to connect to

If you only want to connect to one space, add the Space's name in the "S3 bucket name" field when connecting your resource. If you plan on building an app that connects to multiple Spaces, you can add that information in later on a query-by-query basis.

CORS for DigitalOcean Spaces

If you're looking to write to your Spaces, you'll need to add a CORS configuration to allow PUT and POST operations. Head to your Space's settings and add a config that allows requests from Retool:

In the “Origin” field, we’ve put in a wildcard to let in traffic from any Retool subdomain. You can limit traffic to your specific subdomain if you’d like, too (probably a good idea).