Skip to main content

Paginated API

The following lab covers how to integrate with a paginated API using Retool Workflows.

Steps​

The following steps focus on developing a Retool Workflow with a single-step function and code block to demonstrate paginated invocation.

Overview of Paginated API​

This example relies on a simulated, paginated API that provides details of customers and the urgency level of their shipment. This workflow is focused on urgent shipments only, and thus we need to invoke the API to query for all records and then develop logic to filter through what has been received and extract only urgent shipments.

API Request​

The paginated shipping API is describe below:

InputDescriptionValue
MethodThis is the HTTP Verb supported by the APIGET
URIThe HTTP URI where the API is available athttps://qe1mvn42vg.execute-api.us-east-2.amazonaws.com/shipments/{{pageNum}}?disableDelay=false
Path ParameterpageNum identifies which page of data is being requestedThis can be either 1 or 2
Query ParameterThe query parameter disable or enables an artificial delayThis can be either true to disable and false to enable.

API Response​

The response to the API returns 5 unique shipping records identified by per_page. When using the pageNum of 1 the attribute has_more will be true. When using the pageNum of 2, it will be false. The following is a portion of the API response take note of priority and zone, these will be used later in a function to calculate shipping costs:

{
    "page": 1,
    "per_page": 5,
    "has_more": true,
    "data": 
        [
            {
                "id": 1,
                "name": "Leanne Graham",
                "username": "lgraham",
                "email": "lgraham@example.co",
                "priority": "regular",
                "zone": 1,
                "address": {
                    "street": "915 Broadway",
                    "suite": "Suite 789", 
                    "city": "New York",
                    "zipcode": "10001-3874",
                    "geo": {
                        "lat": "40.7128",
                        "lng": "-74.0060"
                    }
                },
                "phone": "212-555-5555 x56442",
                "website": "simcompany.co",
                "company": {
                    "name": "Sim Company",
                    "catchPhrase": "Multi-layered client-server neural-net",
                    "summary": "harness real-time e-markets"
                }
            },
            {
                "id": 2,
                "name": "Thomas Walker",
                "username": "twalker",
                "email": "twalker@example.co",
                "priority": "regular",
                "zone": 1,
                "address": {
                    "street": "51 Washington Ave",
                    "suite": "",
                    "city": "New Bedford",
                    "zipcode": "02740",
                    "geo": {
                        "lat": "41.6362",
                        "lng": "-70.933705"
                    }
                },
                "phone": "508-555-1212 x56442",
                "website": "example.org",
                "company": {
                    "name": "Example Widgets",
                    "catchPhrase": "Multi-layered client-server neural-net",
                    "summary": "harness real-time e-markets"
                }
            }...

startTrigger Block​

This step focuses on initiating the workflow and specify a scheduled time for this workflow to occur.

  • When creating the workflow, begin by selecting Workflows from the Retool Home Screen.
  • Then select Create > Workflow and provide a name, primary-shipping-workflow. This will create a Workflow with a Start Trigger Block and a Code Block.
  • Select the code block and rename it to paginatedAPIFunction.

Create a single-step function​

The next step focuses on calling the paginated API using a single-step function. We will use this function to call the api n-number of times based on the response from the API.

  • Select Functions > +
  • Update name from function# to callAPI.
  • Select + Add parameter and change the param1 to pageNum and provide a test value of 123.
  • Keep the Query as REST API, GET, and a URL shown below:
  • Use the following URL including disabling the delay by setting it to true.
https://qe1mvn42vg.execute-api.us-east-2.amazonaws.com/shipments/{{pageNum}}?disableDelay=true
Single-step function
Single-step function

Update Code block to invoke function, handle pagination and flatten results.​

The next step is to invoke the single-step function, examine the results (has_more), and return the total aggregated response.

The following code sample does the following core actions:

  • Create an empty response array called result.
  • Create a counter to keep track of current page.
  • Call the CallAPI using that counter.
  • Examine response to determine if there is more data that can be accessed has_more and store the first page of results.
  • While there are more results, loop on the api, store the result and check if has_more is true or false.
  • Finally return the aggregated results.
//Create an empty array to store the results from API calls
const result = [];

//Call the API via a function using the starting pageNum;
let counter = 1;

//Call the callAPI function for the first page
let response = await callAPI(counter);

//Store the response in the results array
for(let i=0; i<response.data.data.length; i++){
  result.push(response.data.data[i]);
}

//Does the response indicate more data
while (response.data.has_more){
  counter = counter + 1;
  response = await callAPI(counter);

  //Store the response in the results array
  for(let i=0; i<response.data.data.length; i++){
    result.push(response.data.data[i]);
  }
}

return result

Run the workflow​

Run the workflow to confirm an aggregate set of results are provided.

{
  "data": [
    {
      "id": 1,
      "name": "Leanne Graham",
      "username": "lgraham",
      "email": "lgraham@example.co",
      "priority": "regular",
      "zone": 1,
      "address": {
        "street": "915 Broadway",
        "suite": "Suite 789",
        "city": "New York",
        "zipcode": "10001-3874",
        "geo": {
          "latitude": 40.7128,
          "longitude": -74.006
        }
      },
      "phone": "212-555-5555 x56442",
      "website": "simcompany.co",
      "company": {
        "name": "Sim Company",
        "catchPhrase": "Multi-layered client-server neural-net",
        "summary": "harness real-time e-markets"
      }
    },
    {
      "id": 2,
      "name": "Thomas Walker",
      "username": "twalker",
      "email": "twalker@example.co",
      "priority": "regular",
      "zone": 1,
      "address": {
        "street": "51 Washington Ave",
        "suite": "",
        "city": "New Bedford",
        "zipcode": "02740",
        "geo": {
          "latitude": 41.6362,
          "longitude": -70.933705
        }
      },
      "phone": "508-555-1212 x56442",
      "website": "example.org",
      "company": {
        "name": "Example Widgets",
        "catchPhrase": "Multi-layered client-server neural-net",
        "summary": "harness real-time e-markets"
      }
    },
    {
      "id": 3,
      "name": "Marie Gomes",
      "username": "mgomes",
      "email": "mgomes@example.co",
      "priority": "regular",
      "zone": 1,
      "address": {
        "street": "22 Harrison St",
        "suite": "",
        "city": "Jersey City",
        "zipcode": "07030",
        "geo": {
          "latitude": 40.719074,
          "longitude": -74.0500552
        }
      },
      "phone": "214-555-5555 x56442",
      "website": "tooling.io",
      "company": {
        "name": "Tooling Company",
        "catchPhrase": "Multi-layered client-server neural-net",
        "summary": "harness real-time e-markets"
      }
    },
    {
      "id": 4,
      "name": "Victor Manson",
      "username": "vmanson",
      "email": "vmanson@example.co",
      "priority": "urgent",
      "zone": 1,
      "address": {
        "street": "901 Providence Way",
        "suite": "3310",
        "city": "Manchester",
        "zipcode": "03101",
        "geo": {
          "latitude": 42.9956,
          "longitude": 71.4548
        }
      },
      "phone": "401-555-5555 x56442",
      "website": "acme.org",
      "company": {
        "name": "Acme Widgets",
        "catchPhrase": "Multi-layered client-server neural-net",
        "summary": "harness real-time e-markets"
      }
    },
    {
      "id": 5,
      "name": "Cecilia Orange",
      "username": "corange",
      "email": "corange@example.co",
      "priority": "regular",
      "zone": 1,
      "address": {
        "street": "101 Oak Drive",
        "suite": "",
        "city": "Burlington",
        "zipcode": "05401-3874",
        "geo": {
          "latitude": 44.4759,
          "longitude": -73.2121
        }
      },
      "phone": "781-555-5555 x56442",
      "website": "example.com",
      "company": {
        "name": "Example Pins",
        "catchPhrase": "Multi-layered client-server neural-net",
        "summary": "harness real-time e-markets"
      }
    },
    {
      "id": 6,
      "name": "Mackenzie Lewis",
      "username": "mlewis",
      "email": "mlewise@example.co",
      "priority": "urgent",
      "zone": 2,
      "address": {
        "street": "Rt. 6a",
        "suite": "",
        "city": "Hyannis",
        "zipcode": "02601",
        "geo": {
          "latitude": 41.6525,
          "longitude": -70.2881
        }
      },
      "phone": "508-555-5555 x56442",
      "website": "work.biz",
      "company": {
        "name": "Work Business",
        "catchPhrase": "Multi-layered client-server neural-net",
        "summary": "harness real-time e-markets"
      }
    },
    {
      "id": 7,
      "name": "Dominic Veras",
      "username": "dveras",
      "email": "dveras@example.co",
      "priority": "regular",
      "zone": 1,
      "address": {
        "street": "22 Progress St.",
        "suite": "",
        "city": "New Haven",
        "zipcode": "06501",
        "geo": {
          "latitude": 41.3083,
          "longitude": -72.9279
        }
      },
      "phone": "203-555-5555 x56442",
      "website": "prov.press",
      "company": {
        "name": "Providence Press",
        "catchPhrase": "Multi-layered client-server neural-net",
        "summary": "harness real-time e-markets"
      }
    },
    {
      "id": 8,
      "name": "Selena Gomez",
      "username": "sgomez",
      "email": "sgomez@example.co",
      "priority": "urgent",
      "zone": 3,
      "address": {
        "street": "225 West 86th Street",
        "suite": "Apt. 777",
        "city": "New York",
        "zipcode": "10001-3874",
        "geo": {
          "latitude": 40.7128,
          "longitude": -74.006
        }
      },
      "phone": "212-555-5555 x56442",
      "website": "apartment.co",
      "company": {
        "name": "Apartments Company",
        "catchPhrase": "Multi-layered client-server neural-net",
        "summary": "harness real-time e-markets"
      }
    },
    {
      "id": 9,
      "name": "Steve Martin",
      "username": "smartin",
      "email": "smartin@example.co",
      "priority": "regular",
      "zone": 3,
      "address": {
        "street": "225 West 86th Street",
        "suite": "Apt. 556",
        "city": "New York",
        "zipcode": "10001-3874",
        "geo": {
          "latitude": 40.7128,
          "longitude": -74.006
        }
      },
      "phone": "212-555-5555 x56442",
      "website": "detective.show",
      "company": {
        "name": "Detective Shows",
        "catchPhrase": "Multi-layered client-server neural-net",
        "summary": "harness real-time e-markets"
      }
    },
    {
      "id": 10,
      "name": "Martin Short",
      "username": "mshort",
      "email": "mshort@example.co",
      "priority": "regular",
      "zone": 3,
      "address": {
        "street": "225 West 86th Street",
        "suite": "Apt. 444",
        "city": "New York",
        "zipcode": "10001-3874",
        "geo": {
          "latitude": -37.3159,
          "longitude": 81.1496
        }
      },
      "phone": "212-555-5555 x56442",
      "website": "broadway.show",
      "company": {
        "name": "Broadway Shows",
        "catchPhrase": "Multi-layered client-server neural-net",
        "summary": "harness real-time e-markets"
      }
    }
  ]
}

Summary​

This example demonstrates the power of single-step functions with a Code block to connect to paginated APIs.