Paginated API
The following lab covers how to integrate with a paginated API using Retool Workflows.
Steps
The following steps focus on developing a Retool Workflow with a single-step function and code block to demonstrate paginated invocation.
Overview of Paginated API
This example relies on a simulated, paginated API that provides details of customers and the urgency level of their shipment. This workflow is focused on urgent shipments only, and thus we need to invoke the API to query for all records and then develop logic to filter through what has been received and extract only urgent shipments.
API Request
The paginated shipping API is describe below:
Input | Description | Value |
---|---|---|
Method | This is the HTTP Verb supported by the API | GET |
URI | The HTTP URI where the API is available at | https://qe1mvn42vg.execute-api.us-east-2.amazonaws.com/shipments/{{pageNum}}?disableDelay=false |
Path Parameter | pageNum identifies which page of data is being requested | This can be either 1 or 2 |
Query Parameter | The query parameter disable or enables an artificial delay | This can be either true to disable and false to enable. |
API Response
The response to the API returns 5 unique shipping records identified by per_page
. When using the pageNum
of 1
the attribute has_more
will be true. When using the pageNum
of 2
, it will be false
. The following is a portion of the API response take note of priority
and zone
, these will be used later in a function to calculate shipping costs:
{
"page": 1,
"per_page": 5,
"has_more": true,
"data":
[
{
"id": 1,
"name": "Leanne Graham",
"username": "lgraham",
"email": "lgraham@example.co",
"priority": "regular",
"zone": 1,
"address": {
"street": "915 Broadway",
"suite": "Suite 789",
"city": "New York",
"zipcode": "10001-3874",
"geo": {
"lat": "40.7128",
"lng": "-74.0060"
}
},
"phone": "212-555-5555 x56442",
"website": "simcompany.co",
"company": {
"name": "Sim Company",
"catchPhrase": "Multi-layered client-server neural-net",
"summary": "harness real-time e-markets"
}
},
{
"id": 2,
"name": "Thomas Walker",
"username": "twalker",
"email": "twalker@example.co",
"priority": "regular",
"zone": 1,
"address": {
"street": "51 Washington Ave",
"suite": "",
"city": "New Bedford",
"zipcode": "02740",
"geo": {
"lat": "41.6362",
"lng": "-70.933705"
}
},
"phone": "508-555-1212 x56442",
"website": "example.org",
"company": {
"name": "Example Widgets",
"catchPhrase": "Multi-layered client-server neural-net",
"summary": "harness real-time e-markets"
}
}...
startTrigger Block
This step focuses on initiating the workflow and specify a scheduled time for this workflow to occur.
- When creating the workflow, begin by selecting Workflows from the Retool Home Screen.
- Then select Create > Workflow and provide a name,
primary-shipping-workflow
. This will create a Workflow with a Start Trigger Block and a Code Block. - Select the code block and rename it to
paginatedAPIFunction
.
Create a single-step function
The next step focuses on calling the paginated API using a single-step function. We will use this function to call the api n-number of times based on the response from the API.
- Select Functions > +
- Update name from
function#
tocallAPI
. - Select + Add parameter and change the
param1
topageNum
and provide a test value of123
. - Keep the Query as
REST API
,GET
, and a URL shown below: - Use the following URL including disabling the delay by setting it to true.
https://qe1mvn42vg.execute-api.us-east-2.amazonaws.com/shipments/{{pageNum}}?disableDelay=true
Update Code block to invoke function, handle pagination and flatten results.
The next step is to invoke the single-step function, examine the results (has_more), and return the total aggregated response.
The following code sample does the following core actions:
- Create an empty response array called
result
. - Create a counter to keep track of current page.
- Call the CallAPI using that
counter
. - Examine response to determine if there is more data that can be accessed
has_more
and store the first page of results. - While there are more results, loop on the api, store the result and check if
has_more
is true or false. - Finally return the aggregated results.
//Create an empty array to store the results from API calls
const result = [];
//Call the API via a function using the starting pageNum;
let counter = 1;
//Call the callAPI function for the first page
let response = await callAPI(counter);
//Store the response in the results array
for(let i=0; i<response.data.data.length; i++){
result.push(response.data.data[i]);
}
//Does the response indicate more data
while (response.data.has_more){
counter = counter + 1;
response = await callAPI(counter);
//Store the response in the results array
for(let i=0; i<response.data.data.length; i++){
result.push(response.data.data[i]);
}
}
return result
Run the workflow
Run the workflow to confirm an aggregate set of results are provided.
{
"data": [
{
"id": 1,
"name": "Leanne Graham",
"username": "lgraham",
"email": "lgraham@example.co",
"priority": "regular",
"zone": 1,
"address": {
"street": "915 Broadway",
"suite": "Suite 789",
"city": "New York",
"zipcode": "10001-3874",
"geo": {
"latitude": 40.7128,
"longitude": -74.006
}
},
"phone": "212-555-5555 x56442",
"website": "simcompany.co",
"company": {
"name": "Sim Company",
"catchPhrase": "Multi-layered client-server neural-net",
"summary": "harness real-time e-markets"
}
},
{
"id": 2,
"name": "Thomas Walker",
"username": "twalker",
"email": "twalker@example.co",
"priority": "regular",
"zone": 1,
"address": {
"street": "51 Washington Ave",
"suite": "",
"city": "New Bedford",
"zipcode": "02740",
"geo": {
"latitude": 41.6362,
"longitude": -70.933705
}
},
"phone": "508-555-1212 x56442",
"website": "example.org",
"company": {
"name": "Example Widgets",
"catchPhrase": "Multi-layered client-server neural-net",
"summary": "harness real-time e-markets"
}
},
{
"id": 3,
"name": "Marie Gomes",
"username": "mgomes",
"email": "mgomes@example.co",
"priority": "regular",
"zone": 1,
"address": {
"street": "22 Harrison St",
"suite": "",
"city": "Jersey City",
"zipcode": "07030",
"geo": {
"latitude": 40.719074,
"longitude": -74.0500552
}
},
"phone": "214-555-5555 x56442",
"website": "tooling.io",
"company": {
"name": "Tooling Company",
"catchPhrase": "Multi-layered client-server neural-net",
"summary": "harness real-time e-markets"
}
},
{
"id": 4,
"name": "Victor Manson",
"username": "vmanson",
"email": "vmanson@example.co",
"priority": "urgent",
"zone": 1,
"address": {
"street": "901 Providence Way",
"suite": "3310",
"city": "Manchester",
"zipcode": "03101",
"geo": {
"latitude": 42.9956,
"longitude": 71.4548
}
},
"phone": "401-555-5555 x56442",
"website": "acme.org",
"company": {
"name": "Acme Widgets",
"catchPhrase": "Multi-layered client-server neural-net",
"summary": "harness real-time e-markets"
}
},
{
"id": 5,
"name": "Cecilia Orange",
"username": "corange",
"email": "corange@example.co",
"priority": "regular",
"zone": 1,
"address": {
"street": "101 Oak Drive",
"suite": "",
"city": "Burlington",
"zipcode": "05401-3874",
"geo": {
"latitude": 44.4759,
"longitude": -73.2121
}
},
"phone": "781-555-5555 x56442",
"website": "example.com",
"company": {
"name": "Example Pins",
"catchPhrase": "Multi-layered client-server neural-net",
"summary": "harness real-time e-markets"
}
},
{
"id": 6,
"name": "Mackenzie Lewis",
"username": "mlewis",
"email": "mlewise@example.co",
"priority": "urgent",
"zone": 2,
"address": {
"street": "Rt. 6a",
"suite": "",
"city": "Hyannis",
"zipcode": "02601",
"geo": {
"latitude": 41.6525,
"longitude": -70.2881
}
},
"phone": "508-555-5555 x56442",
"website": "work.biz",
"company": {
"name": "Work Business",
"catchPhrase": "Multi-layered client-server neural-net",
"summary": "harness real-time e-markets"
}
},
{
"id": 7,
"name": "Dominic Veras",
"username": "dveras",
"email": "dveras@example.co",
"priority": "regular",
"zone": 1,
"address": {
"street": "22 Progress St.",
"suite": "",
"city": "New Haven",
"zipcode": "06501",
"geo": {
"latitude": 41.3083,
"longitude": -72.9279
}
},
"phone": "203-555-5555 x56442",
"website": "prov.press",
"company": {
"name": "Providence Press",
"catchPhrase": "Multi-layered client-server neural-net",
"summary": "harness real-time e-markets"
}
},
{
"id": 8,
"name": "Selena Gomez",
"username": "sgomez",
"email": "sgomez@example.co",
"priority": "urgent",
"zone": 3,
"address": {
"street": "225 West 86th Street",
"suite": "Apt. 777",
"city": "New York",
"zipcode": "10001-3874",
"geo": {
"latitude": 40.7128,
"longitude": -74.006
}
},
"phone": "212-555-5555 x56442",
"website": "apartment.co",
"company": {
"name": "Apartments Company",
"catchPhrase": "Multi-layered client-server neural-net",
"summary": "harness real-time e-markets"
}
},
{
"id": 9,
"name": "Steve Martin",
"username": "smartin",
"email": "smartin@example.co",
"priority": "regular",
"zone": 3,
"address": {
"street": "225 West 86th Street",
"suite": "Apt. 556",
"city": "New York",
"zipcode": "10001-3874",
"geo": {
"latitude": 40.7128,
"longitude": -74.006
}
},
"phone": "212-555-5555 x56442",
"website": "detective.show",
"company": {
"name": "Detective Shows",
"catchPhrase": "Multi-layered client-server neural-net",
"summary": "harness real-time e-markets"
}
},
{
"id": 10,
"name": "Martin Short",
"username": "mshort",
"email": "mshort@example.co",
"priority": "regular",
"zone": 3,
"address": {
"street": "225 West 86th Street",
"suite": "Apt. 444",
"city": "New York",
"zipcode": "10001-3874",
"geo": {
"latitude": -37.3159,
"longitude": 81.1496
}
},
"phone": "212-555-5555 x56442",
"website": "broadway.show",
"company": {
"name": "Broadway Shows",
"catchPhrase": "Multi-layered client-server neural-net",
"summary": "harness real-time e-markets"
}
}
]
}
Summary
This example demonstrates the power of single-step functions with a Code block to connect to paginated APIs.