Monitor resource utilization

Learn how to monitor the memory and CPU utilization of your Retool instance.

When self-hosting Retool, you can allocate a specific amount of memory or CPU to the hosts where Retool’s containers are deployed. As more users access Retool apps, Retool’s containers use more of these resources.

Retool logs events that you can use to understand the resource utilization of your Retool deployment. Monitoring these logs helps you ensure that resource utilization stays within the limits of your hosting environment, and proactively scale resources as utilization increases.

Access logs

The method to access logs depends on your deployment provider. Retool uses several containers to deliver the application, and resource utilization logs are output by the api container.

For deployments using Docker Compose, SSH into your instance and use docker-compose logs:

1. Run sudo docker-compose ps to list running containers. Find the name of the container running the retool-onpremise_api image.
2. Run sudo docker-compose logs <API_CONTAINER_NAME> to view the container's logs.

For deployments using Kubernetes, connect to your instance and use kubectl logs:

kubectl logs -p <pod_name> -n <namespace>

You can use docker logging drivers to export logs to various log management services like AWS CloudWatch, Google Cloud Logging, DataDog, Splunk, etc. For example, send logs to Datadog by registering an agent. See the documentation for your log aggregator for more details.

Logs overview

The Retool api container runs several processes that support the Retool application. Each process has a random pid value assigned to it. Retool emits a resource utilization event per process every 20 seconds. See below for an example event:

{
  "namespace": "memoryUsage",
  "pid": NN,
  "message": {
    "pid": NN,
    "Memory Stats for process": {
        "rss": "254.099456 MB",
        "heapTotal": "159.8464 MB",
        "heapUsed": "144.332648 MB",
        "percentageHeapUsed": "90.29%"
    },
    "CPU Usage % for process": "0.02%",
    "Memory Stats for container": {
        "freeSystemMemory": "359.3216 MB",
        "totalSystemMemory": "4054.708224 MB",
        "percentageFreeMemory": "8.86%"
    }
},
"level": "info",
"timestamp": "2022-03-29T12:52:13.280Z"
}

Each event contains information for a given process and for the container as a whole. You can use the Memory Stats for container object to track memory utilization over time.

You can also approximate CPU utilization for the container by summing the most recent values of the CPU Usage % for process key across all running processes.

Use cases for logs

You can accomplish several common use cases by exporting resource utilization events to your log management service, including:

• Visualize resource utilization trends over time.
• Alert when resource utilization exceeds a predetermined threshold.