Skip to main content

Cost management

The Retool Platform when run in a self-hosted environment (e.g., AWS, Azure, Google) will rely on compute, storage, networking, and security infrastructure. Many of these services have cost associated with them so it is important to configure budget and cost thresholds and supporting notifications. The following best practices provide links to vendor specific suggestions.

Best practices​

Leverage cloud provider cost management (self-hosted)​

Resource: https://docs.aws.amazon.com/wellarchitected/latest/framework/cost-cfm.html
Resource: https://learn.microsoft.com/en-us/azure/well-architected/#cost-optimization
Resource: https://cloud.google.com/cost-management

To manage costs effectively with the respective cloud provider, please refer to your Cloud Provider’s cost management solutions and best practices for tracking spend. These provide mechanisms for setting up budgets and associated notifications when approaching or exceeding a prescribed budget.

Employ tagging best practices (self-hosted)​

Resource: https://docs.aws.amazon.com/whitepapers/latest/tagging-best-practices/tagging-best-practices.html
Resource: https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/ready/azure-best-practices/naming-and-tagging
Resource: https://cloud.google.com/resource-manager/docs/tags/tags-creating-and-managing

Tagging within cloud providers is the mechanism where resources can be decorated with additional metadata. Cost management solutions utilize tags for filtering on collected cost data. An examples is to add a cost center tag and add these to virtual machines on startup (e.g., Development, Operations etc). At a later time, analysts can then perform queries on the cost center tag and categorize who owns the particular resource (e.g. VM, Disk, Database) when doing monthly spend reporting.

Enable cost monitoring with Kubernetes (self-hosted)​

Resource: https://docs.aws.amazon.com/eks/latest/userguide/cost-monitoring.html
Resource: https://docs.kubecost.com/

Kubecost is a cost tracking provider that supports AWS, Azure, and GCP Kubernetes implementations. Using this solution, one can monitor the costs of a Kubernetes cluster above and beyond what is provided by the cloud provider’s cost management solutions.

Leverage elastic infrastructure to scale in and out (self-hosted)​

Resource: https://aws.github.io/aws-eks-best-practices/cluster-autoscaling/
Resource: https://docs.aws.amazon.com/eks/latest/userguide/horizontal-pod-autoscaler.html
Resource: https://learn.microsoft.com/en-us/azure/aks/concepts-scale
Resource: https://cloud.google.com/kubernetes-engine/docs/how-to/scaling-apps
Resource: https://cloud.google.com/kubernetes-engine/docs/how-to/cluster-autoscaler

Optimize your infrastructure to employ only what you need based on current demand. Using monitoring solutions, the health of your cluster(s) can be monitored and automated scaling policies applied to optimize resources and reduce your overall spend. Compute solutions such as Kubernetes provide support for Cluster Autoscaler and Karpenter for node provisioning and Horizontal Pod Autoscaler (HPA) / Vertical Pod Autoscaler (VPA) for pod provisioning.

Manage AI spend​

Resource: AI Spend Management
Resource: App Spend Management lab
Resource: AI Monitoring lab
Resource: Token management best practices for app building
Resource: Prompting best practices for apps
Resource: AI credits

Retool AI features add a second cost beside infrastructure. Usage draws on AI credits when you use Retool-managed keys, or on tokens billed by your model provider when you bring your own key (BYOK). Builder and organization credit pools reset each billing cycle and don't roll over. Purchased credit packs remain valid until the end of the next billing cycle. When they run out, AI-powered building pauses until the next cycle or until you add more credits. Decide which workloads use Retool-managed keys and which use BYOK, and name an owner for AI spend.

Use the AI Monitoring page in Settings to monitor credit and token consumption by feature, user, and model, and hold a monthly review to catch spikes early. Restrict AI resources to approved models, grant access by group, and use spend controls to set notification and blocking limits. Start with notification limits, set limits by group instead of by person, and add blocking limits last. Train builders to plan before building, use one feature per prompt, start a new thread for each feature, and make small changes in the Code tab, which uses no credits.