Retool AI providers and models
Learn about AI providers, their models, and how they're configured with Retool.
For a list of all the models that Retool supports, refer to the AI models reference.
AI providers use models that are trained on specific types of data to make relevant decisions and perform tasks. For example, you would interact with one type of model to generate chat responses and a different model to generate images. AI models are also known as LLMs (large language models).
In general, a model is a collective group of model instances, each of which vary in terms of functionality or data. For example, there are different instances of OpenAI's GPT-4 model, such as GPT-4o and GPT-4o-mini. You can specify the exact model instance when configuring Retool AI.
Retool supports a number of AI providers and their models for use with AI actions and agents.
AI tokens
Use OpenAI's tokenizer tool to calculate how many tokens would result from a passage of text.
AI models use tokens, rather than characters or bytes, to represent the commonality of sequences and usage. In general, a single token can equate to approximately four characters.
The text you provide in an AI query is converted by the model into tokens, after which it can generate the necessary response.
AI credits
Refer to the Billing and usage guide to learn more about how Retool monitors and charges for usage.
Retool organizations receive a monthly credit allocation that resets at midnight UTC on the first day of each billing cycle. Credits are the unit Retool uses to measure AI usage across your team and are consumed when using AI features such as Retool Agents, Assist, AI resource queries, and other AI functionality.
Admins can monitor AI credit usage and balance in the organization's Plans & Billing settings under the AI Credits tab.
Retool-managed and self-managed models
LLM interactions work by making API requests with an AI provider. As with most other APIs, these requests require authorization credentials, such as an API key. Retool provides access to certain AI models using Retool-managed keys.
A Retool-managed LLM connection enables you to start building and testing with AI models straight away without needing any additional tools or services. Once you're ready to use your AI actions or agents in production, you then need to provide your own authorization credentials. The connection then becomes self-managed.
For instance, you can immediately start building an agent using GPT-4o-mini with the Retool-managed OpenAI connection. Once your agent is ready for use, you then update the OpenAI connection and provide your own OpenAI API key.
Some AI models are only available for use as self-managed connections.
AI usage rate limiting
Retool imposes the following rate limits on Retool-managed key usage. Rate limits are per organization, per hour (over a rolling time window).
| Feature | Hourly rate limit |
|---|---|
| Retool Agents | 20,000,000 tokens |
| Assist | 100,000,000 tokens |
| All other AI functionality | 250,000 tokens |
All other AI functionality includes AI resource queries, Ask AI, Retool Vectors, and the LLM Chat component.
This rate limit is intended to prevent abuse of resources and to limit Retool-managed key usage to non-production instances. Each AI provider also imposes their own rate limits, which also apply to Retool users. If you encounter a rate limit imposed by Retool or by an AI provider, you receive a 429 error with a message describing the kind of rate limit that was hit.
Rate limits are cumulative and encompass all AI usage. These rate limits apply to any customer using Retool-provided keys, regardless of deployment method or plan type. Retool is unable to make exceptions to or raise this rate limit.
For production use, configure AI platforms to use your own API credentials. Self-managed connections are not subject to Retool-imposed rate limits, though your provider's own limits still apply.
Using your own key with Assist
Organizations on an enterprise plan can choose to use their own key with Assist. If you choose to do so, ensure that your API key can support the number of tokens that Assist requires. For each simultaneous user, Retool recommends that you account for 80,000 input tokens per minute (ITPM). This correlates to a minimum of Tier 2 in Anthropic's pricing model. Pricing for OpenAI differs based on your plan and is measured in increments of 1 million input tokens.