Token management for app building

Token management best practices for app building with AI.

What is a token?

A token is the atomic unit of text that LLMs process. It's roughly four characters, or about 0.75 English words, so every 100 tokens is about 75 English words. Everything the model receives counts toward your token limit, including your instructions, resource schemas, conversation history, tool outputs, and the model's own responses.

You can use OpenAI's tokenizer tool to estimate how a given passage will be counted.

Token costs grow faster than you might expect. In practice, your message is often only 1–2% of the total tokens processed. The rest is system context, history, and tool outputs that accumulate whether you notice it or not. This means optimizing the total size and shape of your context matters more than optimizing individual prompts.

How Retool charges for AI: credits vs. hours

Retool uses two different billing mechanisms for AI, depending on the feature:

• AI credits apply to apps, Assist, and AI resource queries. Credits are organization-level and shared across all builders. They reset monthly on your billing cycle and do not roll over. Prompts requiring more complex reasoning, larger outputs, or more data consume more credits. When an organization's monthly credits are exhausted, AI-powered features become unavailable until credits renew. Admins can track and buy more credits via the Plans & billing page in Settings. Refer to AI credits billing and usage for more information.
• Retool Agents are billed based on hourly usage. Agent usage is measured by the hour at a rate that varies by provider and whether you use a Retool-managed or self-managed LLM key. Refer to Agents billing and usage for more information.

Retool-managed keys vs. bring your own key (BYOK)

Retool provides managed API keys for OpenAI and Anthropic models, which lets you start building immediately without external accounts. However, Retool-managed connections are subject to rate limits and are intended for non-production use.

Retool imposes the following rate limits on Retool-managed key usage. Rate limits are per organization, per hour (over a rolling time window).

Feature	Hourly rate limit
Retool Agents	20,000,000 tokens
Assist	100,000,000 tokens
All other AI functionality	250,000 tokens

All other AI functionality includes AI resource queries, Ask AI, Retool Vectors, and the LLM Chat component.

This rate limit is intended to prevent abuse of resources and to limit Retool-managed key usage to non-production instances. Each AI provider also imposes their own rate limits, which also apply to Retool users. If you encounter a rate limit imposed by Retool or by an AI provider, you receive a 429 error with a message describing the kind of rate limit that was hit.

Rate limits are cumulative and encompass all AI usage. These rate limits apply to any customer using Retool-provided keys, regardless of deployment method or plan type. Retool is unable to make exceptions to or raise this rate limit.

For production use, configure AI platforms to use your own API credentials. Self-managed connections are not subject to Retool-imposed rate limits, though your provider's own limits still apply.

Token management for app building

Before you type anything into the prompt box, Retool pre-loads the following context:

• Resource schemas and TypeScript definitions.
• Function outputs (to understand your data shape).
• Frontend source files (.tsx, .css).
• Function source files.
• The package.json and dependency metadata.
• File tree listings.

This is necessary for the agent to understand your app's current state, but it means your baseline context is already significant. The following token management best practices can help reduce the additional context cost burden.

Be as precise as possible with prompting

Each failed attempt or clarification round costs credits because the agent re-reads the full context on every exchange. Precise prompts, specifying resources by name, explicit component names, and verified data shapes, produce accurate results on fewer passes. When working with data, pass queries into the chat by name rather than describing their output. The agent reads query results directly, which is more accurate and uses fewer tokens. Refer to Prompting best practices for guidance on how to structure effective prompts.

Select a model based on task complexity

Not every task requires the most expensive model. Simpler edits such as reorganizing a component, updating a label, and adjusting styles work well with less expensive models and use fewer credits. Reserve higher-capability models for complex, multi-step changes. Refer to AI models for a list of available models.

Use new threads for new information

Each thread in the Chat tab is a separate conversation with its own context window. Starting a new thread opens a fresh context and resets the preview canvas to the latest published version. Don't let one thread accumulate unrelated changes across multiple features. When the task shifts, publish your changes, and open a new thread. A clean context produces better results and uses fewer credits.