Assist Token management

Token management best practices for building apps with Assist.

What is a token?

A token is the atomic unit of text that LLMs process. It's roughly four characters, or about 0.75 English words, so every 100 tokens is about 75 English words. Everything the model receives counts toward your token limit, including your instructions, resource schemas, conversation history, tool outputs, and the model's own responses.

You can use OpenAI's tokenizer tool to estimate how a given passage will be counted.

Token costs grow faster than you might expect. In practice, your message is often only 1–2% of the total tokens processed. The rest is system context, history, and tool outputs that accumulate whether you notice it or not. This means optimizing the total size and shape of your context matters more than optimizing individual prompts.

How Retool charges for AI: credits vs. hours

Retool uses two different billing mechanisms for AI, depending on the feature:

• AI credits apply to apps, Assist, and AI resource queries. Credits are organization-level and shared across all builders. They reset monthly on your billing cycle and do not roll over. Prompts requiring more complex reasoning, larger outputs, or more data consume more credits. When an organization's monthly credits are exhausted, AI-powered features become unavailable until credits renew. Admins can track and buy more credits via the Plans & billing page in Settings. Refer to AI credits billing and usage for more information.
• Retool Agents are billed based on hourly usage. Agent usage is measured by the hour at a rate that varies by provider and whether you use a Retool-managed or self-managed LLM key. Refer to Agents billing and usage for more information.

Retool-managed keys vs. bring your own key (BYOK)

Retool provides managed API keys for OpenAI and Anthropic models, which lets you start building immediately without external accounts. However, Retool-managed connections are subject to rate limits and are intended for non-production use.

Retool imposes the following rate limits on Retool-managed key usage. Rate limits are per organization, per hour (over a rolling time window).

Feature	Hourly rate limit
Retool Agents	20,000,000 tokens
Assist	100,000,000 tokens
All other AI functionality	250,000 tokens

All other AI functionality includes AI resource queries, Ask AI, Retool Vectors, and the LLM Chat component.

This rate limit is intended to prevent abuse of resources and to limit Retool-managed key usage to non-production instances. Each AI provider also imposes their own rate limits, which also apply to Retool users. If you encounter a rate limit imposed by Retool or by an AI provider, you receive a 429 error with a message describing the kind of rate limit that was hit.

Rate limits are cumulative and encompass all AI usage. These rate limits apply to any customer using Retool-provided keys, regardless of deployment method or plan type. Retool is unable to make exceptions to or raise this rate limit.

For production use, configure AI platforms to use your own API credentials. Self-managed connections are not subject to Retool-imposed rate limits, though your provider's own limits still apply.

Token management for Assist

Use Ask mode first

Use Ask mode prior to building to get code snippets or explanations before asking Assist to create or rewrite app components directly. This prevents Assist from repeatedly rebuilding your layout.

Be as precise as possible with prompting

Each failed attempt or clarification round costs credits because Assist re-reads the full context on every exchange. Precise prompts, specifying resources by name, explicit component names, and verified data shapes, produce accurate results on fewer passes. Refer to Prompt Assist for guidance on how to structure effective prompts.

Use new threads for new information

Each thread in the Chat tab is a separate conversation with its own context window. Don't let one thread accumulate unrelated changes across multiple features. When the task shifts, open a new thread. A clean context produces better results and uses fewer credits.

Use custom components

If your app requires complex data visualization, rendering, or text editing, every native component you add to a canvas adds a substantial JSON object to the Component tree. Custom components help optimize token efficiency by acting as a black box within Assist. When you use native Retool components, Assist reads their configuration, layout metadata, property values, and data dependencies as part of the context, which drives up token cost. When you use a custom component, Assist only sees a single component object in the tree.