Skip to main content

Configure your organization for Agents

Configure Agents for cloud and self-hosted instances.

An organization admin must complete the steps in the following sections to make sure your organization is ready to use Agents.

note

To learn how to create and use agents, visit the Agents quickstart or the Agents tutorials.

Configure organization settings​

Ensure that Agents is enabled and that it has access to the proper AI models.

  1. To ensure your users can access Agents, navigate to Settings > AI. Verify that the Retool AI and AI Agents settings are toggled on.
  2. Navigate to Resources > Retool AI. Ensure that at least one AI provider is enabled. You can choose to use Retool-managed keys or self-managed keys.
note

Only AI models that support tool creation can be used with Agents. Refer to AI providers and models for supported models.

When users in your organization create a new agent, Retool automatically selects the first available enabled AI provider resource as the default model. Make sure the providers you want to expose are enabled in the order you prefer.

Self-hosted configuration​

Complete the steps in the following sections if you self-host Retool. Skip these steps if you use cloud.

Requirements​

To use Agents on self-hosted instances, your organization must:

  • Use Retool version 3.234.0 or later (3.253.0 or later for stable releases).
  • Have its own AI model key(s) configured.
  • Meet all of the requirements outlined in the tutorial for your chosen deployment infrastructure.

Installation​

Agents is supported for the following self-hosted deployment options.

Follow the instructions outlined in the Docker deployment tutorial.

Docker is for non-production use only

Docker deployments are not suitable for production use. For production deployments, use Kubernetes with Helm.

Additional configuration​

Agents on self-hosted instances rely on Temporal. To deploy agents, you must create two additional containers with configuration and networking egress identical to the workflows-worker container:

ContainerSERVICE_TYPE Environment Variable 1NetworksDependenciesRestart Policy
agent-workerAGENT_TEMPORAL_WORKERbackend, code-executorpostgresalways
agent-eval-workerAGENT_EVAL_TEMPORAL_WORKERbackend, code-executorpostgresalways

The infrastructure of the Agents edge deployment is very similar to a Retool deployment with workflows. The following example code block shows the agent-worker and agent-eval-worker container configuration for a Docker Compose deployment.

docker-compose.yaml example
services:
  # Temporal workers for Agents
  agent-worker:
    build:
      context: .
    env_file: docker.env
    environment:
      - SERVICE_TYPE=AGENT_TEMPORAL_WORKER
    networks:
      - backend
      - code-executor
    depends_on:
      - postgres
    restart: always

  agent-eval-worker:
    build:
      context: .
    env_file: docker.env
    environment:
      - SERVICE_TYPE=AGENT_EVAL_TEMPORAL_WORKER
    networks:
      - backend
      - code-executor
    depends_on:
      - postgres
    restart: always
note

Using a Retool-managed agents observability API is required in the edge release. This API strictly does not receive, persist, or have access to sensitive customer data, agent inputs, or agent outputs. It is required to power the Monitor dashboard.

Set up evals​

Evals work in self-hosted deployments without any special configuration. To optimize performance, you can configure Amazon Simple Storage Service (Amazon S3) to store test case inputs and outputs (which can be large) instead of Postgres. To use Amazon S3, first configure Amazon S3 as a resource, and then set the following environment variables:

  # Replace the following example values with your S3 values.
  AGENT_EVALS_S3_BUCKET='retool-agent-evals'
  AGENT_EVALS_S3_ACCESS_KEY_ID='AKIAIOSFODNN7EXAMPLE'
  AGENT_EVALS_S3_SECRET_ACCESS_KEY='wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY'
  AGENT_EVALS_S3_REGION='us-west-2'

Configure AI providers​

After installation, navigate to <your domain>/resources/retool_ai and configure one or more AI model providers.

Configure advanced model settings​

After selecting an AI model for your agent, you can configure advanced settings from the agent's Configuration tab.

Extended thinking​

Extended thinking is available for the following OpenAI models: o3, o3-mini, GPT-5, GPT-5.1, GPT-5.2, and GPT-5 Mini.

When enabled, the model reasons through complex problems step-by-step before responding. This can improve accuracy on multi-step tasks but increases response time and may use more tokens.

Extended thinking is disabled by default. To enable it:

  1. Open the agent in Retool and navigate to the Configuration tab.
  2. Under Model, select a supported OpenAI model.
  3. Toggle on Extended thinking in the advanced settings section.

Choose a change management strategy​

Agents can be versioned and protected using Retool's change management features:

  • Source Control: Organizations on the Enterprise plan can use Source Control to protect agents and manage changes through Git workflows. Consider creating feature branches when making significant changes to agents.
  • Version History: All changes to agents are automatically tracked in version history, allowing you to review and restore previous versions.

Track usage​

Organization admins can monitor Agents usage and costs.

Monitor agent runs​

Use the Monitor dashboard to track:

  • Agent run success or failure rates
  • LLM token usage
  • Runtime performance
  • Tool invocations

Billing and usage limits​

Retool organizations receive up to $50 worth of agent runtime per month. For self-hosted instances using self-managed LLMs, this equals 10 hours of free usage per month at $5/hour.

If you exceed the cap, agents will not execute and you'll see a warning message. Contact your account team to discuss additional capacity.

For more information about pricing, refer to the Model pricing and the Agents billing and usage pages.

Next steps​

Once configuration is complete:

  1. Create your first agent using the Agents tutorial page.
  2. Configure agent tools to connect to your data sources.
  3. Set up permissions to control who can create and use agents.
  4. (Optional) Configure email triggers or A2A triggers for automated agent invocation.
  5. For answers to common questions about Agents, review the Agents FAQ.

Footnotes​

  1. For a complete list of Agents-related environment variables, refer to the Environment variables reference. ↩