App testing
Write and run tests that verify your app's behavior.
App testing enables you to validate that your app is functioning as expected and catch unintended changes before you publish your app to end users.
App testing is currently rolling out to cloud organizations. It is not yet available to self-hosted organizations.
How it works
When app testing is enabled, the agent writes or updates tests whenever it adds or changes behavior in your app. By default, the agent writes a small set of behavioral tests, which evaluate a user flow and check the outcomes, rather than evaluate how the code is implemented. You can also ask the agent for other kinds of tests, such as unit tests for a specific function.
After the agent makes any changes to your app, Retool runs the tests related to those changes. If any of them fail, Retool sends the failures to the agent, which continues working to fix them. The agent doesn't weaken a test to make it pass, and it only changes what a test expects when you ask for a change in behavior.
Tests never query your resources or make network requests. They use mock data in place of your functions and resources.
Tests are displayed in the Tests tab of the debug console. Test code is stored as .test.ts or .test.tsx files in the frontend and backend folders of your app code. Tests use Vitest and React Testing Library.
Frontend tests
Frontend tests render your app's components and interact with them the way a user would. For example, a test might fill out a form, check field validation, submit the form, and confirm the result shown to the user. Calls to your functions return mock data, and the test checks the parameters your app sends.
Backend tests
Backend tests call a function directly and check its return value, the data it reads and writes, and the errors it returns for invalid parameters or unauthorized users. For example, a test might check that creating an order rejects a missing field, writes the expected rows, and returns the new order's ID.
The agent writes backend tests only for functions with their own logic, such as validation, permission checks, or data transforms. Frontend tests cover functions that pass a query straight to a resource.
Credit usage
Tests are code like the rest of your app, so the agent uses AI credits to write and update them. When the agent runs tests as part of building, it also uses credits to read the results and decide what to do next, just like any other step it takes. Because the agent fixes failing tests, a thread with Build with tests turned on can use more credits than the same thread without it.
Running tests from the Tests tab doesn't use credits, so you can run them as often as you like.
Enable app testing
App testing is controlled by the Build with tests setting, which controls whether the agent writes and runs tests in the current thread.
You can toggle the Build with tests setting in two places:
- In the Chat tab, click and toggle Build with tests.
- In the debug console, switch to the Tests tab and toggle Build with tests.
For apps without tests, such as new apps, Build with tests is turned off by default in new threads. When it's turned off, the agent doesn't write, update, or run tests, but if you have existing tests, they are preserved in your app. You can still run them manually from the Tests tab.
For apps with tests, Build with tests is turned on by default in new threads.
Ask the agent to write tests
When Build with tests is enabled, the agent writes tests as part of its normal work. You can also ask for tests that cover specific parts of your app, or for other kinds of tests.
Review test results
Each time the agent runs tests, the result appears in the chat. Expand a result to view the tests that ran. When tests fail, only the failures are listed.
| Result | Description |
|---|---|
| App verified | All tests passed. The result includes the number of behaviors tested. |
| Couldn't verify app behavior | One or more tests failed, or the tests couldn't run. Expand the result to view the cause. |
| No behaviors modified | The agent's changes in this turn aren't covered by any tests. |
| No matching behaviors tested | The agent ran a specific set of tests, but none matched. |
Test failures also appear in the Errors tab of the debug console, and every test run appears in the Logs tab with the Test type.
Run tests from the debug console
The Tests tab of the debug console lists every test in your app. Each test or group shows a status of Passed, Failed, Mixed results, or Not run. If your app changes after a test runs, the result is marked out of date.
From the Tests tab, you can:
- Click Run all tests to run every test in your app.
- Filter the list to show All tests or only Test Failures.
- Select a test or group to view its output, then click Run test or Run tests to run only those tests.
- Click Open test file to view the code for the selected test.
- Click + Add to chat on a failed test to add its details to your prompt, or click Fix with AI to ask the agent to fix the app so that the test passes.
Limitations
App testing has the following limitations:
- Tests run only while you are building in the app builder. They don't run when you publish an app, and the Retool CLI doesn't run them.
- Each test run is limited to two minutes. If your tests regularly reach this limit, let your Retool account manager know.