---
title: Writing prompts for codeless AI tests & troubleshooting
description: How the TestingBot AI agent reads a codeless test prompt, how to write
  prompts that pass reliably, and how to fix tests that fail with auth_required, blocked_by_overlay,
  captchas or step limits.
source_url:
  html: https://testingbot.com/support/ai/codeless/prompts
  md: https://testingbot.com/support/ai/codeless/prompts.md
---

# Writing prompts for codeless AI tests

A [codeless AI test](https://testingbot.com/support/ai/codeless) is a plain-language prompt that an AI agent carries out in a real browser in the TestingBot cloud, on the schedule you choose. This page explains how the agent interprets that prompt, how to write prompts that give stable results, and what to do when a test fails.

## How the agent reads a prompt

The agent is deliberately strict, so a test fails when your site breaks instead of quietly working around the problem:

- **It does exactly what the prompt says.** It does not add steps, try alternative paths or navigate elsewhere to “help”. If the prompt cannot be carried out as written, the test fails.
- **It fails fast.** It stops at the first blocking problem and does not retry unless the prompt tells it to. The failure reason is phrased after the check in your prompt, so write checks you would want to read in an alert.
- **It starts at the test's URL.** If the prompt contains no URL, the agent opens the URL you entered for the test.
- **It stays on that website.** The agent does not leave the domain it started on unless the prompt explicitly allows it. Embedded frames from other domains, such as a payment or video iframe, are fine.
- **It uses dummy data.** When it has to submit a form, it fills in only the required fields, with safe test values such as the name _Test User_ and an `@example.com` email address, unless your prompt gives the values to use.
- **It does not log in on its own.** If a page needs a login and the prompt does not contain credentials, the test fails with `auth_required`.
- **It deals with pop-ups once.** At the start it looks for cookie banners, consent dialogs, newsletter pop-ups and chat widgets that cover the page, and closes them with the smallest action, preferring “Reject”, “Essential only” or “Close” over “Accept all”.
- **It only uses the browser.** No search engines, external tools or APIs.

The agent stops and fails the test when:

- an element the prompt needs is missing, hidden or disabled after waiting for it;
- a captcha appears;
- the page needs a login and the prompt has no credentials;
- the flow leaves the website and the prompt did not allow it;
- a pop-up still covers what it needs to click after it tried to close it (`blocked_by_overlay`).

When a run passes, the steps the agent took are saved as the test's steps, so you can see exactly what it did. When a run fails, the reason is added to the test's log and your [alerts](https://testingbot.com/support/ai/codeless#alerting) are sent.

## Writing a good prompt

- **Test one thing.** One flow and one or two checks per test. Create separate tests for separate flows; they are easier to read in an alert and each runs in a limited number of steps.
- **Name what is on the screen.** Use the visible text of buttons, links and headings, in quotes: _Click “Add to cart”_, not _click the second button_.
- **Say what success looks like.** End with a check the agent can see: _Verify the page shows “Thank you for your order”_. Without one, the agent can only report that it got through the steps.
- **Write steps in order.** A short numbered list works well for flows with several steps.
- **Give the data it needs.** Search terms, product names, coupon codes and test account credentials go in the prompt. Use a dedicated test account, never a real one: the prompt is stored with the test.
- **Allow other domains explicitly.** If the flow continues on another site, for example a hosted login or checkout page, say so: _The login page on accounts.example.net is part of this test._
- **Avoid what a real browser session cannot check.** The agent cannot read your email or SMS, so it cannot complete sign-up confirmation or one-time-code steps.

## Before and after

| Vague prompt | Better prompt |
| --- | --- |
| Test the search. | Type “running shoes” in the search box and press Enter. Verify at least one product result is shown. |
| Check that checkout works. | 1. Open the first product on the page. 2. Click “Add to cart”. 3. Open the cart. Verify the cart shows 1 item. |
| Log in and check the dashboard. | Click “Sign in”, log in with email qa+monitor@example.com and password _(test account password)_. Verify the page shows the heading “Dashboard”. |
| Make sure the contact form is OK. | Submit the contact form without filling it in and verify an error appears for the email field. |

## Generating a test from a description

When you create an AI test, TestingBot can also turn your description into test steps straight away. To find real elements, it downloads the test's URL and up to four more URLs written in the description (five pages in total) and reads their HTML.

- Only **public** pages can be read. Pages behind a login, on a private network or on `localhost` are skipped, so steps for them are guesses. Run the test instead, or add the steps manually.
- If the description cannot be turned into steps, you see _“The AI could not generate test steps for this description”_. Rephrase it as in [Writing a good prompt](https://testingbot.com#write), or add the steps manually.
- AI tests run on Chrome and Edge on Windows and Linux. Other browsers selected for the test are removed when the steps are generated.

## Troubleshooting

| Failure or symptom | Likely cause | What to do |
| --- | --- | --- |
| `auth_required` | A page in the flow needs a login and the prompt has no credentials. | Add test-account credentials to the prompt, or start the test on a public page. |
| `blocked_by_overlay` | A cookie banner, consent dialog, newsletter or chat pop-up still covers the element after the agent tried to close it. | Tell the agent how to close it, e.g. _Click “Accept all” on the cookie banner first_, or suppress the pop-up for your test traffic. |
| Fails on a captcha | The agent never solves captchas. | Disable the captcha for test traffic on a test environment, or test a page without one. |
| “The AI agent stopped before reaching a result” | The agent ran out of steps or gave up, usually because the prompt is long or ambiguous. | Make the prompt more specific, or split it into several tests. |
| Element not found, but it is on the page | The prompt names it differently from the page, or it only appears after an action the prompt does not mention (scrolling, opening a menu). | Use the exact visible text, and add the step that reveals it. |
| Fails when the flow goes to another site | The agent stays on the starting domain unless allowed. | Say in the prompt that the other domain is part of the test. |
| Passes, but did not check what you meant | The prompt has no explicit check, or the check is vague. | End with a concrete “Verify …” on visible text or an element. |
| A scheduled run starts late | All your parallel slots were in use, so the run was queued and retried a few minutes later. | Spread schedules out, or add parallel capacity. |
| Scheduled runs stop | The test is disabled, or the account has no testing time left. | Check the test is enabled and your account's remaining minutes. |
| “Daily limit of 50 AI test generations” | Generating steps from a description is limited per account per day. | Try again the next day. Running existing tests is not affected. |

To see what the agent did, open the test run in your dashboard: the steps, the failure reason and, when video recording is on for the test, the video are all there.

## Limits

- Generating steps from a description: 50 per account per day, reading at most five public pages per generation.
- Each run is limited to a fixed number of agent steps; keep prompts focused.
- Runs count against your parallel slots and testing minutes, like other tests.
- Browsers: the latest Chrome and Edge, on Windows and Linux.

## What is sent to the AI

Codeless AI tests use third-party AI providers. Generating steps sends your description and the HTML of the pages it reads; running a test lets the AI agent read the pages it visits. This content is not masked, so keep real passwords and personal data out of prompts and use test accounts. See the [Trust Center](https://testingbot.com/trust) for the providers and what each receives.

## Looking for more help?

Have questions or need more information? Reach out via email or Slack.

[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
