--- title: AI Test Automation Quiz | TestingBot description: Nine questions on the decisions that matter when AI agents write, fix and triage your tests. Each answer comes with a short explanation, so a wrong answer… source_url: html: https://testingbot.com/resources/courses/ai-test-automation/knowledge-check md: https://testingbot.com/resources/courses/ai-test-automation/knowledge-check.md --- Course [AI Test Automation: A Hands-On Course](https://testingbot.com/resources/courses/ai-test-automation) Chapter 9 of 9 1. [1 AI Test Automation: A Hands-On Course](https://testingbot.com/resources/courses/ai-test-automation) 2. [2 Set Up an Agent with Playwright MCP](https://testingbot.com/resources/courses/ai-test-automation/set-up-playwright-mcp) 3. [3 From Test Plan to Playwright Test with an Agent](https://testingbot.com/resources/courses/ai-test-automation/generate-tests-from-a-plan) 4. [4 Playwright Test Agents: Planner, Generator and Healer](https://testingbot.com/resources/courses/ai-test-automation/playwright-test-agents) 5. [5 Self-Healing Tests: When Healing Hides a Regression](https://testingbot.com/resources/courses/ai-test-automation/self-healing-tests) 6. [6 AI for Failure Triage: Traces, Logs and Privacy](https://testingbot.com/resources/courses/ai-test-automation/ai-failure-triage) 7. [7 Run AI-Written Suites on TestingBot](https://testingbot.com/resources/courses/ai-test-automation/run-on-testingbot) 8. [8 Guardrails for AI in a Test Suite](https://testingbot.com/resources/courses/ai-test-automation/guardrails) 9. [9 AI Test Automation Quiz](https://testingbot.com/resources/courses/ai-test-automation/knowledge-check) [Exit course](https://testingbot.com/resources/courses) Chapter 9 of 9 Intermediate 7 min read Updated Nov 2026 # AI Test Automation Quiz Nine questions on the decisions that matter when AI agents write, fix and triage your tests. Each answer comes with a short explanation, so a wrong answer points you back to the right chapter. 1. Which use of AI in test automation keeps the run that gates a release fully deterministic? A model that interprets natural-language test steps on every CI run An agent that writes Playwright test files which a person reviews and commits A runtime tool that replaces failing locators while the suite runs A model that decides at run time which tests are worth running When the AI works at authoring time, CI runs ordinary committed code: the same commit does the same thing every time. The other three options put a model's decision inside the run, where nobody reviews it. 2. Playwright MCP returns an accessibility snapshot after each action. Why do agents using it tend to produce good locators? The snapshot contains the page's full CSS, so the agent can pick stable classes The model was trained on your application The snapshot lists roles and accessible names, the same information `getByRole` and `getByLabel` use Playwright MCP rejects any locator that is not role based The agent sees "checkbox 'Toggle Todo'" rather than pixels or markup, so the natural way to refer to it is the role-based locator. The flip side: elements without accessible names are just as hard for the agent to tell apart as for a screen-reader user. 3. The Playwright planner wrote a plan for your app. Its summary admits it only looked at the start page. What does that tell you about the plan's "expect" lines? They may come from the model's general knowledge rather than observation, so each one needs checking against the requirements They are reliable, because the planner always validates expectations later They only matter for the healer, not for the generated tests The plan is unusable and must be deleted Every expect line becomes an assertion. Expectations the planner did not observe are guesses, which can be right on a well-known demo and wrong on your app. The plan is the cheapest place to fix them, and the planner's honest summary is a good reason to look. 4. A generated test asserts `toHaveCSS('text-decoration-line', 'line-through')` for a completed todo. The requirement only says completed items are "marked as completed". What is the right review decision? Keep it, because more assertions catch more bugs Replace it with a screenshot comparison Wrap it in a `try/catch` so it cannot fail the test Remove it or replace it with a check of the completed state, since the style is an implementation detail the requirement does not fix An assertion on styling breaks on a harmless redesign and trains the team to ignore failures. Assert the state the requirement describes (checked box, completed state, counter) and leave visual details to visual testing. 5. A healer agent's diff changes a button locator from "Add to cart" to "Add to bag" and leaves the test failing on a wrong total. How should you read that outcome? The healer failed; it should have updated the expected total too That is correct behaviour: it fixed a stale locator and left the assertion that caught a real regression The test should now be marked `fixme` until the next release The locator change is risky and should be reverted Fixing what finds elements and leaving what judges results is exactly the right split. A healer that also "fixed" the total would have turned a pricing regression into a passing test. 6. After an unattended healer run, CI reports "1 skipped, 11 passed" and exits with code 0. The skip is new. What should have prevented this from looking like a green build? `forbidOnly: true` in the config More retries on CI A pull request check that fails when a change adds `test.skip` or `test.fixme`, so a person must approve it Running the healer twice `forbidOnly` only catches `test.only`. Playwright has no built-in check for newly added skips, so add one. Retries and repeated healing do not address a test that was turned off. 7. You want a model to triage a failed test. Which input most improves the chance it says correctly whether the app or the test is wrong? The requirement for the feature, plus the application diff since the last green run The full `trace.zip` including network bodies A longer instruction to "think step by step" The test's history over the last year The error shows that the test and the page disagree. Only the requirement says which one is right, and the diff says what changed. The trace adds detail and a lot of sensitive data, and it cannot settle what the correct behaviour is. 8. Your reviewed suite runs on TestingBot with two projects and two workers. Eight tests pass, and the dashboard shows four sessions. Why? Four tests did not report a status, so their sessions were hidden The grid merged duplicate tests The build name groups sessions in pairs Each worker connects once per project and runs several tests in that browser, each in a fresh context Sessions follow Playwright's workers, not individual tests. A worker that hits a failure is replaced, so a failed test is always the last in its session and its status is not overwritten by a later pass. 9. A teammate wants to give an exploring agent the production admin login "because staging data is unrealistic". What is the strongest objection? Agents cannot log in to admin areas The browser MCP server is not a security boundary: whatever the agent can reach (and whatever page text it reads) can lead to real actions on production and to data leaving your network Production is slower than staging It would use more tokens An agent with production credentials can click "delete" as easily as "view", and page content can carry prompt injection. Limit what the agent can reach. Do not rely on the model to behave. Check answers ## Where to go next[#](https://testingbot.com#where-to-go-next) If you want a firmer grip on the code agents write, the [Playwright course](https://testingbot.com/resources/courses/playwright) covers locators, auto-waiting, fixtures and CI in depth, and the [Page Object Model course](https://testingbot.com/resources/courses/page-object-model) shows how to keep a growing suite readable, which also makes it easier for agents to extend. For the agent side, the official [Playwright Test Agents guide](https://playwright.dev/docs/test-agents) and the [Playwright MCP README](https://github.com/microsoft/playwright-mcp) are the primary references, and the [TestingBot MCP documentation](https://testingbot.com/support/ai/mcp) lists every tool the TestingBot server provides. [Previous · Chapter 8 Guardrails for AI in a Test Suite](https://testingbot.com/resources/courses/ai-test-automation/guardrails) ## You finished AI Test Automation: A Hands-On Course Put what you learned into practice. Run your tests on 6100+ real browsers and physical iOS and Android devices. [Start free trial](https://testingbot.com/users/sign_up) [See pricing](https://testingbot.com/pricing) [Browse other courses](https://testingbot.com/resources/courses) ## Earn the TestingBot certification for this course Pass the exam and run the course suite on TestingBot. Free, with a public link you can add to LinkedIn. [How it works](https://testingbot.com/resources/courses/ai-test-automation/certification)