---
title: Flaky Tests Knowledge Check | TestingBot
description: 'Ten questions across the course. Pick an answer for each and check them
  at the end; every answer comes with an explanation. This is a self-check: the answers…'
source_url:
  html: https://testingbot.com/resources/courses/flaky-tests/knowledge-check
  md: https://testingbot.com/resources/courses/flaky-tests/knowledge-check.md
---

Course

[Flaky Tests: Find, Fix and Prevent](https://testingbot.com/resources/courses/flaky-tests)

Chapter 10 of 10

1. [1 Flaky Tests: How to Find, Fix and Prevent Them](https://testingbot.com/resources/courses/flaky-tests)
2. [2 Measure Flakiness Before You Fix It](https://testingbot.com/resources/courses/flaky-tests/measuring-flakiness)
3. [3 Root Cause 1: Timing, Sleeps and Waits](https://testingbot.com/resources/courses/flaky-tests/timing-and-waits)
4. [4 Root Cause 2: Test Data and Shared State](https://testingbot.com/resources/courses/flaky-tests/test-data-and-shared-state)
5. [5 Root Cause 3: Test Order and Isolation](https://testingbot.com/resources/courses/flaky-tests/test-order-and-isolation)
6. [6 Root Cause 4: When the App Is the Flaky Part](https://testingbot.com/resources/courses/flaky-tests/when-the-app-is-flaky)
7. [7 Retries and Quarantine, Done Right](https://testingbot.com/resources/courses/flaky-tests/retries-and-quarantine)
8. [8 Debugging Flaky Tests on a Cloud Grid](https://testingbot.com/resources/courses/flaky-tests/debugging-flaky-tests-on-a-cloud-grid)
9. [9 A Team Process for Flaky Tests](https://testingbot.com/resources/courses/flaky-tests/a-team-process-for-flaky-tests)
10. [10 Flaky Tests Knowledge Check](https://testingbot.com/resources/courses/flaky-tests/knowledge-check)

[Exit course](https://testingbot.com/resources/courses)

 Chapter 10 of 10 Intermediate 6 min read Updated Nov 2026 

# Flaky Tests Knowledge Check

Ten questions across the course. Pick an answer for each and check them at the end; every answer comes with an explanation. This is a self-check: the answers are in the page source. The [certification exam](https://testingbot.com/resources/courses/flaky-tests/certification) uses a different set of questions.

1. What makes a test flaky, as opposed to failing?

 It takes longer than the timeout It both passes and fails on the same code, without any change to the test or the app It fails on one browser only It was written without page objects

A failing test fails every time on the same code. A flaky test's result changes between runs with no change in between; a test that always fails on one browser is failing there, not flaky.

2. Which command gives you the most useful first number for a suspect Playwright test?

 `npx playwright test --headed` `npx playwright test --retries 3` `npx playwright test path/to.spec.ts --repeat-each 50 --workers 4` `npx playwright test --update-snapshots`

Repeating the test many times, with the parallelism CI uses, gives a pass rate to compare against after the fix. Retries hide the failures you are trying to count.

3. Which Playwright line is the reliable way to check a confirmation that appears after an API call?

 `await expect(page.getByRole('status')).toHaveText('Saved')` `expect(await page.getByRole('status').textContent()).toBe('Saved')` `await page.waitForTimeout(1000)` followed by a `textContent()` check `await page.waitForLoadState('networkidle')`

The web-first assertion re-checks the element until the text matches. Reading `textContent()` once and asserting on the string cannot retry, and sleeps and network idle are guesses.

4. A test resets a shared table in `beforeEach` and asserts that it then contains exactly one row. It passes with one worker and fails often with four. What is the fix?

 Run the whole suite with one worker Reset the table twice Retry the test on failure Create a uniquely named row and assert only on that row

Other workers add and delete rows at the same time. A unique row and a scoped assertion make the test independent of them; a global reset destroys other tests' data and makes them flaky too.

5. Playwright gives every test a fresh browser context. Why can a Playwright suite still have order-dependent tests?

 Contexts share cookies between tests in the same file State on the server, such as a shared account's settings, is not reset by a new browser context Fresh contexts are only created when retries are on

A browser context isolates cookies and storage in the browser. Anything the app stores on the server for a shared account survives from one test to the next.

6. pytest-randomly reports a failure with seed 1234. You replay `--randomly-seed=1234` twice against the same running server, and the second replay fails a different test. Why?

 pytest-randomly ignores the seed on the second run The browser cache was warm The server kept state from the first replay, so the second started from a different state The seed only applies with pytest-xdist

The seed fixes the order, not the environment. To reproduce an order failure exactly, reset the environment (restart or reseed the server) before each replay.

7. Forcing an older search response to arrive last with `page.route` makes a 4% flaky test fail every time. What does that show?

 The app has a race condition, and you now have a deterministic regression test for it `page.route` breaks the page and should not be used in tests The test needs a longer timeout The failure is caused by the grid

If forcing the suspected order makes the failure certain, the product mishandles that order. Fix the app, and keep the forced-order test so the bug cannot return unnoticed.

8. With `retries: 2`, a test fails on its first attempt and passes on the second. How does Playwright Test report it?

 As passed, with no trace of the failure As failed As skipped As flaky; the run still exits successfully unless `--fail-on-flaky-tests` is set

Flaky is its own outcome in the report. By default it does not fail the run, which is why the flaky count needs to be read or enforced.

9. Comparing a passing and a failing grid run, the steps are identical until "Get Text From Element", which returned `"Saved"` in one and `""` in the other, after 0.5 s of idle time. What is the likely cause?

 A broken locator A fixed sleep that is sometimes shorter than the operation it waits for A grid infrastructure failure A different browser version in the failing run

The element was found in both runs, so the locator works. An empty value after a fixed pause means "not yet": the test read the status before the save finished.

10. Which quarantine practice keeps a flaky test from being forgotten?

 Deleting it and writing a new one later Marking it skipped until the next release Running it in a non-blocking job, linked to an issue with an owner and an expiry date Moving it to a separate repository

The test keeps running and producing history, someone is responsible for it, and the expiry forces a decision. Skipped tests produce no data and tend to stay skipped.

Check answers

[Previous · Chapter 9 A Team Process for Flaky Tests](https://testingbot.com/resources/courses/flaky-tests/a-team-process-for-flaky-tests)
## You finished Flaky Tests: Find, Fix and Prevent

Put what you learned into practice. Run your tests on 6100+ real browsers and physical iOS and Android devices.

[Start free trial](https://testingbot.com/users/sign_up) [See pricing](https://testingbot.com/pricing) [Browse other courses](https://testingbot.com/resources/courses)

## TestingBot Flaky Test Fixing Certification

Turn this course into a free, resume-worthy credential: pass the exam, run the course suite on TestingBot, and get a verifiable certificate for your LinkedIn profile.

[How it works](https://testingbot.com/resources/courses/flaky-tests/certification)
