Exit course
Chapter 10 of 10 Intermediate 6 min read Updated

Flaky Tests Knowledge Check

Ten questions across the course. Pick an answer for each and check them at the end; every answer comes with an explanation. This is a self-check: the answers are in the page source. The certification exam uses a different set of questions.

1. What makes a test flaky, as opposed to failing?

A failing test fails every time on the same code. A flaky test's result changes between runs with no change in between; a test that always fails on one browser is failing there, not flaky.

2. Which command gives you the most useful first number for a suspect Playwright test?

Repeating the test many times, with the parallelism CI uses, gives a pass rate to compare against after the fix. Retries hide the failures you are trying to count.

3. Which Playwright line is the reliable way to check a confirmation that appears after an API call?

The web-first assertion re-checks the element until the text matches. Reading textContent() once and asserting on the string cannot retry, and sleeps and network idle are guesses.

4. A test resets a shared table in beforeEach and asserts that it then contains exactly one row. It passes with one worker and fails often with four. What is the fix?

Other workers add and delete rows at the same time. A unique row and a scoped assertion make the test independent of them; a global reset destroys other tests' data and makes them flaky too.

5. Playwright gives every test a fresh browser context. Why can a Playwright suite still have order-dependent tests?

A browser context isolates cookies and storage in the browser. Anything the app stores on the server for a shared account survives from one test to the next.

6. pytest-randomly reports a failure with seed 1234. You replay --randomly-seed=1234 twice against the same running server, and the second replay fails a different test. Why?

The seed fixes the order, not the environment. To reproduce an order failure exactly, reset the environment (restart or reseed the server) before each replay.

7. Forcing an older search response to arrive last with page.route makes a 4% flaky test fail every time. What does that show?

If forcing the suspected order makes the failure certain, the product mishandles that order. Fix the app, and keep the forced-order test so the bug cannot return unnoticed.

8. With retries: 2, a test fails on its first attempt and passes on the second. How does Playwright Test report it?

Flaky is its own outcome in the report. By default it does not fail the run, which is why the flaky count needs to be read or enforced.

9. Comparing a passing and a failing grid run, the steps are identical until "Get Text From Element", which returned "Saved" in one and "" in the other, after 0.5 s of idle time. What is the likely cause?

The element was found in both runs, so the locator works. An empty value after a fixed pause means "not yet": the test read the status before the save finished.

10. Which quarantine practice keeps a flaky test from being forgotten?

The test keeps running and producing history, someone is responsible for it, and the expiry forces a decision. Skipped tests produce no data and tend to stay skipped.

You finished Flaky Tests: Find, Fix and Prevent

Put what you learned into practice. Run your tests on 6100+ real browsers and physical iOS and Android devices.