---
title: 'Flaky Tests: Find, Fix and Prevent: Free Online Course'
description: 'Fix flaky tests for good: measure them, then remove the four root causes
  (waits, shared data, test order, real app races) in Playwright and Selenium, use
  retries and quarantine safely, and debug flaky runs on a cloud grid.'
source_url:
  html: https://testingbot.com/resources/courses/flaky-tests
  md: https://testingbot.com/resources/courses/flaky-tests.md
---

Course

[Flaky Tests: Find, Fix and Prevent](https://testingbot.com/resources/courses/flaky-tests)

Chapter 1 of 10

1. [1 Flaky Tests: How to Find, Fix and Prevent Them](https://testingbot.com/resources/courses/flaky-tests)
2. [2 Measure Flakiness Before You Fix It](https://testingbot.com/resources/courses/flaky-tests/measuring-flakiness)
3. [3 Root Cause 1: Timing, Sleeps and Waits](https://testingbot.com/resources/courses/flaky-tests/timing-and-waits)
4. [4 Root Cause 2: Test Data and Shared State](https://testingbot.com/resources/courses/flaky-tests/test-data-and-shared-state)
5. [5 Root Cause 3: Test Order and Isolation](https://testingbot.com/resources/courses/flaky-tests/test-order-and-isolation)
6. [6 Root Cause 4: When the App Is the Flaky Part](https://testingbot.com/resources/courses/flaky-tests/when-the-app-is-flaky)
7. [7 Retries and Quarantine, Done Right](https://testingbot.com/resources/courses/flaky-tests/retries-and-quarantine)
8. [8 Debugging Flaky Tests on a Cloud Grid](https://testingbot.com/resources/courses/flaky-tests/debugging-flaky-tests-on-a-cloud-grid)
9. [9 A Team Process for Flaky Tests](https://testingbot.com/resources/courses/flaky-tests/a-team-process-for-flaky-tests)
10. [10 Flaky Tests Knowledge Check](https://testingbot.com/resources/courses/flaky-tests/knowledge-check)

[Exit course](https://testingbot.com/resources/courses)

 Chapter 1 of 10 Intermediate 6 min read Updated Nov 2026 

# Flaky Tests: How to Find, Fix and Prevent Them

Fix flaky tests for good: measure them, then remove the four root causes (waits, shared data, test order, real app races) in Playwright and Selenium, use retries and quarantine safely, and debug flaky runs on a cloud grid.

## What you will learn

1. Flaky Tests: How to Find, Fix and Prevent Them
2. [Measure Flakiness Before You Fix It](https://testingbot.com/resources/courses/flaky-tests/measuring-flakiness)
3. [Root Cause 1: Timing, Sleeps and Waits](https://testingbot.com/resources/courses/flaky-tests/timing-and-waits)
4. [Root Cause 2: Test Data and Shared State](https://testingbot.com/resources/courses/flaky-tests/test-data-and-shared-state)
5. [Root Cause 3: Test Order and Isolation](https://testingbot.com/resources/courses/flaky-tests/test-order-and-isolation)
6. [Root Cause 4: When the App Is the Flaky Part](https://testingbot.com/resources/courses/flaky-tests/when-the-app-is-flaky)
7. [Retries and Quarantine, Done Right](https://testingbot.com/resources/courses/flaky-tests/retries-and-quarantine)
8. [Debugging Flaky Tests on a Cloud Grid](https://testingbot.com/resources/courses/flaky-tests/debugging-flaky-tests-on-a-cloud-grid)
9. [A Team Process for Flaky Tests](https://testingbot.com/resources/courses/flaky-tests/a-team-process-for-flaky-tests)
10. [Flaky Tests Knowledge Check](https://testingbot.com/resources/courses/flaky-tests/knowledge-check)

[Continue where you left off](https://testingbot.com#)

## TestingBot Flaky Test Fixing Certification

Turn this course into a free, resume-worthy credential: pass the exam, run the course suite on TestingBot, and get a verifiable certificate for your LinkedIn profile.

[How it works](https://testingbot.com/resources/courses/flaky-tests/certification)On this page
1. [What a flaky test costs](https://testingbot.com#what-a-flaky-test-costs)
2. [Where flakiness comes from](https://testingbot.com#where-flakiness-comes-from)
3. [Measure first, then fix](https://testingbot.com#measure-first-then-fix)
4. [How the course is organised](https://testingbot.com#how-the-course-is-organised)
5. [Which framework](https://testingbot.com#which-framework)
6. [Prerequisites](https://testingbot.com#prerequisites)

_Last reviewed November 2026 against @playwright/test 1.65.0, Selenium 4.52.0 and pytest 9.2.0._

A flaky test is a test that passes and fails on the same code. Nothing in the commit changed, nothing in the test changed, and yet one CI run is red and the re-run is green. That last part is what makes flakiness expensive: the cheapest response to a red build is to press re-run, and every time that works, the team learns that red does not mean broken.

This course treats flakiness as a bug with a cause you can find, not as weather. Every example is a tiny test that really is flaky, measured over many runs before and after the fix, in Playwright (TypeScript) and in Selenium (Python) wherever the two frameworks behave differently.

## What a flaky test costs[#](https://testingbot.com#what-a-flaky-test-costs)

The obvious cost is CI time: a pipeline that needs a re-run takes twice as long, and a suite with retries enabled spends minutes re-running tests that should have passed the first time. The larger cost is trust.

- **Real failures get ignored.** When a test is known to flake, its failures stop being read. The day it fails for a real reason, someone presses re-run, it passes by luck, and the bug ships.
- **People stop adding tests.** If the suite already feels unreliable, writing one more end-to-end test feels like adding noise.
- **Debugging gets harder over time.** A flaky test that is retried silently for months collects more causes, and by the time someone looks, nobody remembers when it started.

Large engineering organisations have measured this. In a 2016 post on the Google Testing Blog, [Flaky Tests at Google and How We Mitigate Them](https://testing.googleblog.com/2016/05/flaky-tests-at-google-and-how-we.html), Google reported that about 1.5% of all its test runs had a flaky result and that almost 16% of its tests had some level of flakiness. Your numbers will differ, but the shape is common: a small rate per run, spread over a large share of the suite, adds up to many red builds.

## Where flakiness comes from[#](https://testingbot.com#where-flakiness-comes-from)

A failing run has a cause, even when the next run passes. In practice the causes fall into a few groups, and this course takes them one at a time:

| Root cause | What it looks like | Chapter |
| --- | --- | --- |
| Timing and waits | The test checks the page before the page is ready | 3 |
| Test data and shared state | Two tests running at the same time change the same record | 4 |
| Test order and isolation | A test only passes when another test ran before it, or did not | 5 |
| The app itself | The product has a race condition; the test is right to fail | 6 |

An often-cited study of flaky tests in open source projects, [An Empirical Analysis of Flaky Tests](https://dl.acm.org/doi/10.1145/2635868.2635920) (Luo, Hariri, Eloussi and Marinov, FSE 2014), found that asynchronous waits were the most common root cause in its sample of fixed flaky tests, followed by concurrency and test order dependency. The order in this course follows the same idea: check the cheap, common cause first.

The infrastructure can also be the cause: a browser that crashes, a network that drops a request, a CI runner that is overloaded. Those failures are real, but they tend to look different (the session never started, the browser disconnected) and they hit random tests rather than the same one. Chapter 8 shows how to tell them apart on a cloud grid.

## Measure first, then fix[#](https://testingbot.com#measure-first-then-fix)

The most common mistake with a flaky test is fixing it by guessing: add a sleep, bump a timeout, run it once, see green, merge. One green run proves almost nothing about a test that fails one time in ten. Chapter 2 sets up the measurements this course uses throughout:

- running one test many times in a row (`--repeat-each` in Playwright, `--count` with pytest-repeat),
- running the suite in a random order (`--shuffle` in Playwright, pytest-randomly),
- reading the "flaky" result that retries produce, and the history of a test across CI runs.

Every fix in this course comes with its numbers. For example, the timing test in chapter 3 passed 25 of 50 runs before the fix and 50 of 50 after.

## How the course is organised[#](https://testingbot.com#how-the-course-is-organised)

1. What a flaky test is and what it costs (this page)
2. Measuring flakiness before you change anything
3. Root cause 1: timing, sleeps and waits
4. Root cause 2: test data and shared state
5. Root cause 3: test order and isolation
6. Root cause 4: when the app is the flaky part
7. Retries and quarantine, and when retries hide real bugs
8. Debugging flaky tests on a cloud grid
9. A team process: ownership, a flaky budget and CI policy
10. A knowledge check

The examples run against a small demo app that you start on your own machine. It has four pages, and each one has one kind of flakiness built in on purpose: a save button with a slow response, a shared user list, a theme setting stored on the server, and a search box with a real race condition. Because you control the app, you can reproduce every failure and every fix yourself.

## Which framework[#](https://testingbot.com#which-framework)

The root causes are the same in every framework; the details differ. Playwright's actions and assertions wait automatically and every test gets a fresh browser context, so some classic Selenium flakiness never happens there, and some new kinds do. Where the code differs, the chapters show both. If you are new to either framework, start with the basics first:

- [Playwright Tutorial: Zero to CI](https://testingbot.com/resources/courses/playwright) for TypeScript and Playwright Test,
- [Selenium with Python and pytest](https://testingbot.com/resources/courses/selenium-python) for WebDriver with pytest,
- [Selenium with Java and TestNG](https://testingbot.com/resources/courses/selenium-testng) if your suite is in Java. The waits, data and order fixes in this course carry over directly; TestNG's own retry hook (`IRetryAnalyzer`) is covered in that course.

## Prerequisites[#](https://testingbot.com#prerequisites)

- Node.js 22 or newer for the demo app and the Playwright tests.
- Python 3.10 or newer and Chrome for the Selenium tests (Selenium Manager downloads the driver).
- Basic experience with one of the two frameworks.
- For chapter 8, a TestingBot account; the free trial is enough.

The course is free and needs no account. At the end there is a self-check, and an optional [TestingBot certification](https://testingbot.com/resources/courses/flaky-tests/certification) that asks for a passed exam and the course's fixed suite running green on TestingBot.

[Next · Chapter 2 Measure Flakiness Before You Fix It](https://testingbot.com/resources/courses/flaky-tests/measuring-flakiness)

On this page

1. [What a flaky test costs](https://testingbot.com#what-a-flaky-test-costs)
2. [Where flakiness comes from](https://testingbot.com#where-flakiness-comes-from)
3. [Measure first, then fix](https://testingbot.com#measure-first-then-fix)
4. [How the course is organised](https://testingbot.com#how-the-course-is-organised)
5. [Which framework](https://testingbot.com#which-framework)
6. [Prerequisites](https://testingbot.com#prerequisites)

Certification

Earn the free, resume-worthy TestingBot Flaky Test Fixing Certification for this course.

[How it works](https://testingbot.com/resources/courses/flaky-tests/certification)

## Other free courses

- [Mobile App Testing: From First Test to CI A free, hands-on mobile app testing course: emulators vs real devices, Appium vs Espresso vs XCUITest vs Ma...](https://testingbot.com/resources/courses/mobile-app-testing)
- [Accessibility Testing Tutorial: WCAG 2.2 A free, hands-on accessibility testing course: WCAG 2.2 in practice, manual keyboard and screen reader chec...](https://testingbot.com/resources/courses/accessibility-testing)
- [Manual Testing Course A hands-on manual testing course: test design, test cases, exploratory charters, bug reports developers act...](https://testingbot.com/resources/courses/manual-testing)
- [Selenium Locators and Locator Strategies A free course on Selenium locators and locator strategies: CSS selectors, XPath, role-based and test-id loc...](https://testingbot.com/resources/courses/locator-strategies)

[Browse all courses](https://testingbot.com/resources/courses)
