---
title: 'Thresholds: How Much Difference to Allow | TestingBot'
description: Once screenshots are stable, the remaining question is how strict the
  comparison should be. Playwright has three knobs for it, and they are easy to misread:…
source_url:
  html: https://testingbot.com/resources/courses/visual-regression-testing/thresholds-and-diff-tolerance
  md: https://testingbot.com/resources/courses/visual-regression-testing/thresholds-and-diff-tolerance.md
---

Course

[Visual Regression Testing Tutorial](https://testingbot.com/resources/courses/visual-regression-testing)

Chapter 4 of 10

1. [1 Visual Regression Testing Tutorial](https://testingbot.com/resources/courses/visual-regression-testing)
2. [2 Your First Screenshot Comparison with Playwright](https://testingbot.com/resources/courses/visual-regression-testing/first-screenshot-comparison)
3. [3 Making Visual Tests Stable](https://testingbot.com/resources/courses/visual-regression-testing/stable-screenshots)
4. [4 Thresholds: How Much Difference to Allow](https://testingbot.com/resources/courses/visual-regression-testing/thresholds-and-diff-tolerance)
5. [5 Full Page, Element and Component Snapshots](https://testingbot.com/resources/courses/visual-regression-testing/page-element-and-component-snapshots)
6. [6 Cross-Browser Visual Testing](https://testingbot.com/resources/courses/visual-regression-testing/cross-browser-visual-testing)
7. [7 Visual Testing on a Cloud Grid with TestingBot](https://testingbot.com/resources/courses/visual-regression-testing/visual-testing-on-a-cloud-grid)
8. [8 Visual Regression Testing in CI with GitHub Actions](https://testingbot.com/resources/courses/visual-regression-testing/visual-tests-in-github-actions)
9. [9 A Visual Testing Strategy That Lasts](https://testingbot.com/resources/courses/visual-regression-testing/visual-testing-strategy)
10. [10 Visual Regression Testing Quiz](https://testingbot.com/resources/courses/visual-regression-testing/knowledge-check)

[Exit course](https://testingbot.com/resources/courses)

 Chapter 4 of 10 Intermediate 8 min read 2 quick checks Updated Nov 2026 

# Thresholds: How Much Difference to Allow
On this page
1. [How Playwright compares two images](https://testingbot.com#how-playwright-compares-two-images)
2. [An experiment with a button](https://testingbot.com#an-experiment-with-a-button)
3. [Why a big allowance is dangerous](https://testingbot.com#why-a-big-allowance-is-dangerous)
4. [Setting defaults for the whole suite](https://testingbot.com#setting-defaults-for-the-whole-suite)
5. [Pixel comparison and "perceptual" comparison](https://testingbot.com#pixel-comparison-and-perceptual-comparison)
6. [Where the threshold is not the answer](https://testingbot.com#where-the-threshold-is-not-the-answer)

Once screenshots are stable, the remaining question is how strict the comparison should be. Playwright has three knobs for it, and they are easy to misread: one decides whether a single pixel counts as different, the other two decide how many different pixels a test may have. This chapter measures what each one does on a real element, so you can choose values instead of guessing.

## How Playwright compares two images[#](https://testingbot.com#how-playwright-compares-two-images)

The default comparator walks both images pixel by pixel. For each pair it computes a colour difference in the YIQ colour space, which weights brightness more than hue, roughly the way human eyes do. If the difference is above `threshold`, the pixel counts as different. Then it counts the different pixels and compares the count with `maxDiffPixels` or `maxDiffPixelRatio`.

| Option | Question it answers | Default |
| --- | --- | --- |
| `threshold` | Is this pixel different? (0 strict to 1 lax) | `0.2` |
| `maxDiffPixels` | How many different pixels may the image have? | unset, so 0 |
| `maxDiffPixelRatio` | What share of all pixels may differ? (0 to 1) | unset, so 0 |

So with the defaults, a test fails as soon as one pixel differs by more than 0.2.

## An experiment with a button[#](https://testingbot.com#an-experiment-with-a-button)

Create `tests/threshold.spec.ts`. It renders a button in one of four variants, chosen by an environment variable, and takes the comparison options from another:

```typescript
import { test, expect } from '@playwright/test';

const variant = process.env.VARIANT ?? 'base';
const styles: Record<string, string> = {
  base: 'background:#1a73e8; padding:12px 24px',
  nudge: 'background:#1b74e9; padding:12px 24px', // +1 on every colour channel
  rebrand: 'background:#1e88e5; padding:12px 24px', // a visibly different blue
  shift: 'background:#1a73e8; padding:12px 23px 12px 25px', // text 1px to the right
};

test('buy button', async ({ page }) => {
  await page.setContent(`
    <button style="${styles[variant]}; color:white; border:0;
                   font:600 16px system-ui; border-radius:6px">Buy now</button>`);
  const options = JSON.parse(process.env.OPTS ?? '{}');
  await expect(page.getByRole('button')).toHaveScreenshot('buy-button.png', options);
});
```

Record the baseline with the base variant, then run each variant against it:

```bash
npx playwright test tests/threshold.spec.ts --update-snapshots=missing
VARIANT=rebrand npx playwright test tests/threshold.spec.ts
VARIANT=rebrand OPTS='{"threshold":0.05}' npx playwright test tests/threshold.spec.ts
```

These are the results we got (Chromium on macOS; your counts will be close but not identical):

| Variant | Options | Result |
| --- | --- | --- |
| nudge | defaults | passed |
| nudge | `threshold: 0` | failed, 4,218 pixels (0.89 of the image) |
| rebrand | defaults | **passed** |
| rebrand | `threshold: 0.05` | failed, 4,200 pixels |
| shift | defaults | **failed** , 101 pixels (0.03) |
| shift | `maxDiffPixels: 150` | passed |
| shift | `maxDiffPixelRatio: 0.05` | passed |
| rebrand | `threshold: 0.05, maxDiffPixels: 150` | failed, 4,200 pixels |

Three lessons are in that table.

**The default `threshold` lets real colour changes through.** Changing the brand blue from `#1a73e8` to `#1e88e5` is a change a designer would notice and might want reviewed, and with the defaults it passes. Every pixel of the button changed, but each by less than 0.2 in YIQ terms. If colours matter to you (brand colours, status colours, contrast fixes), lower `threshold` to around 0.05 to 0.1.

**`threshold: 0` is too strict for real use.** A change of one unit on each colour channel, invisible to anyone, fails 89% of the button. Rendering noise between runs, GPU paths and browser updates produce exactly this kind of tiny change.

**Pixel allowances and colour tolerance cover different failures.** A one-pixel text shift changes few pixels, but changes them a lot (white text becomes blue background), so `threshold` does not help and `maxDiffPixels` does. A colour change touches many pixels by a little, so `maxDiffPixels` does not hide it and `threshold` does. Tune the knob that matches the noise you are actually seeing.

A designer changes a status badge from one green to a slightly different green. The visual test still passes. Which change makes the suite catch this kind of change?

 Set `maxDiffPixels: 0` Switch to `fullPage: true` Raise `threshold` to 0.3 Lower `threshold`, for example to 0.05

`threshold` decides whether each pixel counts as different. A subtle colour change moves every pixel a little, under the default 0.2, so no pixel counts and the pixel allowance never comes into play. `maxDiffPixels` is already 0 by default.

## Why a big allowance is dangerous[#](https://testingbot.com#why-a-big-allowance-is-dangerous)

`maxDiffPixels: 150` made the one-pixel shift pass. It would also make many real bugs pass: a missing icon, a changed digit in a price, a one-word label change. On a full-page screenshot of 1280x720 (921,600 pixels), `maxDiffPixelRatio: 0.01` allows 9,216 different pixels, enough for a whole paragraph to vanish.

That is why stabilisation comes first (chapter 2) and tolerance second. A useful process:

1. Make the test stable with the defaults if you can. Many tests need no tolerance at all, because the same browser in the same environment produces identical pixels.
2. If a test still fails intermittently, look at the diff image. Speckles along text edges point to anti-aliasing noise; a block of red points to content you did not stabilise.
3. For edge noise, add the smallest `maxDiffPixels` that covers it, on that one test, and keep it well below the size of anything you care about.
4. Write down why. A comment next to the option saves the next person from doubling it.

Prefer `maxDiffPixels` for element and component screenshots, where the image size is small and known, and `maxDiffPixelRatio` only where image size varies, such as full-page screenshots of pages with dynamic length.

## Setting defaults for the whole suite[#](https://testingbot.com#setting-defaults-for-the-whole-suite)

Per-test options override suite defaults, which live in the config:

```typescript
export default defineConfig({
  expect: {
    toHaveScreenshot: {
      threshold: 0.1,
      maxDiffPixels: 0,
      animations: 'disabled',
      caret: 'hide',
    },
  },
});
```

Keep the global values strict and loosen individual tests, not the other way round. A loose global default silently weakens every new test somebody adds later.

A full-page visual test fails one run in five with about 40 red pixels scattered along text edges. The rest of the suite is stable. What is the most proportionate fix?

 Add a small `maxDiffPixels`, for example 60, to that one test, with a comment saying why Set `maxDiffPixelRatio: 0.05` globally in the config Set `threshold: 1` on that test Delete the baseline and let it regenerate on every run

Scattered edge pixels are anti-aliasing noise, and a small pixel allowance on the affected test absorbs it while still catching real changes. A 5% global ratio would let a whole section of any page change unnoticed, and `threshold: 1` treats every pixel as equal, which turns the test off.

## Pixel comparison and "perceptual" comparison[#](https://testingbot.com#pixel-comparison-and-perceptual-comparison)

Playwright's comparator is a pixel comparison with a perceptual colour metric: YIQ distances, plus detection of anti-aliased pixels along edges. Other tools describe their algorithms differently, but most fall into the same few ideas:

- **Exact pixel equality.** Fast and simple, and fails on any rendering noise. Usable only in a perfectly controlled environment.
- **Per-pixel colour tolerance** (Playwright's `threshold`, and the `threshold` option of TestingBot's visual comparison in chapter 6). Ignores small colour shifts.
- **Anti-aliasing detection.** Ignores pixels that differ only because an edge was smoothed differently. TestingBot's comparison does this by default (its `antialiasing` option), and its scheduled visual tests additionally only count a pixel when a neighbouring pixel changed too, so isolated stray pixels do not fail a run.
- **Structural or AI-based comparison.** Some commercial tools compare layout and content structure rather than raw pixels, to ignore noise and group related changes. These trade transparency for fewer false failures; you can no longer reason about exactly which pixel failed.

Whatever the tool, the decision is the same one you made above: which differences are noise in your environment, and which ones must a human see.

## Where the threshold is not the answer[#](https://testingbot.com#where-the-threshold-is-not-the-answer)

If you find yourself raising tolerances again and again for one test, the test is probably wrong, not the threshold:

- The screenshot covers too much. A full page with a news feed will never be stable; screenshot the component you care about (chapter 4).
- Something dynamic is not stabilised. Go back to the chapter 2 checklist.
- The baseline came from a different environment. A macOS baseline compared on Linux is thousands of pixels off, and no sane tolerance fixes that.

The next chapter is about choosing what to screenshot: the whole page, one element, or one component at several screen sizes.

[Previous · Chapter 3 Making Visual Tests Stable](https://testingbot.com/resources/courses/visual-regression-testing/stable-screenshots) [Next · Chapter 5 Full Page, Element and Component Snapshots](https://testingbot.com/resources/courses/visual-regression-testing/page-element-and-component-snapshots)

On this page

1. [How Playwright compares two images](https://testingbot.com#how-playwright-compares-two-images)
2. [An experiment with a button](https://testingbot.com#an-experiment-with-a-button)
3. [Why a big allowance is dangerous](https://testingbot.com#why-a-big-allowance-is-dangerous)
4. [Setting defaults for the whole suite](https://testingbot.com#setting-defaults-for-the-whole-suite)
5. [Pixel comparison and "perceptual" comparison](https://testingbot.com#pixel-comparison-and-perceptual-comparison)
6. [Where the threshold is not the answer](https://testingbot.com#where-the-threshold-is-not-the-answer)

Certification

This course has a free, hands-on certificate.

[How it works](https://testingbot.com/resources/courses/visual-regression-testing/certification)