Making Visual Tests Stable

A visual test that fails when nothing is wrong is worse than no test: people learn to click "update" without looking, and then the real regression gets approved too. This chapter goes through the causes of false failures in the order you will meet them, and shows the Playwright setting or habit that removes each one.

The page that changes on every load#

Save this as tests/fixtures/dashboard.html. It has the usual troublemakers: a timestamp, a random number, a spinning loader and a focused input.

<!doctype html>
<html>
<head>
<style>
  body { font: 16px/1.4 system-ui, sans-serif; margin: 24px; }
  .spinner { width: 40px; height: 40px; border: 4px solid #ddd; border-top-color: #1a73e8;
             border-radius: 50%; animation: spin 1s linear infinite; }
  @keyframes spin { to { transform: rotate(360deg); } }
</style>
</head>
<body>
  <h1>Dashboard</h1>
  <p id="updated"></p>
  <div class="spinner"></div>
  <p id="tip"></p>
  <input id="search" placeholder="Search">
  <script>
    document.getElementById('updated').textContent =
      'Last updated ' + new Date().toLocaleString();
    document.getElementById('tip').textContent =
      'Tip of the day #' + Math.floor(Math.random() * 1000);
    document.getElementById('search').focus();
  </script>
</body>
</html>

And a naive test in tests/dashboard.spec.ts:

import { test, expect } from '@playwright/test';
import fs from 'node:fs';
import path from 'node:path';

const html = fs.readFileSync(path.join(__dirname, 'fixtures/dashboard.html'), 'utf8');

test('dashboard, naive', async ({ page }) => {
  await page.setContent(html);
  await expect(page).toHaveScreenshot('dashboard-naive.png');
});

Run it twice. The first run writes the baseline; the second fails with about 130 different pixels, because the time and the tip number changed. Nothing is broken. That is the false failure to eliminate.

What Playwright already handles#

Some sources of noise you get for free with toHaveScreenshot():

• CSS animations and transitions. animations: 'disabled' is the default. Finite animations are fast-forwarded to their end state and infinite ones (like the spinner) are reset to their initial state, so the spinner is captured in the same position every time.
• The text caret. caret: 'hide' is the default, so the blinking cursor in the focused search box never appears.
• Web fonts. The assertion waits for fonts to load before it captures.
• A page still settling. It retakes screenshots until two in a row are identical.

To see why the defaults matter, turn animations back on:

await expect(page).toHaveScreenshot('dashboard-anim.png', { animations: 'allow' });

The test now times out with Failed to take two consecutive stable screenshots, because each screenshot catches the spinner at a different angle. Leave the defaults alone.

These defaults apply to toHaveScreenshot(). Plain page.screenshot() does not disable animations unless you ask it to, which is one reason not to build your own comparison on top of it.

Freeze the clock, the locale and the timezone#

Dates and times are the most common dynamic content. Rather than hide them, make them deterministic. Playwright's clock API fixes Date.now() and new Date() in the page, and the context options pin how dates are formatted:

test.describe('dashboard, stabilised', () => {
  test.use({ locale: 'en-US', timezoneId: 'UTC' });

  test('dashboard', async ({ page }) => {
    await page.clock.setFixedTime(new Date('2026-01-15T09:00:00Z'));
    await page.setContent(html);
    await expect(page).toHaveScreenshot('dashboard.png', {
      mask: [page.locator('#tip')],
    });
  });
});

Call page.clock.setFixedTime() (or page.clock.install() if timers must still run) before the page loads, so the page's scripts see the fixed time from the start. The page now always says Last updated 1/15/2026, 9:00:00 AM, whatever the machine's timezone or language.

Locale and timezone are worth pinning even on pages without dates. Number formatting (1,234.5 versus 1.234,5), default fonts for some languages and translated UI strings all follow the locale, and a CI runner in another region will happily use a different one.

Mask what you cannot control#

The tip is random, and in a real app you often cannot control things like ads, third-party widgets, user avatars or live prices. mask takes a list of locators and paints a solid box over each one before comparing:

mask: [page.locator('#tip')],

The box is magenta (#FF00FF) by default; change it with maskColor. Because the box covers the element's bounding box in both the baseline and the new screenshot, nothing inside it can fail the test. The element's size still counts, though: if the masked element grows and pushes content down, the content below it moves and the test fails. That is usually what you want.

With the clock frozen and the tip masked, the dashboard test passes on every run.

Hide things with a screenshot stylesheet#

When there are many volatile elements, or you want to hide rather than paint over them, stylePath applies a CSS file only while the screenshot is taken:

/* tests/screenshot.css: applied only while toHaveScreenshot captures */
#tip { visibility: hidden; }

await expect(page).toHaveScreenshot('dashboard-stylepath.png', {
  stylePath: path.join(__dirname, 'screenshot.css'),
});

Use visibility: hidden rather than display: none: it keeps the element's space, so the layout around it is still tested. A shared stylesheet is also a good place to hide things that are noise on every page, such as embedded videos, chat widgets or iframes from third parties. You can set it once for the whole suite in the config under expect.toHaveScreenshot.stylePath.

Wait for the right state, not for time#

toHaveScreenshot() waits for two identical frames, but a page that is stable while still showing a skeleton loader is stable in the wrong state. Before the screenshot, assert on what the finished page contains:

await expect(page.getByRole('row')).toHaveCount(25);
await expect(page.getByRole('img', { name: 'Product photo' })).toBeVisible();
await expect(page).toHaveScreenshot('products.png');

Never use page.waitForTimeout() here. A fixed wait is too long on a fast machine and too short on a busy CI runner, which is the definition of flaky. For data that comes from an API, consider serving a fixed response with page.route(), so the screenshot does not depend on what the staging database contains today.

Same viewport, same pixel density#

The screenshot size comes from the viewport, and devices['Desktop Chrome'] sets it to 1280x720. If you leave it to chance (or let one test resize the window), baselines and new screenshots end up with different sizes, and Playwright fails with Expected an image 1280px by 720px, received 375px by 812px before comparing a single pixel.

The device pixel ratio matters too. A test with deviceScaleFactor: 3 (common in mobile device presets) renders text and images at three times the resolution. By default toHaveScreenshot() uses scale: 'css', which captures one pixel per CSS pixel: a 375x812 viewport gives a 375x812 image at any scale factor. With scale: 'device' the same page gives 1125x2436, a file nine times larger whose comparison is also slower. Keep css unless you are testing high-density artwork specifically.

Operating systems render differently#

This one surprises everyone. The same Chromium build renders the same page differently on macOS, Linux and Windows, because each OS has different fonts installed and different text rendering (anti-aliasing, hinting, sub-pixel positioning). We generated baselines for this course's TodoMVC and dashboard tests on macOS, then ran the identical tests with the identical Playwright version in the official Linux Docker image:

All three failed, and nothing was wrong with the page. Baselines recorded inside the Linux container then passed in the Linux container every time.

That is why Playwright puts the platform in the baseline file name (-darwin, -linux, -win32): a Mac baseline is never compared against a Linux screenshot by accident. The practical rule follows from it: create and update baselines in the same environment that runs the tests. If CI runs on Linux, generate baselines on Linux, either in CI itself or locally with the same Docker image CI uses:

docker run --rm -v "$PWD":/work -w /work --ipc=host \
  mcr.microsoft.com/playwright:v1.65.0-noble \
  npx playwright test --update-snapshots=changed

The image tag must match your @playwright/test version exactly. Chapter 7 builds the CI workflow around this.

A stability checklist#

Before you trust a new visual test, run it several times in a row with --repeat-each=5. If it passes every time, check it against this list anyway:

• Clock fixed, locale and timezone pinned.
• Random, personalised and third-party content mocked, masked or hidden.
• The page asserted into its finished state before the screenshot.
• Viewport and device scale factor fixed by the project.
• Baselines created in the same OS, browser and Playwright version that runs the test.

Only after that should you reach for tolerances, which are the subject of the next chapter.