A Visual Testing Strategy That Lasts

You can now write, stabilise, tune, multiply and automate visual tests. The remaining question is which ones to write. Visual suites rarely fail because of the tooling. They fail because they grow to hundreds of noisy screenshots that nobody reviews properly, until a team turns them off. This chapter is about avoiding that.

Start from what a visual bug costs#

Rank candidate screens by what a visual defect there would cost, not by how easy they are to screenshot:

1. Revenue paths: pricing, sign-up, checkout, payment forms. A broken layout here costs money within hours.
2. Brand-critical first impressions: the homepage hero, the logged-in dashboard's first screen.
3. Shared components that appear everywhere: navigation, buttons, form fields, modals, tables. One test of the component protects every page that uses it.
4. Screens with a history of visual bugs, especially browser-specific ones.
5. Everything else, only when there is a reason.

A good first suite is ten to thirty element or component screenshots from the first three groups, in one or two browsers. That is enough to catch most CSS regressions and small enough that every failure gets real attention.

What not to test visually#

A visual test is the wrong tool when:

• The content changes by design: news feeds, search results, dashboards with live data. Test the layout with fixed data (mock the API) or test a component instead.
• A functional assertion says it better. "The error message is shown" is toBeVisible(); "the total is 42.00" is toHaveText(). These fail with a clear message, need no baseline and do not break when the font changes.
• The page is under active redesign. Baselines that change daily are just churn. Add visual tests when the design settles.
• You need to know whether the design is correct. Visual regression tests compare against what was approved. They cannot tell you whether the spacing matches the design file, whether the contrast is accessible or whether the copy is right. Those need design review and dedicated checks, such as an accessibility scanner for contrast.
• The pixels come from somewhere you do not control: embedded maps, third-party widgets, video. Mask or hide them (chapter 2), or leave them out.

A useful rule: every visual test should answer a question you can state in one sentence, such as "does the pricing table still look right in Safari at mobile width". If you cannot state it, the screenshot is probably too broad.

Ownership and review#

Every failed visual test ends with a human decision. Make it clear whose:

• The author of the change reviews first. If their pull request changes a baseline, they confirm it is intended and update it (chapter 7). Nobody else should be surprised by a visual change in their own pull request.
• A second person approves. The pull request reviewer sees the PNG diff next to the CSS diff. For design-heavy changes, add a designer as a reviewer on pull requests that touch baseline folders, for example with a CODEOWNERS rule on **/*-snapshots/**.
• Shared components need an owner. A change to the button component that updates 40 baselines should be reviewed by whoever owns the design system, not approved in bulk by whoever happened to make the change.

If designers or product managers review without using git, server-side baselines with a review screen fit better (chapter 6): they approve or reject runs in a dashboard.

Reviewing well#

A diff deserves more than a glance at the red pixels. For each failed comparison, ask:

1. Is this change caused by this pull request? If not, something is unstable (chapter 2) or the baseline is stale.
2. Is it intended? Compare with the ticket or the design.
3. Is it the only change? A diff can contain an intended change and an unintended one. Use the slider view to look at the whole image, not just the part you expected.
4. Does it affect other browsers and breakpoints the same way? One approval per browser, each looked at.

Reject the urge to approve all at once after a large refactor. If 60 baselines changed, either the change was broader than intended or the tests are too broad.

Keeping noise near zero#

Track two numbers, even informally: how often the visual job fails, and how often a failure was a real problem. If most failures are noise, fix the noisiest tests before writing new ones. The checklist from chapter 2 (clock, data, masks, viewport, environment) solves most cases; the rest usually need a narrower screenshot.

A flaky visual test is worse than a flaky functional test, because the fix people reach for, updating the baseline, also removes the test's protection. Quarantine a noisy test (skip it with a ticket) rather than re-approving it every week.

Maintaining baselines over time#

• Upgrade Playwright on purpose, in its own pull request with updated baselines (chapter 5).
• Delete baselines with their tests. Playwright does not remove orphaned snapshot files; check for them when you rename or delete tests.
• Re-review baselines occasionally. A baseline approved a year ago may preserve a bug everyone got used to. When a page is redesigned, look at its baselines with fresh eyes before approving the new ones.
• Keep the matrix honest. Revisit which browsers and breakpoints you test as your analytics change (chapters 4 and 5).

The tool landscape#

Visual regression tools fall into a few groups, and the concepts in this course apply to all of them:

• Test-runner assertions, such as Playwright's toHaveScreenshot(), used throughout this course. Baselines are files in your repository and review happens in pull requests. Free, and strongest when developers own the tests.
• Open-source standalone tools that drive a browser over a list of URLs or scenarios defined in a config file, then produce an HTML report of differences. Useful when the people writing visual checks do not maintain a test suite.
• Hosted visual testing services, which take or receive screenshots, store baselines on a server, render across browsers, and provide a review dashboard with approvals. TestingBot's visual testing (chapter 6) is in this group, along with several commercial products. They differ in comparison algorithms (pixel tolerances, anti-aliasing handling, layout-aware or AI-assisted comparison), in which frameworks they integrate with, and in pricing. Compare them on a trial with your own pages: false-failure rate on your real UI matters more than any feature list.

Whatever you pick, the hard parts are the same: stable screenshots, baselines from a consistent environment, a deliberate tolerance and a review that people actually do.

A one-page plan#

If you take one thing from this course into your project, make it this sequence:

1. Pick ten high-value components or screens.
2. Write element-level toHaveScreenshot() tests with a fixed clock, mocked data and masks.
3. Generate Linux baselines in the Playwright Docker image and review every one by eye.
4. Run them on every pull request in the same image; upload the report with the diffs.
5. Add the manual baseline-update workflow and require review on snapshot changes.
6. Add browsers, breakpoints and real-browser grid runs (nightly) once the first ten have been quiet for a few weeks.

Then grow the suite only as fast as your team can review it.