Visual Regression Testing Quiz

1. Which bug is a visual regression test most likely to catch that a functional test with toBeVisible() and toHaveText() assertions misses?

Overlap is a property of how the page is painted. Both elements are still in the DOM with the right text, so DOM assertions pass. The other three are behaviour, which functional tests check better.

2. A visual test passes for months. A customer then reports that the page it covers has looked wrong since launch. How is that possible?

Visual regression tests detect change, not correctness. Whatever you approve as the first baseline becomes "correct". That is why every new baseline needs a careful human look.

3. A product card shows a random "Only N left" badge. You want to keep testing the rest of the card. What do you use?

Masking paints over just the badge in both images; mocking the data makes it deterministic. A 50% pixel allowance would let half the card change unnoticed, and waiting does not make random content stable.

4. Your baselines were created on a Mac and CI runs Ubuntu. CI fails every visual test with a few thousand different pixels, mostly along text. What is the right fix?

Font rendering differs between operating systems, so baselines must come from the environment that runs the tests. Sharing the Mac files on Linux turns "missing" into "different", and tolerance large enough to cover OS differences hides real regressions.

5. With Playwright's default comparison settings, a button's background changes from #1a73e8 to #1e88e5 and the test still passes. Why?

threshold decides per pixel whether a colour difference counts. Small colour shifts stay under 0.2. maxDiffPixels is unset by default, which means zero different pixels are allowed; it never came into play here.

6. Your CSS has breakpoints at 600px and 1100px. A colleague adds visual tests at 768, 834, 1024 and 1080 pixels wide. What is the problem?

Breakpoints define layouts. 768 to 1080 is all one layout here, so the tests duplicate each other while the layouts below 600px and above 1100px are not tested at all. Pick one width per range.

7. Why does Playwright include the project name in a baseline's file name by default?

Browsers render the same page differently even when nothing is wrong. Per-project baselines keep those normal differences out of the comparison, so each project can use a strict tolerance.

8. A pull request intentionally changes the primary button style. What is the best way to get the new baselines into the pull request?

The baseline update is part of the change and should be reviewed with it. Generating it in the CI environment avoids OS differences. Config-level updates or deleted baselines remove the check for everything else too.

9. A designer wants to approve visual changes without using git. Which setup fits best?

When baselines are files in git, approving means committing. Server-side baselines move approval into a web interface, which suits reviewers outside the code workflow. Mailing images around loses the record of who approved what.