---
title: Scoping Scans, Rules and a Baseline of Known Issues
description: 'The first scan of a real application almost never passes. That leaves
  you with an awkward choice: fix everything before the check can go into CI, which
  can…'
source_url:
  html: https://testingbot.com/resources/courses/accessibility-testing/scoping-rules-and-baselines
  md: https://testingbot.com/resources/courses/accessibility-testing/scoping-rules-and-baselines.md
---

Course

[Accessibility Testing Tutorial: WCAG 2.2](https://testingbot.com/resources/courses/accessibility-testing)

Chapter 5 of 10

1. [1 Accessibility Testing Tutorial: WCAG 2.2 in Practice](https://testingbot.com/resources/courses/accessibility-testing)
2. [2 Manual Accessibility Checks You Still Need](https://testingbot.com/resources/courses/accessibility-testing/manual-accessibility-checks)
3. [3 axe-core with Playwright: Your First Scan](https://testingbot.com/resources/courses/accessibility-testing/axe-core-with-playwright)
4. [4 axe-core with Selenium in Java](https://testingbot.com/resources/courses/accessibility-testing/axe-with-selenium-java)
5. [5 Scoping Scans, Rules and a Baseline of Known Issues](https://testingbot.com/resources/courses/accessibility-testing/scoping-rules-and-baselines)
6. [6 Fixing Common Violations: Contrast, Forms and ARIA](https://testingbot.com/resources/courses/accessibility-testing/fixing-common-violations)
7. [7 Running Accessibility Checks in CI](https://testingbot.com/resources/courses/accessibility-testing/accessibility-checks-in-ci)
8. [8 Accessibility Testing on TestingBot](https://testingbot.com/resources/courses/accessibility-testing/accessibility-testing-on-testingbot)
9. [9 What Automated Accessibility Testing Cannot Catch](https://testingbot.com/resources/courses/accessibility-testing/what-automation-cannot-catch)
10. [10 Knowledge check](https://testingbot.com/resources/courses/accessibility-testing/knowledge-check)

[Exit course](https://testingbot.com/resources/courses)

 Chapter 5 of 10 Intermediate 8 min read 2 quick checks Updated Nov 2026 

# Scoping Scans, Rules and a Baseline of Known Issues
On this page
1. [A shared fixture](https://testingbot.com#a-shared-fixture)
2. [Scoping: include and exclude](https://testingbot.com#scoping-include-and-exclude)
3. [Choosing rules on purpose](https://testingbot.com#choosing-rules-on-purpose)
4. [A baseline of known violations](https://testingbot.com#a-baseline-of-known-violations)
5. [Keeping the baseline honest](https://testingbot.com#keeping-the-baseline-honest)

The first scan of a real application almost never passes. That leaves you with an awkward choice: fix everything before the check can go into CI, which can take months, or leave the check out of CI, which means new violations pile on top of the old ones. Neither is good.

This chapter shows the way out: decide what a scan covers, configure the rules on purpose rather than by accident, and record today's violations in a baseline so the build fails only on new ones while the backlog gets fixed.

## A shared fixture[#](https://testingbot.com#a-shared-fixture)

Every scan in a suite should use the same tags and exclusions, so put the builder in a Playwright fixture. Create `tests/axe.ts`:

```typescript
import { test as base } from '@playwright/test';
import AxeBuilder from '@axe-core/playwright';

export const WCAG_AA = ['wcag2a', 'wcag2aa', 'wcag21a', 'wcag21aa', 'wcag22a', 'wcag22aa'];

type AxeFixtures = { makeAxeBuilder: () => AxeBuilder };

export const test = base.extend<AxeFixtures>({
  makeAxeBuilder: async ({ page }, use) => {
    await use(() =>
      new AxeBuilder({ page })
        .withTags(WCAG_AA)
        // Third-party credits footer we do not own; tracked with its maintainers instead.
        .exclude('footer.info'),
    );
  },
});

export { expect } from '@playwright/test';
```

The fixture returns a function rather than a builder, so each call gets a fresh `AxeBuilder` and a test can still add its own `include` or `disableRules` on top. Tests now import `test` and `expect` from `./axe` instead of `@playwright/test`. The single test from chapter 2 is replaced by the specs below, so you can delete `tests/todo.a11y.spec.ts`.

## Scoping: include and exclude[#](https://testingbot.com#scoping-include-and-exclude)

`include(selector)` limits the scan to part of the page; `exclude(selector)` removes part of it. Both take CSS selectors and can be chained.

Exclude content you do not control and cannot change, and write down why next to the selector, as the fixture does. Typical candidates are embedded third-party widgets such as video players, maps and chat bubbles, which is exactly where the Mars demo's ARIA violations came from in chapter 3. Excluding them does not make them accessible; it moves them from "red build" to "a vendor issue someone owns". If nobody owns it, do not exclude it.

Include is useful for component-level tests. To check just TodoMVC's filter bar:

```typescript
test('filter bar, scanned on its own', async ({ page }) => {
  const input = page.getByPlaceholder('What needs to be done?');
  await input.fill('Buy milk');
  await input.press('Enter');

  const results = await new AxeBuilder({ page })
    .include('.filters')
    .disableRules(['color-contrast'])
    .analyze();

  expect(results.violations).toEqual([]);
});
```

This test also shows the riskiest option: `disableRules`.

## Choosing rules on purpose[#](https://testingbot.com#choosing-rules-on-purpose)

You have four levers:

| Method | Effect |
| --- | --- |
| `withTags([...])` | Run only rules that carry at least one of these tags |
| `withRules([...])` | Run only these rule ids |
| `disableRules([...])` | Run everything selected, except these rule ids |
| `options({...})` | Pass raw axe-core run options, for anything the helpers do not cover |

Useful tags beyond the WCAG ones: `best-practice` for Deque's recommendations (heading order, one `h1`, landmarks), `wcag2aaa` for the few AAA rules axe has, such as enhanced contrast, and `experimental` for rules still being tuned. A sensible policy is to fail the build on WCAG A and AA, and to run `best-practice` in a separate, non-blocking report.

`disableRules` switches a check off everywhere the builder is used. Disabling `color-contrast` in the component test above is defensible only because contrast is covered by the full-page scans; disabling it in the shared fixture would hide every contrast bug in the product. Treat `disableRules` in a shared fixture like a skipped test: rare, commented and reviewed. For a single known issue, a baseline is a better tool, because it stays specific to the element that is broken.

The header logo fails `color-contrast` and the designer needs two sprints to fix it. Which change keeps the build useful in the meantime?

 `disableRules(['color-contrast'])` in the shared fixture Remove `wcag2aa` from the tag list until it is fixed Record that one rule and selector in a baseline so only new violations fail the build Wrap the scan in `test.fixme()`

A baseline accepts exactly the known element while still failing on any new contrast problem. Disabling the rule or the whole AA tag would hide every future regression of that kind, and `fixme` stops checking the page altogether.

## A baseline of known violations[#](https://testingbot.com#a-baseline-of-known-violations)

A baseline is a committed file listing the violations you already know about. The test compares each scan against it and fails on anything new. Create `tests/a11y-baseline.ts`:

```typescript
import fs from 'node:fs';
import path from 'node:path';
import type { AxeResults } from 'axe-core';

const DIR = path.join(__dirname, '..', 'a11y-baseline');

// One line per failing element: "rule|selector". Stable, readable in a diff.
export function fingerprints(results: AxeResults): string[] {
  return results.violations
    .flatMap(v => v.nodes.map(n => `${v.id}|${n.target.join(' ')}`))
    .sort();
}

export function compareWithBaseline(name: string, results: AxeResults) {
  const file = path.join(DIR, `${name}.json`);
  const current = fingerprints(results);

  if (process.env.UPDATE_A11Y_BASELINE) {
    fs.mkdirSync(DIR, { recursive: true });
    fs.writeFileSync(file, JSON.stringify(current, null, 2) + '\n');
    return { newIssues: [], fixedIssues: [] };
  }

  const known: string[] = fs.existsSync(file) ? JSON.parse(fs.readFileSync(file, 'utf8')) : [];
  return {
    newIssues: current.filter(f => !known.includes(f)),
    fixedIssues: known.filter(f => !current.includes(f)),
  };
}
```

(`axe-core` is installed as a dependency of `@axe-core/playwright`, so the type import works without installing anything else.)

Design choices worth keeping if you write your own:

- **The fingerprint is rule plus selector** , not the failure message. Messages contain measured values such as contrast ratios that change with every tweak; the rule and element are what you are tracking. TestingBot's scheduled scans use the same `rule|selector` idea to decide which violations are new.
- **One file per scanned state** , so parallel tests never write the same file, and a diff shows which page changed.
- **Fixed issues fail too.** If an entry disappears from the scan, the test asks you to remove it from the baseline. Otherwise the file only ever grows, and a fixed bug can quietly come back later without failing anything.

Now the spec, `tests/todo.baseline.spec.ts`, scanning two states:

```typescript
import AxeBuilder from '@axe-core/playwright';
import { test, expect } from './axe';
import { compareWithBaseline } from './a11y-baseline';

test.beforeEach(async ({ page }) => {
  await page.goto('https://demo.playwright.dev/todomvc/');
});

test('empty list', async ({ makeAxeBuilder }) => {
  const results = await makeAxeBuilder().analyze();

  const { newIssues, fixedIssues } = compareWithBaseline('todo-empty', results);
  expect(newIssues, 'new accessibility violations').toEqual([]);
  expect(fixedIssues, 'fixed, so remove them from the baseline').toEqual([]);
});

test('list with an active and a completed item', async ({ page, makeAxeBuilder }) => {
  const input = page.getByPlaceholder('What needs to be done?');
  await input.fill('Buy milk');
  await input.press('Enter');
  await input.fill('Pay rent');
  await input.press('Enter');
  await page.getByRole('listitem').filter({ hasText: 'Pay rent' }).getByRole('checkbox').check();
  await expect(page.getByTestId('todo-count')).toHaveText('1 item left');

  const results = await makeAxeBuilder().analyze();

  const { newIssues, fixedIssues } = compareWithBaseline('todo-with-items', results);
  expect(newIssues, 'new accessibility violations').toEqual([]);
  expect(fixedIssues, 'fixed, so remove them from the baseline').toEqual([]);
});
```

The filter bar test from earlier goes in the same file. Note the `toHaveText('1 item left')` assertion before the second scan: it waits until the UI has settled, so axe does not scan a half-updated page.

The first run without a baseline fails and lists every violation as new. Review that list, then record it:

```bash
UPDATE_A11Y_BASELINE=1 npx playwright test tests/todo.baseline.spec.ts
npx playwright test tests/todo.baseline.spec.ts
```

The second run passes. `a11y-baseline/todo-with-items.json` now reads:

```json
[
  "color-contrast|.selected",
  "color-contrast|a[href$=\"#/active\"]",
  "color-contrast|a[href$=\"#/completed\"]",
  "color-contrast|h1"
]
```

The footer text from chapter 2 is gone because the fixture excludes `footer.info`. The heading and filter links are real, owned problems, so they are in the baseline, not excluded.

A developer fixed the filter link contrast and the baseline test now fails with "fixed, so remove them from the baseline". Why fail on good news?

 So the entry is removed; otherwise the baseline keeps accepting that violation and a regression would pass silently Because axe cannot tell a fixed element from a removed one Because baselines must never shrink

A baseline entry is permission for that violation to exist. Once it is fixed, keeping the permission means the bug can return without failing the build. Updating the file locks in the improvement.

## Keeping the baseline honest[#](https://testingbot.com#keeping-the-baseline-honest)

A baseline is debt, so manage it like debt:

- **Review baseline diffs like code.** A pull request that adds lines to `a11y-baseline/` is accepting new violations. Make it visible: require a code owner on the folder, and never regenerate the baseline as a reflex to get a build green.
- **Track the count.** The number of lines across the baseline files is a simple metric that should only go down.
- **Expect selector churn.** axe generates the shortest unique selector it can, so a markup refactor can change a selector and turn a known issue into a "new" one. Read the diff before you regenerate; if the rule and element are the same, updating is correct.
- **Re-check the baseline per browser if you scan several.** The next chapter and chapter 7 show that WebKit can report a violation Chromium does not.

You now have scans that are scoped, configured on purpose and safe to put in front of every pull request. Before doing that, the next chapter fixes the violations themselves.

[Previous · Chapter 4 axe-core with Selenium in Java](https://testingbot.com/resources/courses/accessibility-testing/axe-with-selenium-java) [Next · Chapter 6 Fixing Common Violations: Contrast, Forms and ARIA](https://testingbot.com/resources/courses/accessibility-testing/fixing-common-violations)

On this page

1. [A shared fixture](https://testingbot.com#a-shared-fixture)
2. [Scoping: include and exclude](https://testingbot.com#scoping-include-and-exclude)
3. [Choosing rules on purpose](https://testingbot.com#choosing-rules-on-purpose)
4. [A baseline of known violations](https://testingbot.com#a-baseline-of-known-violations)
5. [Keeping the baseline honest](https://testingbot.com#keeping-the-baseline-honest)

Certification

Earn the free, resume-worthy TestingBot Accessibility Testing Certification for this course.

[How it works](https://testingbot.com/resources/courses/accessibility-testing/certification)
