Exit course
Chapter 11 of 12 Intermediate 7 min read 1 quick check Updated

A Locator Review Checklist

On this page
  1. The checklist
  2. Automate the obvious part
  3. Run the suites on TestingBot
  4. Summary

Locators are written once and read in every failed build afterwards. The cheapest moment to fix a brittle one is in code review, before it is copied into ten more tests. This chapter turns the course into a checklist you can paste into a pull request template, adds a small script that flags the obvious smells automatically, and runs the course suites on two browsers on TestingBot.

The checklist#

For every new or changed locator, the reviewer asks:

  1. Does it match exactly one element (or exactly the set the test means)? Checked in DevTools with $$ / $x, or with a count assertion.
  2. Does it describe what the element is, not where it sits? No absolute paths, no chains of layout divs, no :nth-child or [n] unless order is what the test checks.
  3. Is it the best available handle? Role and name or label first, then a test id, then a meaningful attribute, then scoped CSS or XPath. If a better handle is missing, is there a ticket to add it?
  4. Does it avoid generated values? No UUIDs, build hashes, ember123-style ids or CSS-in-JS class names.
  5. Does it avoid visual classes? .btn-primary, .mt-3 and grid classes describe looks, which change.
  6. Is text matching deliberate? normalize-space() rather than text(), exact: true where a substring could match more, and no reliance on copy that is translated or A/B tested.
  7. Is it scoped? Repeated elements (rows, cards, "Edit" buttons) are found inside their container, which is found by content.
  8. Does it cross boundaries explicitly? Shadow roots and frames are handled once, in a helper or page object, and frames are always switched back.
  9. Is it defined once? Locators live in a page object or component object, not repeated across tests (see the page object model course).
  10. Is it stored as a locator, not an element? No cached WebElements that can go stale.

A pull request adds getByRole('button', { name: 'Speichern' }). The app is tested in German only, but the copy team rewrites button labels often. Which review comment applies?

Role locators are the default because names usually change only when users would notice. If the copy changes for reasons that should not break tests, the checklist's text-matching item applies and a test id is more stable. Role locators can be unique, and a German label is not generated.

Automate the obvious part#

Half the checklist can be checked by a script. This one scans test files for common smells and exits non-zero when it finds any, so it can run in CI next to the linter:

"""Flag locator smells in test files: python locator_lint.py tests/"""
import re
import sys
from pathlib import Path

SMELLS = [
    (r"""["'](xpath=)?/html""", "absolute XPath from the document root"),
    (r"""(/[a-z]+\[\d+\]){2,}""", "XPath built from positions"),
    (r"""contains\(@class""", "contains(@class, ...) also matches longer class names"),
    (r"""text\(\)\s*=""", "text()= breaks on whitespace and nested tags; prefer normalize-space()"),
    (r"""[#'"=][\w-]*?[0-9a-f]{8}-[0-9a-f]{4}-""", "looks like a generated id (UUID)"),
    (r"""(\.[\w-]+\s+){3,}""", "long chain of class names"),
    (r""":nth-child\(\d+\)\s*>?\s*\S*:nth-child""", "several positional pseudo-classes"),
]


def lint(path):
    findings = 0
    for number, line in enumerate(path.read_text().splitlines(), start=1):
        if "lint: ok" in line:
            continue
        for pattern, message in SMELLS:
            if re.search(pattern, line):
                print(f"{path}:{number}: {message}\n    {line.strip()}")
                findings += 1
    return findings


if __name__ == "__main__":
    roots = [Path(arg) for arg in sys.argv[1:]] or [Path("tests")]
    files = [f for root in roots for f in (root.rglob("*.py") if root.is_dir() else [root])]
    total = sum(lint(f) for f in files)
    print(f"{total} finding(s) in {len(files)} file(s)")
    sys.exit(1 if total else 0)

Run against this course's own Selenium tests, it reports the deliberately bad examples:

$ python locator_lint.py tests/
...
tests/test_xpath.py:45: contains(@class, ...) also matches longer class names
    assert len(xpath(driver, "//a[contains(@class, 'button')]")) == 3
tests/test_brittle.py:16: absolute XPath from the document root
    by_position = driver.find_element(By.XPATH, "/html/body/div[2]/div/div/div/div/div[2]/table/tbody/tr[4]/td[4]")
tests/test_brittle.py:16: XPath built from positions
    by_position = driver.find_element(By.XPATH, "/html/body/div[2]/div/div/div/div/div[2]/table/tbody/tr[4]/td[4]")
...
10 finding(s) in 8 file(s)

It also flags every text()=, including the ones in chapter 4 that are fine because the cells hold plain text. That is the right trade-off for a lint: it prompts a second look, and a reviewer who agrees the locator is fine adds a # lint: ok comment to the line to silence it. Extend the list with your team's own smells, such as your design system's class prefix, and add *.ts to the glob for Playwright files.

Run the suites on TestingBot#

A locator that works in Chrome on your laptop can still fail elsewhere: :has() in an older browser, a shadow root in a different engine, a relative locator on a different window size. Running the suite on more than one browser is the final check.

The Selenium suite needs nothing new: the conftest.py from chapter 1 switches to TestingBot when TB_KEY is set.

export TB_KEY=your-key TB_SECRET=your-secret TB_BUILD=locators-ch10
pytest -s tests/
BROWSER=firefox pytest -s tests/

For Playwright, a helper builds the TestingBot endpoint, and the config connects to it when TB_KEY is set:

// testingbot.ts
export function testingbotEndpoint(browserName: 'chrome' | 'firefox' | 'safari' | 'edge', platform: string): string {
  const capabilities = {
    browserName,
    browserVersion: 'latest',
    platform,
    'tb:options': {
      key: process.env.TB_KEY,
      secret: process.env.TB_SECRET,
      build: process.env.TB_BUILD ?? 'locators',
    },
  };
  return `wss://cloud.testingbot.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}`;
}
// playwright.config.ts
import { defineConfig } from '@playwright/test';
import { testingbotEndpoint } from './testingbot';

const remote = !!process.env.TB_KEY;

export default defineConfig({
  testDir: './tests',
  workers: 1,
  projects: remote
    ? [
        { name: 'chrome-win11', use: { browserName: 'chromium', connectOptions: { wsEndpoint: testingbotEndpoint('chrome', 'WIN11') } } },
        { name: 'firefox-win11', use: { browserName: 'firefox', connectOptions: { wsEndpoint: testingbotEndpoint('firefox', 'WIN11') } } },
      ]
    : [{ name: 'chromium', use: { browserName: 'chromium' } }],
});
TB_BUILD=locators-ch10 npx playwright test

workers: 1 runs one remote session at a time; raise it up to your plan's parallel limit. Unlike the Selenium fixture, which starts a browser per test, Playwright keeps one remote browser per worker and project and gives each test a fresh context in it, so a run produces one TestingBot session per project. The fixture renames the session and reports a result after every test, so the dashboard shows the name and result of the last test that ran in it. A failure is not hidden by later passes: when a test fails, Playwright discards the worker and its browser and starts a new one for the remaining tests, so the failed result is the last one reported on that session and the next tests run in a new session. Rely on the Playwright report for per-test results, and use the sessions for their videos and logs. The Playwright capabilities generator builds the capabilities for other browser and platform combinations, and the build value groups every session of a run in the TestingBot dashboard, where each one has a video, logs and the pass or fail mark set by the fixtures.

In our run, the 20 Selenium tests and the 12 Playwright tests passed in Chrome and Firefox on Windows 11. One difference showed up while writing this course: the error for an XPath query inside a shadow root is worded differently per browser, which is why the test in chapter 7 accepts two exception types.

Summary#

  • Review every locator against the ten questions; put them in your pull request template.
  • Automate the mechanical half with a small lint and silence deliberate exceptions explicitly.
  • Run the suite on more than one browser before trusting a locator.