Exit course
Chapter 2 of 12 Intermediate 6 min read 2 quick checks Updated

Why Tests Break on Locators

On this page
  1. A locator is a prediction
  2. Seeing it happen
  3. Position versus content
  4. What a broken locator costs
  5. The goal for the rest of the course

A UI test makes two kinds of claim. The first is about behaviour: "after logging in, the user sees their account". The second is about structure: "the username field is the first input in the second div of the form". Only the first claim is what you want to test. The second is an accident of how the page happens to be built today, and every locator decides how much of it your test depends on.

A locator is a prediction#

When you write a locator you are predicting that a certain property of an element will still be true next month. Some properties are safe bets, others change every time somebody touches the page:

Property Who changes it How often
A data-testid attribute Only someone working on tests Rarely
The role and visible label ("button", "Save") Product and copywriters Occasionally, and on purpose
A hand-written id or name Developers Occasionally
CSS classes Designers, CSS refactors, framework upgrades Often
Nesting depth and wrapper divs Any layout change, component libraries Very often
Position in a list or table Data, sorting, new features Constantly
Generated ids (input-4821, UUIDs) The framework, on every build or page load Always

The further down this table a locator reaches, the more often it breaks for reasons that have nothing to do with the feature under test.

Seeing it happen#

The "challenging DOM" practice page is built to be hard to automate. Its three buttons get a new random id on every page load. This test, in tests/test_brittle.py, proves it:

from selenium.webdriver.common.by import By

URL = "https://the-internet.herokuapp.com/challenging_dom"


def test_generated_ids_change_on_every_load(driver):
    driver.get(URL)
    before = driver.find_element(By.CSS_SELECTOR, "a.button").get_attribute("id")
    driver.refresh()
    after = driver.find_element(By.CSS_SELECTOR, "a.button").get_attribute("id")
    assert before != after

An id copied from DevTools into a test would pass once and fail on the next run. The buttons' labels are randomised too, so even the text is not a safe bet there.

The driver fixture used throughout the course lives in tests/conftest.py. It starts a local Chrome, or a browser on TestingBot when TB_KEY is set, and reports each test's result to the dashboard:

import os

import pytest
from selenium import webdriver

OPTIONS = {"chrome": webdriver.ChromeOptions, "firefox": webdriver.FirefoxOptions}


@pytest.hookimpl(hookwrapper=True)
def pytest_runtest_makereport(item, call):
    outcome = yield
    report = outcome.get_result()
    if report.when == "call":
        item.passed = report.passed


@pytest.fixture
def driver(request):
    browser = os.environ.get("BROWSER", "chrome")
    options = OPTIONS[browser]()
    remote = "TB_KEY" in os.environ
    if remote:
        options.browser_version = "latest"
        options.platform_name = os.environ.get("PLATFORM", "WIN11")
        options.set_capability("tb:options", {
            "key": os.environ["TB_KEY"],
            "secret": os.environ["TB_SECRET"],
            "name": request.node.name,
            "build": os.environ.get("TB_BUILD", "locators"),
        })
        driver = webdriver.Remote("https://hub.testingbot.com/wd/hub", options=options)
        print(f"\n{request.node.name}: session {driver.session_id}")
    else:
        driver = getattr(webdriver, browser.capitalize())(options=options)
    yield driver
    if remote:
        passed = getattr(request.node, "passed", False)
        driver.execute_script(f"tb:test-result={'passed' if passed else 'failed'}")
    driver.quit()

pytest -s tests/ runs everything locally. export TB_KEY=... TB_SECRET=... first, and the same command runs on TestingBot; BROWSER=firefox switches browser. The tb:options keys are listed in the Selenium options reference, and tb:test-result is described in the test status documentation.

A test copied the id c8571ada-3bc7-406b-920f-5f071c183c64 from DevTools. It passed yesterday and fails today without any deploy. What is the most likely cause?

A UUID-shaped id is almost always generated by the application or a framework. It identifies the element for one page load only. A wait does not help, because the element with that id never appears again.

Position versus content#

The same page has a table with no helpful attributes at all. Here are two ways to read the fourth cell of the fourth row:

def test_position_versus_content(driver):
    driver.get(URL)
    by_position = driver.find_element(By.XPATH, "/html/body/div[2]/div/div/div/div/div[2]/table/tbody/tr[4]/td[4]")
    by_content = driver.find_element(By.XPATH, "//tr[td[text()='Iuvaret3']]/td[4]")
    assert by_position.text == by_content.text == "Definiebas3"

Both pass today. The first one is the kind of path that "Copy full XPath" in DevTools produces: it describes a dozen steps of layout from the root of the document, and adding a cookie banner, a wrapper div or a new row above it breaks it. The second one says "the row that contains the cell Iuvaret3" and only depends on the data you are testing. Chapter 4 shows how to write the second kind fluently.

What a broken locator costs#

A locator failure is expensive in a way that is easy to underestimate:

  • Someone has to triage it. A red build first looks like a product bug. Somebody opens the video or the screenshot, finds that the feature works, and only then starts fixing the test. That is the expensive part, not the one-line fix.
  • It trains people to ignore red builds. If a third of failures are "just the selectors again", real regressions get re-run instead of read. The flaky tests course covers this effect in detail; brittle locators are one of its main sources.
  • It blocks unrelated work. A designer renames a class and twenty tests fail in a pull request that changed nothing functional.
  • It multiplies. The same brittle selector is usually pasted into several tests. Keeping each locator in one place (a page object) limits the fix to one line, but it does not stop the breakage.

Your team already keeps every locator in page objects. A colleague argues that this makes brittle locators harmless. What is the flaw in that argument?

Centralising locators reduces the edit to one place, which is valuable. It does nothing about how often the locator breaks, and every break is a failed build someone has to investigate.

The goal for the rest of the course#

A good locator has three properties:

  1. Unique: it matches exactly one element (or exactly the set you mean).
  2. Meaningful: it describes what the element is to a user or to the test, not where it sits.
  3. Owned: whatever it depends on is something the team agrees not to change casually, such as a label, a role or a test id.

The next chapter starts with the tool you will use to check the first property on every locator you write: the browser's DevTools.