Why Tests Break on Locators
On this page
A UI test makes two kinds of claim. The first is about behaviour: "after logging in, the user sees their account". The second is about structure: "the username field is the first input in the second div of the form". Only the first claim is what you want to test. The second is an accident of how the page happens to be built today, and every locator decides how much of it your test depends on.
A locator is a prediction#
When you write a locator you are predicting that a certain property of an element will still be true next month. Some properties are safe bets, others change every time somebody touches the page:
| Property | Who changes it | How often |
|---|---|---|
A data-testid attribute |
Only someone working on tests | Rarely |
| The role and visible label ("button", "Save") | Product and copywriters | Occasionally, and on purpose |
A hand-written id or name
|
Developers | Occasionally |
| CSS classes | Designers, CSS refactors, framework upgrades | Often |
Nesting depth and wrapper divs |
Any layout change, component libraries | Very often |
| Position in a list or table | Data, sorting, new features | Constantly |
Generated ids (input-4821, UUIDs) |
The framework, on every build or page load | Always |
The further down this table a locator reaches, the more often it breaks for reasons that have nothing to do with the feature under test.
Seeing it happen#
The "challenging DOM" practice page is built to be hard to automate. Its three buttons get a new random id on every page load. This test, in tests/test_brittle.py, proves it:
from selenium.webdriver.common.by import By
URL = "https://the-internet.herokuapp.com/challenging_dom"
def test_generated_ids_change_on_every_load(driver):
driver.get(URL)
before = driver.find_element(By.CSS_SELECTOR, "a.button").get_attribute("id")
driver.refresh()
after = driver.find_element(By.CSS_SELECTOR, "a.button").get_attribute("id")
assert before != after
An id copied from DevTools into a test would pass once and fail on the next run. The buttons' labels are randomised too, so even the text is not a safe bet there.
The driver fixture used throughout the course lives in tests/conftest.py. It starts a local Chrome, or a browser on TestingBot when TB_KEY is set, and reports each test's result to the dashboard:
import os
import pytest
from selenium import webdriver
OPTIONS = {"chrome": webdriver.ChromeOptions, "firefox": webdriver.FirefoxOptions}
@pytest.hookimpl(hookwrapper=True)
def pytest_runtest_makereport(item, call):
outcome = yield
report = outcome.get_result()
if report.when == "call":
item.passed = report.passed
@pytest.fixture
def driver(request):
browser = os.environ.get("BROWSER", "chrome")
options = OPTIONS[browser]()
remote = "TB_KEY" in os.environ
if remote:
options.browser_version = "latest"
options.platform_name = os.environ.get("PLATFORM", "WIN11")
options.set_capability("tb:options", {
"key": os.environ["TB_KEY"],
"secret": os.environ["TB_SECRET"],
"name": request.node.name,
"build": os.environ.get("TB_BUILD", "locators"),
})
driver = webdriver.Remote("https://hub.testingbot.com/wd/hub", options=options)
print(f"\n{request.node.name}: session {driver.session_id}")
else:
driver = getattr(webdriver, browser.capitalize())(options=options)
yield driver
if remote:
passed = getattr(request.node, "passed", False)
driver.execute_script(f"tb:test-result={'passed' if passed else 'failed'}")
driver.quit()
pytest -s tests/ runs everything locally. export TB_KEY=... TB_SECRET=... first, and the same command runs on TestingBot; BROWSER=firefox switches browser. The tb:options keys are listed in the Selenium options reference, and tb:test-result is described in the test status documentation.
A test copied the id c8571ada-3bc7-406b-920f-5f071c183c64 from DevTools. It passed yesterday and fails today without any deploy. What is the most likely cause?
A UUID-shaped id is almost always generated by the application or a framework. It identifies the element for one page load only. A wait does not help, because the element with that id never appears again.
Position versus content#
The same page has a table with no helpful attributes at all. Here are two ways to read the fourth cell of the fourth row:
def test_position_versus_content(driver):
driver.get(URL)
by_position = driver.find_element(By.XPATH, "/html/body/div[2]/div/div/div/div/div[2]/table/tbody/tr[4]/td[4]")
by_content = driver.find_element(By.XPATH, "//tr[td[text()='Iuvaret3']]/td[4]")
assert by_position.text == by_content.text == "Definiebas3"
Both pass today. The first one is the kind of path that "Copy full XPath" in DevTools produces: it describes a dozen steps of layout from the root of the document, and adding a cookie banner, a wrapper div or a new row above it breaks it. The second one says "the row that contains the cell Iuvaret3" and only depends on the data you are testing. Chapter 4 shows how to write the second kind fluently.
What a broken locator costs#
A locator failure is expensive in a way that is easy to underestimate:
- Someone has to triage it. A red build first looks like a product bug. Somebody opens the video or the screenshot, finds that the feature works, and only then starts fixing the test. That is the expensive part, not the one-line fix.
- It trains people to ignore red builds. If a third of failures are "just the selectors again", real regressions get re-run instead of read. The flaky tests course covers this effect in detail; brittle locators are one of its main sources.
- It blocks unrelated work. A designer renames a class and twenty tests fail in a pull request that changed nothing functional.
- It multiplies. The same brittle selector is usually pasted into several tests. Keeping each locator in one place (a page object) limits the fix to one line, but it does not stop the breakage.
Your team already keeps every locator in page objects. A colleague argues that this makes brittle locators harmless. What is the flaw in that argument?
Centralising locators reduces the edit to one place, which is valuable. It does nothing about how often the locator breaks, and every break is a failed build someone has to investigate.
The goal for the rest of the course#
A good locator has three properties:
- Unique: it matches exactly one element (or exactly the set you mean).
- Meaningful: it describes what the element is to a user or to the test, not where it sits.
- Owned: whatever it depends on is something the team agrees not to change casually, such as a label, a role or a test id.
The next chapter starts with the tool you will use to check the first property on every locator you write: the browser's DevTools.