XPath in Selenium: Text, Axes and When to Use It
On this page
XPath has a bad reputation in test automation, and most of it comes from one kind of XPath: the absolute path copied from DevTools. Written by hand and relative to something meaningful, XPath is a precise tool that does two things CSS cannot: match on text, and walk in any direction through the tree. This chapter is an XPath tutorial for Selenium users, built on the tables and login pages of the-internet.herokuapp.com.
The syntax you need#
| Expression | Means |
|---|---|
//td |
every td anywhere in the document |
//table[@id='table1']//td |
every td anywhere inside table 1 |
/td |
a td that is a direct child (after another step) |
[@attr='v'] |
attribute equals v
|
[text()='v'] |
has a text node child equal to v
|
[normalize-space()='v'] |
the element's whole text, whitespace collapsed, equals v
|
[contains(., 'v')] |
the element's text contains v
|
[td[3]='v'] |
has a third td child whose text is v
|
.. |
the parent |
[2] |
the second matching child of its parent |
(expr)[2] |
the second match of the whole expression |
Start every expression you write with // and something meaningful, never with /html.
text() versus normalize-space()
#
The login page's button looks like this:
<button class="radius" type="submit"><i class="fa fa-2x fa-sign-in"> Login</i></button>
The visible text "Login" is inside the <i>, with a leading space. Here is what three common expressions make of it, in tests/test_xpath.py:
from selenium.webdriver.common.by import By
LOGIN = "https://the-internet.herokuapp.com/login"
TABLES = "https://the-internet.herokuapp.com/tables"
DOM = "https://the-internet.herokuapp.com/challenging_dom"
def xpath(driver, expression):
return driver.find_elements(By.XPATH, expression)
def test_text_versus_normalize_space(driver):
driver.get(LOGIN)
assert xpath(driver, "//button[text()='Login']") == []
assert len(xpath(driver, "//button[normalize-space()='Login']")) == 1
assert len(xpath(driver, "//button[contains(., 'Login')]")) == 1
-
text()only looks at the button's own text nodes. It has none, so the first expression finds nothing, even though you can see "Login" on the screen. -
normalize-space()with no argument takes the string value of the whole element, including descendants, trims it and collapses inner whitespace. This is what a user would call "the button's text". -
contains(., 'Login')also includes descendants, but would match "Login with Google" as well.
Use normalize-space()='...' as your default for text matching. Reach for contains() when the text has a variable part ("3 items left"), and accept that it can match more.
//a[text()='Sign in'] finds nothing, but the link clearly says "Sign in". The markup is <a href="/login"><span>Sign in</span></a>. Which expression finds it?
The text belongs to the span, so text() on the link sees nothing. normalize-space() uses the string value of the whole link, descendants included. @text is an attribute that does not exist, and Selenium cannot return a text node as an element.
Axes: from a cell to its row and beyond#
XPath can move in any direction from a node. That makes it the natural tool for "the Edit link in the row for this email address", the most common table task in UI testing:
def test_from_a_cell_to_its_row(driver):
driver.get(TABLES)
edit = driver.find_element(
By.XPATH,
"//table[@id='table1']//td[text()='jdoe@hotmail.com']"
"/following-sibling::td/a[text()='edit']")
edit.click()
assert driver.current_url.endswith("#edit")
last_name = driver.find_element(
By.XPATH, "//table[@id='table1']//tr[td[3]='jdoe@hotmail.com']/td[1]")
assert last_name.text == "Doe"
table = driver.find_element(By.XPATH, "//td[text()='Conway']/ancestor::table")
assert table.get_attribute("id") == "table1"
The three expressions show three styles:
-
following-sibling::walks sideways from the email cell to the cells after it.preceding-sibling::goes the other way, andparent::(or..) goes up one level. -
A predicate on the row,
tr[td[3]='jdoe@hotmail.com'], says "the row whose third cell is this email" and then steps down to the first cell. This reads better than walking sideways, and it is how you should usually write row lookups. (td[3]is a column position: it is stable as long as the columns are, which is usually true. Chapter 8 removes even that dependency.) -
ancestor::walks up any number of levels until it finds a match. Useful for "the form or dialog this field belongs to".
Plain text() is fine in this table because each cell holds nothing but its text. As soon as there is markup inside the element, switch to normalize-space().
Positions need parentheses#
This one catches almost everybody once:
def test_position_needs_parentheses(driver):
driver.get(TABLES)
assert xpath(driver, "//table[@id='table1']//a[text()='edit'][2]") == []
second = xpath(driver, "(//table[@id='table1']//a[text()='edit'])[2]")
assert len(second) == 1
//a[text()='edit'][2] means "every edit link that is the second edit link within its parent". Each action cell has only one, so nothing matches. To get the second match in the whole document, wrap the expression in parentheses first. The same applies to [last()].
A list has five li items, each in its own ul. //li[1] returns five elements. Why?
A positional predicate after a step counts among siblings. Every li is first in its own list, so all five match. Parentheses apply the position to the whole result set. XPath positions start at 1, not 0.
Classes in XPath#
XPath treats class as a plain string, which makes the obvious expression wrong:
def test_class_contains_is_a_trap(driver):
driver.get(DOM)
assert len(xpath(driver, "//a[contains(@class, 'button')]")) == 3
whole_word = "//a[contains(concat(' ', normalize-space(@class), ' '), ' alert ')]"
assert len(xpath(driver, whole_word)) == 1
@class='button' fails when an element has more than one class, and contains(@class, 'button') also matches button-group or big-button. The concat idiom matches the whole class name, but it is ugly. If you are selecting by class, use CSS (a.alert); that is what CSS is good at.
When XPath is the right tool#
Use XPath when you need something CSS cannot express:
-
Text that is the best handle, and you are in Selenium, which has no text locator besides
By.LINK_TEXTfor links. (In Playwright, prefergetByRolewith a name orgetByText.) -
Going up or sideways: the row of a cell, the label before a field, the section that contains a heading. CSS
:has()now covers many of these, and is often the more readable choice when it does. -
A condition on a child's text:
//tr[td[normalize-space()='Doe']]. CSS:has()cannot match on text, so this stays XPath territory.
Prefer CSS for everything else: ids, attributes, classes and simple descent are shorter and easier to read in CSS. Performance is rarely a reason to choose either: on a page of normal size both resolve in a few milliseconds, which is nothing next to a network round trip to the browser.
Two limits to know about: XPath does not see inside shadow roots (chapter 7), and browsers implement XPath 1.0 only, so functions such as matches() or lower-case() from later versions fail with an invalid selector error.
Summary#
- Write XPath relative to something meaningful, never from
/html. - Match text with
normalize-space()='...'; usecontains()only for variable text. - Use axes and row predicates (
tr[td[3]='...']) to go from a known cell to the element you need. -
(expr)[n]is the n-th match;expr[n]is the n-th child of each parent. - Select by class with CSS, not XPath.