Selenium locators in Python: the By API

Almost every line of a Selenium test starts by finding an element. How you find it decides whether the test survives the next redesign or breaks every sprint. This chapter covers the eight By strategies, the difference between find_element and find_elements, and a simple rule for choosing.

The By class#

In Selenium 4 every lookup goes through two methods and a By constant:

from selenium.webdriver.common.by import By

element = driver.find_element(By.ID, "my-text-id")        # one element, or an exception
elements = driver.find_elements(By.CSS_SELECTOR, "input")  # a list, possibly empty

The old helpers such as find_element_by_id() were removed in Selenium 4.3. If a tutorial or a Stack Overflow answer uses them, it predates that, and the code fails with AttributeError.

Under the hood, Selenium translates ID, NAME and CLASS_NAME into CSS selectors before sending them to the driver. You will see this in error messages: a failed By.ID lookup reports {"method":"css selector","selector":"[id=\"box0\"]"}.

Trying them out#

Create tests/test_locators.py. The small fixture at the top starts and stops Chrome for each test; chapter 4 explains fixtures properly and moves this into a shared file.

import pytest
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import Select

URL = "https://www.selenium.dev/selenium/web/web-form.html"


@pytest.fixture
def driver():
    driver = webdriver.Chrome()
    yield driver
    driver.quit()


def test_locator_strategies(driver):
    driver.get(URL)

    driver.find_element(By.ID, "my-text-id").send_keys("by id")
    driver.find_element(By.NAME, "my-password").send_keys("by name")
    driver.find_element(By.CSS_SELECTOR, "textarea[name='my-textarea']").send_keys("by css")
    heading = driver.find_element(By.TAG_NAME, "h1")
    link = driver.find_element(By.LINK_TEXT, "Return to index")
    submit = driver.find_element(By.XPATH, "//button[text()='Submit']")

    assert heading.text == "Web form"
    assert link.get_attribute("href").endswith("/index.html")
    assert submit.is_enabled()


def test_form_controls(driver):
    driver.get(URL)

    Select(driver.find_element(By.NAME, "my-select")).select_by_visible_text("Three")
    driver.find_element(By.ID, "my-check-2").click()

    checked = driver.find_elements(By.CSS_SELECTOR, "input[name='my-check']:checked")
    assert [box.get_attribute("id") for box in checked] == ["my-check-1", "my-check-2"]
    assert not driver.find_element(By.NAME, "my-disabled").is_enabled()


def test_absent_element_returns_empty_list(driver):
    driver.get(URL)
    assert driver.find_elements(By.ID, "does-not-exist") == []


def test_scoped_search(driver):
    driver.get(URL)
    form = driver.find_element(By.TAG_NAME, "form")
    inputs = form.find_elements(By.CSS_SELECTOR, "input[type='text']")
    assert len(inputs) == 4

pytest -q tests/test_locators.py runs four tests, each in a fresh browser.

A few things worth noticing:

• Select wraps a native <select> element and lets you choose by visible text, value or index. It only works on real <select> elements; custom dropdowns built from divs are clicked like anything else.
• get_attribute("href") returns the resolved property (a full URL here), while get_dom_attribute("href") returns exactly what is written in the HTML (./index.html). Use the DOM version when you want to assert on the markup.
• Scoped search. Calling find_elements on an element searches only inside it. That is how you say "the inputs in this form" or "the Delete button in this table row" without writing a long selector.

find_element versus find_elements#

find_element raises NoSuchElementException when nothing matches. find_elements returns an empty list. That makes find_elements the right tool for asserting that something is absent:

assert driver.find_elements(By.ID, "does-not-exist") == []

Wrapping find_element in try/except NoSuchElementException does the same thing with more code, and is easy to get wrong.

Choosing a locator that lasts#

A locator is a bet on which part of the page will not change. Bet on the parts that describe what an element is, not where it sits:

1. A dedicated test attribute, such as [data-testid='checkout'], if your team can add them. Nothing else depends on it, so nobody changes it by accident.
2. An id or name that is stable. Avoid generated IDs such as input-4821 that change per build.
3. A short CSS selector on meaningful attributes: button[type='submit'], input[name='email'].
4. Link text or XPath on visible text when the text is the thing you care about. It breaks when copy changes or the site is translated, which may be what you want.
5. Never an absolute path such as /html/body/div[2]/div/form/div[1]/input, or a chain of layout classes such as .col-md-4 > .form-label:nth-child(3). Both encode the layout, and the layout is what changes most.

CSS selectors are usually shorter and easier to read than XPath. XPath is still worth knowing for two things CSS cannot do: matching on text (//button[text()='Submit']) and walking up the tree (//td[text()='Alice']/..).

Getting locators from the browser#

In Chrome DevTools, open the Elements panel and press Ctrl+F (Cmd+F on macOS): the search box accepts CSS selectors and XPath and highlights every match. Aim for exactly one match. If the page offers no good attribute, that is worth raising with the developers; adding data-testid attributes is cheap and saves hours of maintenance.

Once a page is used by more than one test, keep its locators in one place instead of repeating them in every test. That is the page object pattern, which chapter 6 introduces. Before that, the next chapter fixes the most common reason a correct locator still fails: looking for an element before it exists.