Selenium waits in Python: WebDriverWait and time.sleep

Most flaky Selenium tests are timing bugs. The test looks for an element a few hundred milliseconds before the page's JavaScript has created it, and fails on a slow CI machine but not on your laptop. This chapter shows the failure, the two wrong fixes, and the right one.

The failure#

The practice page dynamic.html has an "Add a box!" button. Clicking it runs setTimeout and appends a red box, #box0, one second later. Create tests/test_waits.py:

import time

import pytest
from selenium import webdriver
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait

DYNAMIC = "https://www.selenium.dev/selenium/web/dynamic.html"


@pytest.fixture
def driver():
    driver = webdriver.Chrome()
    yield driver
    driver.quit()


def test_without_a_wait(driver):
    driver.get(DYNAMIC)
    driver.find_element(By.ID, "adder").click()
    driver.find_element(By.ID, "box0")  # NoSuchElementException

Run it and pytest reports:

selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"[id="box0"]"}

The locator is correct. The element simply does not exist yet: find_element looks once, immediately, and gives up.

Wrong fix 1: time.sleep#

def test_with_sleep(driver):
    driver.get(DYNAMIC)
    driver.find_element(By.ID, "adder").click()
    time.sleep(2)  # passes, but always costs 2 seconds and still guesses
    assert driver.find_element(By.ID, "box0").is_displayed()

This passes. It also costs two full seconds every run, even when the box appears after one, and two seconds is a guess: on a loaded CI agent or a slower remote browser the delay may be longer, and the test fails again. Multiply by a few hundred tests and a suite full of sleeps is both slow and flaky, the worst combination.

Wrong fix 2: a large implicit wait#

driver.implicitly_wait(10)

An implicit wait makes every find_element call keep retrying for up to that many seconds before raising. It looks convenient, but it is a global setting with side effects:

• It only waits for an element to be present in the DOM, not visible, enabled or holding the right text.
• find_elements on something that is correctly absent now takes the full ten seconds to return an empty list.
• It interacts badly with explicit waits. The Selenium documentation warns not to mix them: a 10-second implicit wait combined with a 15-second explicit wait can time out after 20 seconds.

Leave the implicit wait at its default of 0 and use explicit waits everywhere.

The right fix: WebDriverWait#

An explicit wait polls a condition until it is satisfied or a timeout expires:

def test_with_explicit_wait(driver):
    driver.get(DYNAMIC)
    driver.find_element(By.ID, "adder").click()
    wait = WebDriverWait(driver, timeout=5)
    box = wait.until(EC.presence_of_element_located((By.ID, "box0")))
    assert box.get_dom_attribute("class") == "redbox"

wait.until(...) calls the condition every 0.5 seconds (the default poll interval) and returns the first truthy value it produces, here the element itself. If five seconds pass without success it raises TimeoutException. When we timed the two versions with pytest --durations=5, the sleep version took 2.24 seconds and the explicit wait 1.15: it continues as soon as the box exists.

Note the double parentheses. Expected conditions take a locator tuple, (By.ID, "box0"), not two arguments. EC.presence_of_element_located(By.ID, "box0") fails with a TypeError.

Choosing the condition#

The expected_conditions module (imported as EC by convention) has ready-made conditions. Wait for the thing the next line depends on:

The same page has a hidden input that a second button reveals after a second. Waiting for presence would return immediately (the input is always in the DOM), and typing into it would fail with ElementNotInteractableException. Visibility is the right condition:

revealed = driver.find_element(By.ID, "revealed")
driver.find_element(By.ID, "reveal").click()
WebDriverWait(driver, 10).until(EC.visibility_of(revealed))
revealed.send_keys("now visible")

Your own conditions#

A condition is any callable that takes the driver and returns something truthy when done. Falsy means "keep polling":

def test_custom_condition(driver):
    driver.get(DYNAMIC)
    adder = driver.find_element(By.ID, "adder")
    for _ in range(3):
        adder.click()

    def three_boxes(d):
        boxes = d.find_elements(By.CLASS_NAME, "redbox")
        return boxes if len(boxes) == 3 else False  # falsy means "keep polling"

    boxes = WebDriverWait(driver, timeout=5).until(three_boxes)
    assert len(boxes) == 3

Give until a message and a timeout tells you what was missing instead of an empty TimeoutException:

def test_timeout_message(driver):
    driver.get(DYNAMIC)
    wait = WebDriverWait(driver, timeout=2)
    with pytest.raises(TimeoutException):
        wait.until(EC.visibility_of_element_located((By.ID, "revealed")),
                   message="#revealed never became visible")

WebDriverWait also accepts poll_frequency and ignored_exceptions, for example ignored_exceptions=[StaleElementReferenceException] when the page re-renders the element you are polling.

Three rules#

1. Wait for a condition, never for a duration. time.sleep belongs in exactly zero tests.
2. Keep the implicit wait at 0 so explicit waits behave predictably.
3. Do not "fix" an intermittent failure by raising the timeout. Find which condition was not yet true, and wait for that. Our article on handling test flakiness covers the other common causes.

Every test file so far has started with its own copy of the driver fixture. The next chapter moves it into conftest.py and makes it configurable.