Exit course
Chapter 5 of 11 Intermediate 7 min read 2 quick checks Updated

Selenium Across Browsers and Versions

On this page
  1. One options class per browser
  2. Versions and platforms
  3. The matrix as data
  4. A parametrized pytest fixture
  5. The test itself knows nothing about browsers
  6. Checking what you actually got
  7. Running a subset
  8. Summary

Selenium WebDriver is a W3C standard, and every major browser ships a driver for it: chromedriver, msedgedriver, geckodriver for Firefox and safaridriver for Safari. That is its great strength for cross-browser testing: the same test talks to all of them through one protocol. This chapter takes one Python test and runs it on five browser, version and operating system combinations, with a fixture you can reuse for any suite.

One options class per browser#

In Selenium 4 you describe the browser you want with an options object. Each browser has its own class, and each class does two things: it sets browserName, and it gives you a place for that browser's own settings, which travel in a vendor-prefixed key.

from selenium import webdriver

chrome = webdriver.ChromeOptions()
chrome.add_argument("--lang=de-DE")

firefox = webdriver.FirefoxOptions()
firefox.set_preference("intl.accept_languages", "de-DE")

safari = webdriver.SafariOptions()

for options in (chrome, firefox, safari):
    options.browser_version = "latest"
    print(options.to_capabilities())

The output (abridged) shows where everything goes:

{'browserName': 'chrome', 'browserVersion': 'latest', 'goog:chromeOptions': {'args': ['--lang=de-DE'], ...}}
{'browserName': 'firefox', 'acceptInsecureCerts': True, 'browserVersion': 'latest', 'moz:firefoxOptions': {'prefs': {'intl.accept_languages': 'de-DE', ...}}}
{'browserName': 'safari', 'platformName': 'mac', 'browserVersion': 'latest'}

Three things to notice:

  • The same intent needs different code per browser. Setting the language is a command-line argument in Chrome and a preference in Firefox. Browser-specific settings are one of the few places where a cross-browser suite legitimately branches on the browser.
  • Defaults differ. FirefoxOptions sets acceptInsecureCerts to True, SafariOptions sets platformName to mac, and ChromeOptions sets neither. If a test against a self-signed staging certificate passes in Firefox and fails in Chrome, this is why.
  • EdgeOptions works like ChromeOptions, because Edge is Chromium-based; its settings travel under ms:edgeOptions.

Your staging site uses a self-signed certificate. The suite passes in Firefox and fails in Chrome with a certificate error. What is the cleanest fix?

FirefoxOptions accepts insecure certificates by default and the other classes do not. Setting the standard acceptInsecureCerts capability explicitly removes the difference for every browser; a Chrome-only flag fixes one browser and leaves Safari and Edge failing.

Versions and platforms#

Two more standard capabilities pick the exact configuration:

  • browser_version: a version number such as "155", or a relative one. On TestingBot, "latest" is the newest release, "latest-1" the one before it, and so on. Relative versions keep your matrix current without edits.
  • platform_name: the operating system. On TestingBot these are names such as WIN11, WIN10, TAHOE, SEQUOIA, SONOMA and LINUX. The browser list shows every combination.

Everything that is specific to the grid goes in one vendor key, tb:options: your credentials, a test name, a build name and options such as screen resolution. The Selenium options reference lists them all.

The matrix as data#

Keep the matrix out of the tests. A plain list of dictionaries in matrix.py is enough, and it is the list chapter 2 produced:

# One entry per browser/OS combination the suite must pass on.
MATRIX = [
    {"browser": "chrome",  "version": "latest",   "platform": "WIN11"},
    {"browser": "edge",    "version": "latest",   "platform": "WIN11"},
    {"browser": "firefox", "version": "latest",   "platform": "WIN11"},
    {"browser": "safari",  "version": "latest",   "platform": "TAHOE"},
    {"browser": "chrome",  "version": "latest-1", "platform": "SEQUOIA"},
]

A parametrized pytest fixture#

A pytest fixture with params runs every test that uses it once per entry. Put this in tests/conftest.py:

import os

import pytest
from selenium import webdriver

from matrix import MATRIX

OPTIONS = {
    "chrome": webdriver.ChromeOptions,
    "edge": webdriver.EdgeOptions,
    "firefox": webdriver.FirefoxOptions,
    "safari": webdriver.SafariOptions,
}


def config_id(cfg):
    return f"{cfg['browser']}-{cfg['version']}-{cfg['platform']}"


@pytest.hookimpl(hookwrapper=True)
def pytest_runtest_makereport(item, call):
    outcome = yield
    report = outcome.get_result()
    if report.when == "call":
        item.passed = report.passed


@pytest.fixture(params=MATRIX, ids=config_id)
def driver(request):
    cfg = request.param
    options = OPTIONS[cfg["browser"]]()
    options.browser_version = cfg["version"]
    options.platform_name = cfg["platform"]
    options.set_capability("tb:options", {
        "key": os.environ["TB_KEY"],
        "secret": os.environ["TB_SECRET"],
        "name": request.node.name,
        "build": os.environ.get("TB_BUILD", "local"),
    })
    driver = webdriver.Remote("https://hub.testingbot.com/wd/hub", options=options)
    print(f"\n{request.node.name}: session {driver.session_id}")
    yield driver
    passed = getattr(request.node, "passed", False)
    driver.execute_script(f"tb:test-result={'passed' if passed else 'failed'}")
    driver.quit()

What each part does:

  • ids=config_id gives every run a readable name such as test_login[safari-latest-TAHOE], which is also the session name in the dashboard.
  • The pytest_runtest_makereport hook records whether the test body passed, so the fixture can report it.
  • tb:test-result=passed is a special execute_script command that sets the session's status on TestingBot. Without it, the dashboard knows a session ran but not whether your assertions passed. The test status documentation shows the alternatives, including the REST API.
  • TB_BUILD groups every session of one run under one build name, so a CI run is one entry in the dashboard.

Add pythonpath = . under [pytest] in pytest.ini so the tests can import matrix.

The test itself knows nothing about browsers#

from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait

LOGIN = "https://the-internet.herokuapp.com/login"


def test_login(driver):
    driver.get(LOGIN)
    driver.find_element(By.ID, "username").send_keys("tomsmith")
    driver.find_element(By.ID, "password").send_keys("SuperSecretPassword!")
    driver.find_element(By.CSS_SELECTOR, "button[type='submit']").click()

    flash = WebDriverWait(driver, 10).until(
        EC.visibility_of_element_located((By.ID, "flash")))
    assert "You logged into a secure area!" in flash.text

Run it:

export TB_KEY=your-key TB_SECRET=your-secret TB_BUILD=cross-browser-ch4
pytest -v
tests/test_login.py::test_login[chrome-latest-WIN11] PASSED
tests/test_login.py::test_login[edge-latest-WIN11] PASSED
tests/test_login.py::test_login[firefox-latest-WIN11] PASSED
tests/test_login.py::test_login[safari-latest-TAHOE] PASSED
tests/test_login.py::test_login[chrome-latest-1-SEQUOIA] PASSED

One test, five browsers, and five sessions in the cross-browser-ch4 build on TestingBot, each with a video, logs and a pass or fail mark. Add a second test file and it runs on all five too.

All five sessions show up in the TestingBot dashboard, but none is marked passed or failed. What is missing?

The grid sees WebDriver commands, not your assertions. Only an explicit report (the tb:test-result command or the REST API) tells it whether the test passed. The build name groups sessions but does not mark them.

Checking what you actually got#

latest is resolved when the session starts. To log the exact version, read it back from the session:

print(driver.capabilities["browserName"], driver.capabilities["browserVersion"])

In our run, latest resolved to Chrome 156 on Windows 11 and latest-1 to Chrome 155 on macOS Sequoia. Logging this in your test output saves a lot of guessing when a failure appears in "latest" one morning and not the day before: the browser changed, not your code.

Running a subset#

The ids make it easy to run part of the matrix. -k matches on them:

pytest -k "safari or chrome-latest-WIN11"

That is the quickest way to reproduce a failure in one browser while you fix it, without paying for the other four sessions.

Summary#

  • Each browser has an options class that sets browserName and carries its own settings in a vendor-prefixed key; defaults such as acceptInsecureCerts differ.
  • browser_version (including latest-1) and platform_name pick the exact configuration; grid settings go in tb:options.
  • Keep the matrix as data and parametrize one fixture over it; tests never mention a browser.
  • Report pass or fail explicitly, name the build, and log the resolved browser version.