Selenium Across Browsers and Versions
On this page
Selenium WebDriver is a W3C standard, and every major browser ships a driver for it: chromedriver, msedgedriver, geckodriver for Firefox and safaridriver for Safari. That is its great strength for cross-browser testing: the same test talks to all of them through one protocol. This chapter takes one Python test and runs it on five browser, version and operating system combinations, with a fixture you can reuse for any suite.
One options class per browser#
In Selenium 4 you describe the browser you want with an options object. Each browser has its own class, and each class does two things: it sets browserName, and it gives you a place for that browser's own settings, which travel in a vendor-prefixed key.
from selenium import webdriver
chrome = webdriver.ChromeOptions()
chrome.add_argument("--lang=de-DE")
firefox = webdriver.FirefoxOptions()
firefox.set_preference("intl.accept_languages", "de-DE")
safari = webdriver.SafariOptions()
for options in (chrome, firefox, safari):
options.browser_version = "latest"
print(options.to_capabilities())
The output (abridged) shows where everything goes:
{'browserName': 'chrome', 'browserVersion': 'latest', 'goog:chromeOptions': {'args': ['--lang=de-DE'], ...}}
{'browserName': 'firefox', 'acceptInsecureCerts': True, 'browserVersion': 'latest', 'moz:firefoxOptions': {'prefs': {'intl.accept_languages': 'de-DE', ...}}}
{'browserName': 'safari', 'platformName': 'mac', 'browserVersion': 'latest'}
Three things to notice:
- The same intent needs different code per browser. Setting the language is a command-line argument in Chrome and a preference in Firefox. Browser-specific settings are one of the few places where a cross-browser suite legitimately branches on the browser.
-
Defaults differ.
FirefoxOptionssetsacceptInsecureCertstoTrue,SafariOptionssetsplatformNametomac, andChromeOptionssets neither. If a test against a self-signed staging certificate passes in Firefox and fails in Chrome, this is why. -
EdgeOptionsworks likeChromeOptions, because Edge is Chromium-based; its settings travel underms:edgeOptions.
Your staging site uses a self-signed certificate. The suite passes in Firefox and fails in Chrome with a certificate error. What is the cleanest fix?
FirefoxOptions accepts insecure certificates by default and the other classes do not. Setting the standard acceptInsecureCerts capability explicitly removes the difference for every browser; a Chrome-only flag fixes one browser and leaves Safari and Edge failing.
Versions and platforms#
Two more standard capabilities pick the exact configuration:
-
browser_version: a version number such as"155", or a relative one. On TestingBot,"latest"is the newest release,"latest-1"the one before it, and so on. Relative versions keep your matrix current without edits. -
platform_name: the operating system. On TestingBot these are names such asWIN11,WIN10,TAHOE,SEQUOIA,SONOMAandLINUX. The browser list shows every combination.
Everything that is specific to the grid goes in one vendor key, tb:options: your credentials, a test name, a build name and options such as screen resolution. The Selenium options reference lists them all.
The matrix as data#
Keep the matrix out of the tests. A plain list of dictionaries in matrix.py is enough, and it is the list chapter 2 produced:
# One entry per browser/OS combination the suite must pass on.
MATRIX = [
{"browser": "chrome", "version": "latest", "platform": "WIN11"},
{"browser": "edge", "version": "latest", "platform": "WIN11"},
{"browser": "firefox", "version": "latest", "platform": "WIN11"},
{"browser": "safari", "version": "latest", "platform": "TAHOE"},
{"browser": "chrome", "version": "latest-1", "platform": "SEQUOIA"},
]
A parametrized pytest fixture#
A pytest fixture with params runs every test that uses it once per entry. Put this in tests/conftest.py:
import os
import pytest
from selenium import webdriver
from matrix import MATRIX
OPTIONS = {
"chrome": webdriver.ChromeOptions,
"edge": webdriver.EdgeOptions,
"firefox": webdriver.FirefoxOptions,
"safari": webdriver.SafariOptions,
}
def config_id(cfg):
return f"{cfg['browser']}-{cfg['version']}-{cfg['platform']}"
@pytest.hookimpl(hookwrapper=True)
def pytest_runtest_makereport(item, call):
outcome = yield
report = outcome.get_result()
if report.when == "call":
item.passed = report.passed
@pytest.fixture(params=MATRIX, ids=config_id)
def driver(request):
cfg = request.param
options = OPTIONS[cfg["browser"]]()
options.browser_version = cfg["version"]
options.platform_name = cfg["platform"]
options.set_capability("tb:options", {
"key": os.environ["TB_KEY"],
"secret": os.environ["TB_SECRET"],
"name": request.node.name,
"build": os.environ.get("TB_BUILD", "local"),
})
driver = webdriver.Remote("https://hub.testingbot.com/wd/hub", options=options)
print(f"\n{request.node.name}: session {driver.session_id}")
yield driver
passed = getattr(request.node, "passed", False)
driver.execute_script(f"tb:test-result={'passed' if passed else 'failed'}")
driver.quit()
What each part does:
-
ids=config_idgives every run a readable name such astest_login[safari-latest-TAHOE], which is also the session name in the dashboard. - The
pytest_runtest_makereporthook records whether the test body passed, so the fixture can report it. -
tb:test-result=passedis a specialexecute_scriptcommand that sets the session's status on TestingBot. Without it, the dashboard knows a session ran but not whether your assertions passed. The test status documentation shows the alternatives, including the REST API. -
TB_BUILDgroups every session of one run under one build name, so a CI run is one entry in the dashboard.
Add pythonpath = . under [pytest] in pytest.ini so the tests can import matrix.
The test itself knows nothing about browsers#
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
LOGIN = "https://the-internet.herokuapp.com/login"
def test_login(driver):
driver.get(LOGIN)
driver.find_element(By.ID, "username").send_keys("tomsmith")
driver.find_element(By.ID, "password").send_keys("SuperSecretPassword!")
driver.find_element(By.CSS_SELECTOR, "button[type='submit']").click()
flash = WebDriverWait(driver, 10).until(
EC.visibility_of_element_located((By.ID, "flash")))
assert "You logged into a secure area!" in flash.text
Run it:
export TB_KEY=your-key TB_SECRET=your-secret TB_BUILD=cross-browser-ch4
pytest -v
tests/test_login.py::test_login[chrome-latest-WIN11] PASSED
tests/test_login.py::test_login[edge-latest-WIN11] PASSED
tests/test_login.py::test_login[firefox-latest-WIN11] PASSED
tests/test_login.py::test_login[safari-latest-TAHOE] PASSED
tests/test_login.py::test_login[chrome-latest-1-SEQUOIA] PASSED
One test, five browsers, and five sessions in the cross-browser-ch4 build on TestingBot, each with a video, logs and a pass or fail mark. Add a second test file and it runs on all five too.
All five sessions show up in the TestingBot dashboard, but none is marked passed or failed. What is missing?
The grid sees WebDriver commands, not your assertions. Only an explicit report (the tb:test-result command or the REST API) tells it whether the test passed. The build name groups sessions but does not mark them.
Checking what you actually got#
latest is resolved when the session starts. To log the exact version, read it back from the session:
print(driver.capabilities["browserName"], driver.capabilities["browserVersion"])
In our run, latest resolved to Chrome 156 on Windows 11 and latest-1 to Chrome 155 on macOS Sequoia. Logging this in your test output saves a lot of guessing when a failure appears in "latest" one morning and not the day before: the browser changed, not your code.
Running a subset#
The ids make it easy to run part of the matrix. -k matches on them:
pytest -k "safari or chrome-latest-WIN11"
That is the quickest way to reproduce a failure in one browser while you fix it, without paying for the other four sessions.
Summary#
- Each browser has an options class that sets
browserNameand carries its own settings in a vendor-prefixed key; defaults such asacceptInsecureCertsdiffer. -
browser_version(includinglatest-1) andplatform_namepick the exact configuration; grid settings go intb:options. - Keep the matrix as data and parametrize one fixture over it; tests never mention a browser.
- Report pass or fail explicitly, name the build, and log the resolved browser version.