--- title: Run Selenium Python tests on the TestingBot grid description: 'Everything so far ran on your own machine: one operating system, whichever browser versions you have installed, and as much parallelism as your CPU allows. A…' source_url: html: https://testingbot.com/resources/courses/selenium-python/run-on-testingbot md: https://testingbot.com/resources/courses/selenium-python/run-on-testingbot.md --- Course [Selenium with Python and pytest](https://testingbot.com/resources/courses/selenium-python) Chapter 9 of 10 1. [1 Selenium Python Tutorial with pytest](https://testingbot.com/resources/courses/selenium-python) 2. [2 Set up Selenium with Python and pytest](https://testingbot.com/resources/courses/selenium-python/setup-and-first-test) 3. [3 Selenium locators in Python: the By API](https://testingbot.com/resources/courses/selenium-python/locators) 4. [4 Selenium waits in Python: WebDriverWait and time.sleep](https://testingbot.com/resources/courses/selenium-python/waits) 5. [5 pytest fixtures for Selenium: conftest.py and scope](https://testingbot.com/resources/courses/selenium-python/pytest-fixtures) 6. [6 Assertions, screenshots on failure and test reports](https://testingbot.com/resources/courses/selenium-python/assertions-screenshots-reports) 7. [7 Page objects in Python](https://testingbot.com/resources/courses/selenium-python/page-objects) 8. [8 Run Selenium tests in parallel with pytest-xdist](https://testingbot.com/resources/courses/selenium-python/parallel-with-pytest-xdist) 9. [9 Run Selenium Python tests on the TestingBot grid](https://testingbot.com/resources/courses/selenium-python/run-on-testingbot) 10. [10 Selenium Python Quiz](https://testingbot.com/resources/courses/selenium-python/knowledge-check) [Exit course](https://testingbot.com/resources/courses) Chapter 9 of 10 Intermediate 10 min read 2 quick checks Updated Nov 2026 # Run Selenium Python tests on the TestingBot grid On this page 1. [What changes and what doesn't](https://testingbot.com#what-changes-and-what-doesn-t) 2. [Capabilities and tb:options](https://testingbot.com#capabilities-and-tb-options) 3. [The final conftest.py](https://testingbot.com#the-final-conftest-py) 4. [Reporting pass or fail](https://testingbot.com#reporting-pass-or-fail) 5. [Run it](https://testingbot.com#run-it) 6. [Parallel limits](https://testingbot.com#parallel-limits) 7. [Testing a site that is not public](https://testingbot.com#testing-a-site-that-is-not-public) 8. [Continuous integration with GitHub Actions](https://testingbot.com#continuous-integration-with-github-actions) 9. [Debugging a remote failure](https://testingbot.com#debugging-a-remote-failure) Everything so far ran on your own machine: one operating system, whichever browser versions you have installed, and as much parallelism as your CPU allows. A Selenium grid runs the browsers on remote machines instead. Your tests stay exactly the same; only the fixture changes, from `webdriver.Chrome()` to `webdriver.Remote(grid_url, options=...)`. This chapter makes that switch behind a `--grid` flag, using the TestingBot grid. The same `webdriver.Remote` code works with a self-hosted Selenium Grid; only the URL and the vendor-specific options differ. Everything in this chapter was run on TestingBot: the final suite ran as 21 sessions (7 tests on Chrome and Firefox on Windows 11 and Safari on macOS Sonoma), all reported as passed, plus deliberate failures to check that failed status and screenshots come through. ## What changes and what doesn't[#](https://testingbot.com#what-changes-and-what-doesn-t) | Stays the same | Changes | | --- | --- | | Tests, page objects, locators, waits, assertions | How the driver is created | | `--browser`, parametrisation, markers | The browser and OS come from capabilities | | pytest-xdist | `-n` is bounded by your account's parallel limit | | Screenshot on failure | You also get a video and logs of every session | Because chapter 4 put driver creation in the fixture, the switch touches only `conftest.py`. ## Capabilities and tb:options[#](https://testingbot.com#capabilities-and-tb-options) A remote session is requested with the same options classes you use locally (`ChromeOptions`, `FirefoxOptions`, `EdgeOptions`, `SafariOptions`) plus two standard W3C capabilities, which the options classes expose as properties: - `options.browser_version`, such as `"latest"` or a specific major version; - `options.platform_name`, the operating system. On TestingBot these are names such as `WIN11` for Windows 11 or `SONOMA` for macOS Sonoma. Vendor settings go in a namespaced capability. TestingBot's is `tb:options`, which carries your key and secret, `name` (the test name shown in the dashboard) and `build` (which groups the tests of one run). The [capabilities generator](https://testingbot.com/support/web-automate/selenium/capabilities) produces the exact values for any browser and OS, and [test options](https://testingbot.com/support/web-automate/selenium/test-options) lists everything else `tb:options` accepts, such as screen resolution, time zone, or turning off video recording, which is on by default. ## The final conftest.py[#](https://testingbot.com#the-final-conftest-py) ```python import os import re from pathlib import Path import pytest from selenium import webdriver GRID_URL = "https://hub.testingbot.com/wd/hub" # browser name -> (options class, TestingBot platform) BROWSERS = { "chrome": (webdriver.ChromeOptions, "WIN11"), "firefox": (webdriver.FirefoxOptions, "WIN11"), "edge": (webdriver.EdgeOptions, "WIN11"), "safari": (webdriver.SafariOptions, "SONOMA"), } def pytest_addoption(parser): parser.addoption("--browser", action="append", choices=sorted(BROWSERS), help="browser to run on; repeat for several (default: chrome)") parser.addoption("--headless", action="store_true", help="run local browsers headless") parser.addoption("--grid", action="store_true", help="run on the TestingBot grid") def pytest_generate_tests(metafunc): # Every test that uses the driver runs once per --browser value. if "browser" in metafunc.fixturenames: browsers = metafunc.config.getoption("browser") or ["chrome"] metafunc.parametrize("browser", browsers) @pytest.hookimpl(wrapper=True, tryfirst=True) def pytest_runtest_makereport(item, call): report = yield # Remember the outcome of each phase so fixtures can read it during teardown. setattr(item, "rep_" + report.when, report) return report def local_driver(browser, headless): if browser == "firefox": options = webdriver.FirefoxOptions() if headless: options.add_argument("-headless") return webdriver.Firefox(options=options) if browser == "chrome": options = webdriver.ChromeOptions() if headless: options.add_argument("--headless=new") options.add_argument("--window-size=1920,1080") return webdriver.Chrome(options=options) raise pytest.UsageError(f"{browser} is only configured for --grid runs") def remote_driver(browser, test_name): options_class, platform = BROWSERS[browser] options = options_class() options.browser_version = "latest" options.platform_name = platform options.set_capability("tb:options", { "key": os.environ["TB_KEY"], "secret": os.environ["TB_SECRET"], "name": test_name, "build": os.environ.get("TB_BUILD", "local"), }) return webdriver.Remote(command_executor=GRID_URL, options=options) @pytest.fixture def driver(request, browser): config = request.config if config.getoption("grid"): driver = remote_driver(browser, request.node.name) else: driver = local_driver(browser, config.getoption("headless")) yield driver call = getattr(request.node, "rep_call", None) failed = call is None or call.failed try: if failed: save_screenshot(driver, request.node.nodeid) if config.getoption("grid"): status = "failed" if failed else "passed" driver.execute_script(f"tb:test-result={status}") print(f"TestingBot session {driver.session_id}: {status}") finally: driver.quit() def save_screenshot(driver, nodeid): folder = Path("artifacts") folder.mkdir(exist_ok=True) worker = os.environ.get("PYTEST_XDIST_WORKER", "main") name = re.sub(r"[^\w.-]+", "_", nodeid) path = folder / f"{name}-{worker}.png" driver.save_screenshot(str(path)) print(f"screenshot: {path}") ``` What is new compared with chapter 5: - **`--grid`** chooses between `local_driver` and `remote_driver`. Without it, nothing about the local workflow changes. - **`BROWSERS`** maps the `--browser` names to an options class and a platform. Safari only exists on macOS, so it gets `SONOMA`; the others run on Windows 11. Edge and Safari are only configured for grid runs, and asking for them locally fails with a clear `UsageError` rather than silently starting Chrome. - **Credentials come from the `TB_KEY` and `TB_SECRET` environment variables** , never from code or `pytest.ini`. Find both in the TestingBot member area; in CI, store them as secrets. `os.environ[...]` (rather than `.get`) fails loudly when they are missing. - **`name`** is the pytest test name including its parameters, such as `test_dropdown_selection_is_submitted[firefox]`, so each dashboard entry maps to one line of your pytest output. - **`build`** comes from `TB_BUILD`, so all sessions from one run are grouped together. ## Reporting pass or fail[#](https://testingbot.com#reporting-pass-or-fail) The grid only sees WebDriver commands. It cannot know that your `assert` failed; from its side the session simply ended. Unless you tell it, the dashboard cannot show which tests passed. The fixture reports the outcome during teardown, before `quit()`, through the JavaScript executor: ```python driver.execute_script("tb:test-result=passed") # or failed ``` TestingBot recognises `tb:` commands and handles them on the grid side instead of running them as page JavaScript. The same mechanism accepts `tb:test-name=...` and others, all listed with examples in [annotating tests](https://testingbot.com/support/web-automate/selenium/annotating-tests). It needs no extra package, but it must run while the session is open. The alternative is the REST API, through the `testingbotclient` package from `requirements.txt`. Use it when you want to report after the session has closed, or from outside the test process: ```python from testingbotclient import TestingBotClient @pytest.fixture def driver(request): ... # create the remote driver as above session_id = driver.session_id yield driver driver.quit() call = getattr(request.node, "rep_call", None) TestingBotClient().tests.update_test(session_id, passed=bool(call and call.passed)) ``` `TestingBotClient()` with no arguments reads the same `TB_KEY` and `TB_SECRET` variables. The [pytest guide](https://testingbot.com/support/web-automate/selenium/python/pytest#api) shows the other fields `update_test` accepts. Pick one of the two mechanisms; using both is harmless but redundant. The TestingBot dashboard shows no pass/fail result for any test, although pytest reports failures correctly. The fixture calls `driver.quit()` and then `driver.execute_script("tb:test-result=failed")`. What is wrong? The status command runs after the session has closed; it must run before `quit()`. `tb:test-result` only works with Chrome. The grid reads the result from pytest's exit code instead. The executor command travels over the WebDriver session, so it needs that session to be alive. After `quit()` it raises an error instead of reporting. To report after closing, use the REST API with the saved session ID. ## Run it[#](https://testingbot.com#run-it) ```bash export TB_KEY=your-key export TB_SECRET=your-secret export TB_BUILD="course-$(date +%Y%m%d-%H%M)" pytest --grid --browser chrome --browser firefox --browser safari -n 2 ``` ``` 21 passed in 129.10s (0:02:09) ``` Seven tests times three browsers, two at a time. Open the build in the TestingBot dashboard to see every session with its name, status, video and logs. ## Parallel limits[#](https://testingbot.com#parallel-limits) Every grid caps how many sessions can run at once. On TestingBot the limit comes with your plan, and according to the docs, sessions beyond it are queued for up to 6 minutes and start as soon as a slot is free. The queue has a cap too. When we ran this suite with `-n 6` on an account whose plan allows 2 parallel and 4 queued tests, while other jobs were using the same account, some sessions were refused outright: ``` selenium.common.exceptions.WebDriverException: Message: You are currently at the limit of the maximum allowed parallel + queued tests for your TestingBot plan. Your plan allows 2 parallel tests and 4 pending tests. ``` Set `-n` to roughly your parallel limit, and lower it if other pipelines share the account. A higher number does not make the run faster; it only adds queue time and, past the queue cap, errors. Your plan allows 5 parallel sessions. To finish faster, a teammate changes CI to `pytest --grid -n 20`. What should you expect? Four times faster runs, since the grid scales automatically. The same speed and no other effect. No real speed-up: at most 5 sessions run at once, the rest wait in the queue, and requests beyond the queue limit fail to start. The parallel limit, not the worker count, decides how many browsers run. Extra workers only wait, and if parallel plus queued requests exceed what the plan allows, session creation is refused, which shows up as errors in pytest. ## Testing a site that is not public[#](https://testingbot.com#testing-a-site-that-is-not-public) A cloud browser cannot reach `localhost:8000` or a staging server behind your firewall. The TestingBot tunnel solves this: start it on a machine that can reach the site, then point `webdriver.Remote` at the tunnel instead of the public hub, by default `http://localhost:4445/wd/hub` when the tunnel runs on the same machine as the tests. The [pytest guide](https://testingbot.com/support/web-automate/selenium/python/pytest) shows the exact options and the [tunnel documentation](https://testingbot.com/support/tunnel) covers installation; the [local website testing course](https://testingbot.com/resources/courses/local-website-testing) goes into it in depth. ## Continuous integration with GitHub Actions[#](https://testingbot.com#continuous-integration-with-github-actions) Store `TB_KEY` and `TB_SECRET` as repository secrets (Settings, Secrets and variables, Actions), then add `.github/workflows/ui-tests.yml`: ```yaml name: UI tests on: [push, pull_request] jobs: selenium: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-python@v5 with: python-version: "3.13" cache: pip - run: pip install -r requirements.txt - name: Run the suite on TestingBot env: TB_KEY: ${{ secrets.TB_KEY }} TB_SECRET: ${{ secrets.TB_SECRET }} TB_BUILD: github-${{ github.run_id }}-${{ github.run_attempt }} run: > pytest --grid --browser chrome --browser firefox -n 2 --junitxml=junit.xml --html=report.html --self-contained-html - uses: actions/upload-artifact@v4 if: always() with: name: test-results path: | junit.xml report.html artifacts/ ``` `TB_BUILD` built from the run ID and attempt groups every session of one workflow run into one TestingBot build, and a re-run gets a new one. `if: always()` uploads the reports and screenshots even when tests fail, which is exactly when you need them. The [GitHub Actions guide](https://testingbot.com/support/integrations/ci-cd/github-actions) adds the tunnel action for private sites and pull request status checks. ## Debugging a remote failure[#](https://testingbot.com#debugging-a-remote-failure) Remote failures feel harder because you cannot watch the browser. In practice you have more evidence than locally: - **The screenshot fixture still works.** `save_screenshot` is a WebDriver command, so it captures the remote browser. - **The session ID** is printed by the fixture on every grid run, for example `TestingBot session 1014ca63...: failed`, and appears in the failure output next to the screenshot path. It ties a pytest failure to the exact session in the dashboard. - **The video and logs** of that session show what the browser did around the failing command. Most "only fails on the grid" problems turn out to be a slower page load that an explicit wait would have covered, a window size different from your laptop's, or a genuine browser difference you would never have seen running Chrome alone, which is the reason to run on several browsers in the first place. [Previous · Chapter 8 Run Selenium tests in parallel with pytest-xdist](https://testingbot.com/resources/courses/selenium-python/parallel-with-pytest-xdist) [Next · Chapter 10 Selenium Python Quiz](https://testingbot.com/resources/courses/selenium-python/knowledge-check) On this page 1. [What changes and what doesn't](https://testingbot.com#what-changes-and-what-doesn-t) 2. [Capabilities and tb:options](https://testingbot.com#capabilities-and-tb-options) 3. [The final conftest.py](https://testingbot.com#the-final-conftest-py) 4. [Reporting pass or fail](https://testingbot.com#reporting-pass-or-fail) 5. [Run it](https://testingbot.com#run-it) 6. [Parallel limits](https://testingbot.com#parallel-limits) 7. [Testing a site that is not public](https://testingbot.com#testing-a-site-that-is-not-public) 8. [Continuous integration with GitHub Actions](https://testingbot.com#continuous-integration-with-github-actions) 9. [Debugging a remote failure](https://testingbot.com#debugging-a-remote-failure) Certification This course has a free, hands-on certificate. [How it works](https://testingbot.com/resources/courses/selenium-python/certification)