Run Selenium IDE Tests on the TestingBot Grid
On this page
On your laptop, selenium-side-runner uses the browsers you have installed. A cloud grid gives you browsers and operating systems you do not have, plus a video, screenshots and logs of every run. This chapter runs the course project on TestingBot in Chrome and Firefox on Windows 11, groups the sessions in one build, and makes each session show passed in the dashboard, which takes one extra step that most guides leave out.
What you need#
- A TestingBot account (the free trial is enough) and your key and secret from the dashboard's account page.
-
selenium-side-runnerfrom chapter 6. - The course project,
the-internet.side.
Keep the key and secret out of the .side file and out of .side.yml. Set them as environment variables in your shell or CI system:
export TB_KEY=your-key
export TB_SECRET=your-secret
Point the runner at the grid#
The --server option sends the browsers to a remote Selenium endpoint instead of starting them locally. For TestingBot that is the hub URL, with your key and secret in it:
selenium-side-runner --server "https://$TB_KEY:$TB_SECRET@hub.testingbot.com/wd/hub" the-internet.side
The capabilities say which browser and operating system you want. Put them in .side.yml next to the project:
capabilities:
browserName: chrome
browserVersion: latest
platformName: WIN11
"tb:options":
build: certificate-selenium-ide
maxWorkers: 1
-
browserName,browserVersion,platformNameare standard WebDriver capabilities.latestmeans the newest version TestingBot has;latest-1the one before. The capabilities generator lists every combination. -
tb:optionsholds TestingBot's own settings.buildgroups all sessions of this run under one name in the dashboard; test options lists the others, such as screen resolution or turning off video. -
maxWorkers: 1runs one test at a time. Raise it only up to the number of parallel sessions your plan allows; the section on concurrency below shows what happens otherwise.
The quotes around "tb:options" are needed because the key contains a colon.
Mark sessions as passed#
Run the command above as it is, and every session shows up in the dashboard without a result. WebDriver has no standard way to say "this test passed", and selenium-side-runner does not report its results to the grid. The dashboard only knows that a browser was opened and closed.
TestingBot accepts the result from inside the session: when a test calls executeScript with the text tb:test-result=passed, the grid records the result instead of sending the script to the browser. tb:test-name=... sets the name shown in the dashboard. Both are described under annotating tests.
There is a catch. tb:test-result=passed is not valid JavaScript. On TestingBot the grid intercepts it before it reaches the browser, but in the IDE or in a local browser the browser tries to run it and the test fails. So the course project ends every test with a guarded block, using the environment variable trick from chapter 4:
| Command | Target |
|---|---|
if |
${env:TB_REPORT} === "yes" |
executeScript |
tb:test-name=Log in with valid credentials |
executeScript |
tb:test-result=passed |
end |
TB_REPORT is only set in the shell that runs the tests on the grid. In local runs without it, the condition was false and the block was skipped in all eight tests. The IDE reads ${env:...} from its own process, where the variable is not set either.
Why this works as a pass signal: the block is the last step. A failed assert or any error stops the test before it gets there, so only tests that reached the end are marked passed. Keep the two limits in mind:
- A failed test gets no result, not "failed". The runner's output and exit code still tell you which tests failed; the dashboard shows those sessions without a status.
-
A failed
verifydoes not stop the test, so the block still runs and marks it passed. Chapter 3 is why tests that report their own result should decide pass or fail withassertcommands.
The test name has to be typed into the block: executeScript passes variables as script arguments (chapter 4), so tb:test-name=${name} would not work.
You add executeScript | tb:test-result=passed as the last step of a test, without the if block. What happens?
Only the grid understands the tb: command. The if on an environment variable that is set only for grid runs keeps the test green everywhere else.
Run it in two browsers#
With .side.yml in place, run the project once per browser. A -c on the command line is merged over the file, so only the browser changes and the build name stays:
export TB_REPORT=yes
selenium-side-runner --server "https://$TB_KEY:$TB_SECRET@hub.testingbot.com/wd/hub" the-internet.side
selenium-side-runner --server "https://$TB_KEY:$TB_SECRET@hub.testingbot.com/wd/hub" -c "browserName=firefox" the-internet.side
Our runs on 13 November 2026, Chrome 156 and then Firefox 158 on Windows 11:
Running suite Smoke
✓ Running test Log in with valid credentials (7461 ms)
✓ Running test Wrong password shows an error (8025 ms)
✓ Running test Check both checkboxes (7146 ms)
✓ Running test Pick an option from the dropdown (11635 ms)
✓ Running test Wait for content that loads later (14501 ms)
✓ Running test Log in and log out (12213 ms)
Running suite Variables and control flow
✓ Running test Store and reuse values (9124 ms)
✓ Running test Add and remove elements (12859 ms)
Tests: 8 passed, 8 total
Time: 83.223 s
Firefox took 115 seconds for the same eight tests. Each test took longer than locally because it starts its own browser session on a remote machine: one session per test (the shared Log in test runs inside its caller's session). That is 16 sessions in the certificate-selenium-ide build, each with its name and a passed status, for example 3e09162b6c48-d3637e31ee47-2697cd09b7db-179455851348-2205533 (Log in with valid credentials, Chrome) and fce450036f3e-41a1bcf8a2de-6f36ff0b13b0-179455860210-09714402 (the same test in Firefox).
Open the build in the TestingBot dashboard to see every session with its video, screenshots and logs. When a test fails only in one browser, that video is usually the fastest way to see why.
Concurrency, queues and a 30-second limit#
This is the part that cost us the most time, and it is specific to selenium-side-runner.
Your TestingBot plan allows a number of sessions at the same time. Requests above that wait in a queue; when the queue is full too, the grid refuses new ones with "You are currently at the limit of the maximum allowed parallel + queued tests for your TestingBot plan". So far, that is how every grid works.
The runner adds a hard limit: if a browser session has not started within 30 seconds, it gives up with:
Driver took too long to build. This is likely an issue with the browser or driver.
The 30 seconds are fixed in @seleniumhq/side-runtime and there is no option to change them. A session that waits in the queue can easily take longer. Worse, the grid does not know the runner gave up: the queued session still starts later, sits idle without a name until it times out (about two minutes in our account), and occupies a slot that your next tests are waiting for. On a shared account we watched this feed on itself: abandoned sessions filled the slots, so the next attempts queued and were abandoned too.
How to avoid it:
-
Keep
maxWorkers(or-w) at or below your plan's parallel sessions, and mark suites Parallel only with that in mind. - Run browsers one after another (as above), not as separate runner processes at the same time.
- Do not retry immediately after "Driver took too long to build". Wait for the abandoned sessions to time out first, or the retry joins the same queue.
- Do not share an account's slots with other jobs while a suite runs, if you can avoid it.
When the account had a free slot, a Windows 11 session started in about six seconds in our measurements; behind a queue it took close to 30, which is why the limit bites exactly when the grid is busy.
Your plan allows two parallel sessions. Which run is least likely to hit "Driver took too long to build"?
Two workers fit the plan, so no session waits in the queue. The other two ask for six or eight sessions at once; the extra ones queue, pass the 30-second start limit, and the retries add more queued sessions.
Without the command line: TestingBot Codeless Automation#
If you want scheduled runs and alerts rather than a CI job, TestingBot can also run .side files itself. Upload the file in the dashboard's Codeless Test area, pick browsers, set a schedule such as every 15 minutes, and choose email or Slack alerts. The codeless automation guide walks through it, and the Selenium IDE page covers both ways of running .side files. This course used selenium-side-runner throughout, so that the same commands work locally, in CI and on the grid.