Exit course
Chapter 4 of 5 Beginner 7 min read 3 quick checks Updated

Your First Cloud Test

On this page
  1. Step 1: Get credentials
  2. Option A: Selenium with Python
  3. Option B: Playwright with JavaScript
  4. Step 2: Look at the result
  5. Step 3: Try more than one browser
  6. Where to go next

Time to run something. In this chapter you will write a short automated test, run it on a browser in the cloud, and look at what comes back: a video, logs and a pass or fail status. There are two versions of the same test, one with Selenium in Python and one with Playwright in JavaScript. Pick whichever matches what your team uses; the concepts are identical.

The test is deliberately small: open a page, check its title, and report the result. The point is to see the moving parts of a cloud session, not to learn a framework. For that, follow up with the Playwright course or the Selenium with TestNG course.

Step 1: Get credentials#

Create a free TestingBot trial account and open your account settings. You will find a key and a secret. They identify your account to the grid, like a username and password for your tests. Never hard-code them in a test or commit them; export them as environment variables:

export TB_KEY=your_key
export TB_SECRET=your_secret

On Windows PowerShell use $env:TB_KEY="your_key" and $env:TB_SECRET="your_secret".

Your test needs the TestingBot key and secret in CI. Where should they come from?

The key and secret work like a username and password. Anything committed to the repository is readable by everyone with access to it, forever; environment variables from a secret store keep them out of the code.

Option A: Selenium with Python#

Install the Selenium bindings (Python 3.10 or newer):

python3 -m pip install selenium

Create first_cloud_test.py:

import os

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

options = Options()
options.browser_version = "latest"
options.platform_name = "WIN11"
options.set_capability("tb:options", {
    "key": os.environ["TB_KEY"],
    "secret": os.environ["TB_SECRET"],
    "name": "My first cloud test",
    "build": "cloud-testing-course",
})

driver = webdriver.Remote(
    command_executor="https://hub.testingbot.com/wd/hub",
    options=options,
)

try:
    driver.get("https://testingbot.com")
    title = driver.title
    print("Page title:", title)
    passed = "TestingBot" in title
    driver.execute_script(f"tb:test-result={'passed' if passed else 'failed'}")
    assert passed, f"Unexpected title: {title}"
finally:
    driver.quit()

Run it:

python3 first_cloud_test.py

What just happened#

Compare this with a local Selenium script and only three things are different:

  1. webdriver.Remote instead of webdriver.Chrome. Instead of starting Chrome on your machine, Selenium sends a "new session" request to https://hub.testingbot.com/wd/hub.
  2. Capabilities describe the machine you want. ChromeOptions sets browserName to Chrome, browser_version = "latest" asks for the newest stable version, and platform_name = "WIN11" asks for Windows 11. Change these to Options from selenium.webdriver.firefox.options and "LINUX" and the same test runs on Firefox on Linux. The Selenium capabilities builder shows the valid values.
  3. tb:options carries TestingBot settings. Your credentials, plus a name and build that label the run in the dashboard.

The execute_script("tb:test-result=passed") line is a TestingBot-specific command that is intercepted by the grid instead of running in the page. The grid cannot know whether your assertion passed; this tells it, so the dashboard shows the same result as your terminal. In a real project you would do this once in a test framework hook rather than in every test.

driver.quit() in the finally block matters more in the cloud than locally. If a test crashes without quitting, the remote browser stays open until an idle timeout, holding one of your parallel slots.

Option B: Playwright with JavaScript#

Playwright connects to a remote browser over a WebSocket instead of using WebDriver. Install the client library without bundled browsers, since the browser runs in the cloud (Node.js 20 or newer):

mkdir first-cloud-test && cd first-cloud-test
npm init -y
npm install playwright-core

Create first-cloud-test.mjs:

import { chromium } from 'playwright-core'

const capabilities = {
  browserName: 'chrome',
  browserVersion: 'latest',
  platform: 'WIN11',
  'tb:options': {
    key: process.env.TB_KEY,
    secret: process.env.TB_SECRET,
    name: 'My first cloud test (Playwright)',
    build: 'cloud-testing-course'
  }
}

const wsEndpoint =
  'wss://cloud.testingbot.com/playwright?capabilities=' +
  encodeURIComponent(JSON.stringify(capabilities))

// TestingBot cannot see your assertions, so the script reports the result itself.
const setStatus = (page, passed, reason) =>
  page.evaluate(() => {}, `testingbot_executor: ${JSON.stringify({
    action: 'setSessionStatus',
    arguments: { passed, reason }
  })}`)

const browser = await chromium.connect({ wsEndpoint })
const page = await browser.newPage()
try {
  await page.goto('https://testingbot.com')
  const title = await page.title()
  console.log('Page title:', title)
  if (!title.includes('TestingBot')) {
    throw new Error(`Unexpected title: ${title}`)
  }
  await setStatus(page, true, 'Title matched')
} catch (err) {
  await setStatus(page, false, err.message)
  process.exitCode = 1
} finally {
  await browser.close()
}

Run it with node first-cloud-test.mjs.

Two lines are cloud-specific. chromium.connect({ wsEndpoint }) replaces chromium.launch(), and setStatus sends the testingbot_executor command that marks the session as passed or failed, the Playwright equivalent of the tb:test-result call in the Selenium version (marking Playwright test status has the details). The capabilities travel in the URL, encoded as JSON. Everything after that is plain Playwright. If you use the Playwright Test runner, the same endpoint goes into use.connectOptions.wsEndpoint in playwright.config.ts, and every project in the config can target a different cloud browser.

A colleague removed the setStatus(page, false, err.message) call from the catch block. The title check now fails. What does the dashboard show?

The error is thrown in your Node process, not in the remote browser, so the grid never sees it. Without the testingbot_executor call the session is simply not marked, and the dashboard and terminal disagree.

Step 2: Look at the result#

Open the TestingBot dashboard. Your session is listed under the build cloud-testing-course with the name you gave it. Open it and you will find:

  • A video of the whole session. Scrub to any point to see what the browser showed.
  • Screenshots taken during the run.
  • Logs: the commands your test sent and the browser's output.
  • The status your script reported, so a failed assertion shows up as a failed test.

Now break the test on purpose: change the expected title to something wrong and run it again. The test fails in your terminal, the session is marked failed, and the video shows the page as it really was. That loop (fail in CI, watch the video, fix) is most of the day-to-day value of a cloud grid.

Step 3: Try more than one browser#

Edit the capabilities and rerun: platform_name = "LINUX" with Firefox options for Selenium, or browserName: 'firefox' with firefox.connect for Playwright. Then imagine fifty tests across five browsers. On one local machine they run one after another; on a grid they can run side by side, limited only by your plan's parallel sessions. Test runners do this for you: pytest -n 5 with the pytest-xdist plugin, or workers in Playwright Test.

Your plan allows 2 parallel sessions and you start the suite with pytest -n 8. What happens to the extra sessions?

Parallelism is capped by your plan's parallel sessions; extra sessions queue. Size the worker count to the plan, otherwise queued tests can even hit start-up timeouts.

Where to go next#