Name: Playwright Tutorial: Zero to CI
Availability: InStock

Everything so far ran browsers on your own machine or a Linux CI runner. That leaves gaps: branded Chrome and Edge on Windows, WebKit on macOS, and a ceiling on parallelism set by your runner's CPU. Playwright has a built-in way to close them without touching the tests: instead of launching a local browser, each worker connects to a browser running somewhere else over a WebSocket.

This chapter points the course suite at TestingBot's grid. The same mechanism works with any Playwright-compatible remote endpoint, so the concepts transfer.

How remote Playwright works#

Locally, Playwright Test launches a browser per worker. If a project sets connectOptions.wsEndpoint, the runner calls browserType.connect() on that URL instead. Everything after that, contexts, pages, locators, traces, storageState, is unchanged, because the Playwright client in your test process still drives the browser over the Playwright protocol.

Two consequences:

The project's browserName (chromium, firefox or webkit) still decides which Playwright engine connects, so it must match the remote browser.
Each worker opens its own remote session, and Playwright reuses that browser for the tests the worker runs, with a fresh context per test as usual.

The TestingBot endpoint#

TestingBot's Playwright endpoint is wss://cloud.testingbot.com/playwright. You describe the browser you want with a capabilities query parameter containing URL-encoded JSON:

const capabilities = {
  browserName: 'chrome',      // chrome, edge, firefox or safari
  browserVersion: 'latest',   // also latest-1, latest-2, ...
  platform: 'WIN11',          // e.g. WIN11, WIN10, LINUX, TAHOE, SEQUOIA
  'tb:options': {
    key: process.env.TB_KEY,
    secret: process.env.TB_SECRET,
  },
};
const wsEndpoint =
  `wss://cloud.testingbot.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}`;

chrome and edge connect with the chromium engine, firefox with firefox, and safari with webkit. Your key and secret come from the TestingBot member area; keep them in environment variables. The Playwright capabilities generator builds this object for any browser and platform combination, and the Playwright options reference lists every option.

By default TestingBot uses the same Playwright version as the client running your test, so you do not need to keep two versions in sync. If you do need a different server-side version, there is a playwrightVersion option under 'playwright:options'.

A helper for the endpoint#

Create testingbot.ts in the project root:

type TestingBotBrowser = {
  browserName: 'chrome' | 'edge' | 'firefox' | 'safari';
  browserVersion?: string;
  platform?: string;
};

export function testingbotEndpoint(browser: TestingBotBrowser): string {
  const capabilities = {
    ...browser,
    'tb:options': {
      key: process.env.TB_KEY,
      secret: process.env.TB_SECRET,
      build: process.env.GITHUB_RUN_ID ? `ci-${process.env.GITHUB_RUN_ID}` : 'local',
      ...(process.env.TB_TUNNEL ? { tunnelIdentifier: process.env.TB_TUNNEL } : {}),
    },
  };
  return `wss://cloud.testingbot.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}`;
}

build groups every session from one run together in the dashboard. tunnelIdentifier is covered below.

A separate config for the grid#

Keep the local config as it is and add playwright.testingbot.config.ts, which reuses it and replaces the projects:

import { defineConfig } from '@playwright/test';
import baseConfig from './playwright.config';
import { testingbotEndpoint } from './testingbot';

const authFile = 'playwright/.auth/user.json';

export default defineConfig({
  ...baseConfig,
  workers: 4,
  projects: [
    {
      name: 'setup',
      testMatch: /.*\.setup\.ts/,
      use: {
        connectOptions: {
          wsEndpoint: testingbotEndpoint({ browserName: 'chrome', browserVersion: 'latest', platform: 'LINUX' }),
        },
      },
    },
    {
      name: 'chrome-windows',
      dependencies: ['setup'],
      use: {
        browserName: 'chromium',
        storageState: authFile,
        connectOptions: {
          wsEndpoint: testingbotEndpoint({ browserName: 'chrome', browserVersion: 'latest', platform: 'WIN11' }),
        },
      },
    },
    {
      name: 'edge-windows',
      dependencies: ['setup'],
      use: {
        browserName: 'chromium',
        storageState: authFile,
        connectOptions: {
          wsEndpoint: testingbotEndpoint({ browserName: 'edge', browserVersion: 'latest', platform: 'WIN10' }),
        },
      },
    },
    {
      name: 'firefox-linux',
      dependencies: ['setup'],
      use: {
        browserName: 'firefox',
        storageState: authFile,
        connectOptions: {
          wsEndpoint: testingbotEndpoint({ browserName: 'firefox', browserVersion: 'latest', platform: 'LINUX' }),
        },
      },
    },
    {
      name: 'safari-macos',
      dependencies: ['setup'],
      use: {
        browserName: 'webkit',
        storageState: authFile,
        connectOptions: {
          wsEndpoint: testingbotEndpoint({ browserName: 'safari', browserVersion: 'latest', platform: 'SEQUOIA' }),
        },
      },
    },
  ],
});

Run it:

export TB_KEY=your-key TB_SECRET=your-secret
npx playwright test -c playwright.testingbot.config.ts

The setup project from chapter 4 logs in on a remote browser and writes user.json to your local disk, because storageState() returns the data to the client. The browser projects then load it exactly as before. Your HTML report and traces are produced locally as usual.

workers now controls how many remote sessions run at once. How many you can run in parallel depends on your TestingBot plan; if you open more connections than your plan allows, the extra connections are queued until a slot frees up rather than failing.

You set workers: 8 but your TestingBot plan allows four parallel sessions. What happens?

The extra connections are queued until a slot frees up Four workers fail with a connection error Playwright lowers workers to four automatically

Connections beyond your plan's limit wait in a queue rather than failing. They do take wall-clock time, so more workers than your plan allows does not make the run faster.

Reporting results to the dashboard#

Every session appears in the TestingBot dashboard with a video and browser logs. TestingBot cannot know whether your assertions passed, so you tell it through a special page.evaluate call. Add an automatic fixture to tests/fixtures.ts from chapter 4:

function executor(action: string, args: Record<string, unknown>): string {
  return `testingbot_executor: ${JSON.stringify({ action, arguments: args })}`;
}

export const test = base.extend<{ todoApp: TodoApp; testingbotStatus: void }>({
  todoApp: async ({ page }, use) => {
    await page.goto('https://demo.playwright.dev/todomvc/');
    await use(new TodoApp(page));
  },

  testingbotStatus: [async ({ page }, use, testInfo) => {
    const remote = !!testInfo.project.use.connectOptions;
    if (remote) {
      await page.evaluate(() => {}, executor('setSessionName', { name: testInfo.title }));
    }
    await use();
    if (remote) {
      const passed = testInfo.status === testInfo.expectedStatus;
      await page.evaluate(() => {}, executor('setSessionStatus', {
        passed,
        reason: testInfo.error?.message?.slice(0, 200) ?? '',
      }));
    }
  }, { auto: true }],
});

auto: true runs it for every test that imports test from this file, and the connectOptions check makes it a no-op for local runs. The setSessionName and setSessionStatus actions are the ones documented in our Playwright Test integration guide.

Your local HTML report is green, but every session in the TestingBot dashboard shows no pass or fail. What is missing?

trace: 'on' in the grid config A build name in tb:options The auto fixture that sends setSessionStatus through page.evaluate A tunnel

TestingBot cannot see your assertions; it only knows the result when the test reports it with the testingbot_executor call. The build name groups sessions but does not mark them passed.

Testing localhost and staging behind a firewall#

A remote browser cannot reach http://localhost:3000 on your machine. TestingBot Tunnel opens a secure connection so it can. Start the tunnel with an identifier:

java -jar testingbot-tunnel.jar $TB_KEY $TB_SECRET -i myPlaywrightTunnel

then pass the same identifier as tunnelIdentifier in tb:options; the helper above does that when TB_TUNNEL=myPlaywrightTunnel is set. The local website testing course explains how the tunnel works and its command-line options.

Remote browsers report connection refused for http://localhost:3000. What fixes it?

Use your machine's LAN IP instead of localhost Start TestingBot Tunnel with an identifier and pass the same tunnelIdentifier in tb:options Set baseURL to http://127.0.0.1:3000

localhost on a remote machine is that machine. The tunnel gives the remote browser a route back to your network; a LAN address is not reachable from the grid either.

Running the grid suite from GitHub Actions#

Add TB_KEY and TB_SECRET as repository secrets (Settings, then Secrets), and change the test step from chapter 7:

    - name: Run Playwright tests on TestingBot
      run: npx playwright test -c playwright.testingbot.config.ts
      env:
        TB_KEY: ${{ secrets.TB_KEY }}
        TB_SECRET: ${{ secrets.TB_SECRET }}

Because every project connects to a remote browser, the job no longer launches browsers itself, so you can drop the npx playwright install --with-deps step. Keep the report upload. If the app under test only exists inside the CI job, add the testingbot/testingbot-tunnel-action@v1 step with a tunnelIdentifier, as shown in the GitHub Actions integration guide.

Other languages#

The same endpoint works from Playwright's other bindings, because they all support connect. For pytest, see the Playwright pytest guide; the main Playwright documentation has Java, .NET and Ruby examples as well.