Features
Start a free trial

Playwright Test

Playwright comes with its own test-runner called Playwright Test.
It is designed to make end-to-end testing easy with Playwright.

More information about @playwright/test is available in the Playwright documentation.

See the TestingBot Playwright Test Node Example repository for a simple example on how to run Playwright tests with TestingBot.

Installation

To get started, please install the Playwright Test package:

Copy code
npm i -D @playwright/test

Configuration

Next, we will need to create a configuration file so that Playwright Test knows what tests to run and what options to use.

Create a file called playwright.config.js with the following contents:

Copy code
import { defineConfig } from '@playwright/test'
import { getConnectWsEndpoint } from './testingbot.config'

/**
 * See https://playwright.dev/docs/test-configuration.
 */
export default defineConfig({

  testDir: './tests',

  /* Maximum time one test can run for. */
  timeout: 30 * 1000,

  /* Fail the build on CI if you accidentally left test.only in the source code. */
  forbidOnly: !!process.env.CI,

  /* Retry on CI only */
  retries: process.env.CI ? 2 : 0,

  /* Opt out of parallel tests on CI. */
  workers: process.env.CI ? 1 : undefined,

  /* Reporter to use. See https://playwright.dev/docs/test-reporters */
  reporter: 'html',

  /* Shared settings for all the projects below. See https://playwright.dev/docs/api/class-testoptions. */
  use: {
    /* Base URL to use in actions like `await page.goto('/')`. */
    // baseURL: 'http://localhost:3000',

    /* Collect trace when retrying the failed test. See https://playwright.dev/docs/trace-viewer */
    trace: 'on-first-retry',
  },

  projects: [
    {
      name: 'playwright-chrome@latest:Windows 10',
      use: {
        connectOptions: { 
          wsEndpoint: getConnectWsEndpoint({
            browserName: 'chrome',
            browserVersion: 'latest',
            platform: 'WIN10'
          }) 
        }
      },
    },
    {
      name: 'playwright-firefox@latest:Linux',
      use: {
        connectOptions: { 
          wsEndpoint: getConnectWsEndpoint({
            browserName: 'firefox',
            browserVersion: 'latest',
            platform: 'LINUX'
          }) 
        }
      }
    }
  ]
})

You will also need to create a testingbot.config.js file, which will contain code to connect to our Playwright server:

Copy code
export function getConnectWsEndpoint(userCapabilities) {
    const defaultCapabilities = {
        'tb:options': {
            key: process.env.TB_KEY,
            secret: process.env.TB_SECRET
        }
    }
    const capabilities = { ...defaultCapabilities, ...userCapabilities }
    const connectUrl = `wss://cloud.testingbot.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}`
    return connectUrl
}

You're now all set up to run your first Playwright Test on TestingBot. Let's see the example below to run a test.

Example Test

As we've previously defined in our playwright.config.js file, our tests should be placed in the tests directory.

Create a new file in the tests directory called sample.spec.js:

Copy code
const { test, expect } = require('@playwright/test')

test('Sample TestingBot test', async ({ page }, testInfo) => {
    try {
        await page.evaluate(_ => {}, `testingbot_executor: ${JSON.stringify({action: 'setSessionName', arguments: { name: testInfo.project.name }})}`)
        await page.goto('https://testingbot.com/',{ waitUntil: 'networkidle' })
        await page.evaluate(_ => {}, `testingbot_executor: ${JSON.stringify({action: 'setSessionStatus', arguments: { passed: true }})}`)
    } catch (e) {
        await page.evaluate(_ => {}, `testingbot_executor: ${JSON.stringify({action: 'setSessionStatus', arguments: { passed: false, reason: e.message }})}`)
    }
})

To run the test, execute this command in the directory where you placed the configuration file:

Copy code
npx playwright test

This test will set the correct name to be shown in the TestingBot dashboard, navigate with the remote browser to the TestingBot homepage and then set the test as passed.

Reports

There are various Test Reporters that Playwright offers, which you can change in the configuration file.

For every test that runs on TestingBot, you will receive a video of the test, generated logs and other meta-data.

Other Options

You can set various Timeout options, use annotations and set up setup and teardown scripts.

The global setup and teardown scripts can be used to start (and stop) a TestingBot Tunnel, which allows you to test on websites that are not publicly available.

An example of a setup script:

Copy code
const testingbotTunnel = require('testingbot-tunnel-launcher')

module.exports = async () => {
    await new Promise((resolve, reject) => {
        testingbotTunnel({
          apiKey: process.env.TB_KEY,
          apiSecret: process.env.TB_SECRET,
          verbose: true
        }, function (err, tunnel) {
          if (err) {
            console.error(err.message)
            return reject(err)
          }
          resolve()
        })
    })
}