Exit course
Chapter 9 of 11 Beginner 8 min read 3 quick checks Updated

Run Cypress on TestingBot and in CI

Watch this chapter Starts at 5:14 in the 9:03 course video.
On this page
  1. How the TestingBot Cypress CLI works
  2. Install and configure
  3. Run it
  4. Testing an app on localhost
  5. Running in GitHub Actions

Your suite runs on your machine, in the browsers you have installed. That leaves gaps: Windows and macOS when you develop on the other one, older browser versions, and a CI runner that has to install browsers and run every spec itself. This chapter runs the same specs on TestingBot's browser grid with the TestingBot Cypress CLI, first from your laptop and then from GitHub Actions.

How the TestingBot Cypress CLI works#

The CLI does not connect your local Cypress to a remote browser. Instead it:

  1. zips your Cypress project (without node_modules),
  2. uploads it to TestingBot together with the browsers you asked for,
  3. TestingBot runs cypress run with the Cypress version you chose on a virtual machine with that browser and operating system,
  4. the CLI streams the Cypress output back to your terminal and exits with code 0 when every spec passed and 1 otherwise.

Each browser configuration becomes a session in the TestingBot dashboard, with a video and the Cypress logs. Because Cypress itself reports which tests passed, the session is marked passed or failed for you; there is no extra status call to add, unlike Selenium or Playwright.

Install and configure#

Install the CLI next to Cypress:

npm install --save-dev testingbot-cypress-cli
npx testingbot-cypress init

init writes a testingbot.json in the current folder. Replace its contents with:

{
    "auth": {
        "key": "",
        "secret": ""
    },
    "browsers": [
        {
            "browserName": "chrome",
            "platform": "WIN11",
            "version": "latest"
        },
        {
            "browserName": "firefox",
            "platform": "WIN11",
            "version": "latest"
        }
    ],
    "run_settings": {
        "cypress_project_dir": ".",
        "build_name": "certificate-cypress",
        "cypressVersion": "14.2.0",
        "npm_dependencies": {},
        "package_config_options": {},
        "start_tunnel": false,
        "local_ports": [],
        "realTimeLogs": true,
        "parallel": 1
    },
    "tunnel_settings": {
        "verbose": false
    }
}

What each part does:

  • auth: leave the key and secret empty and set the TESTINGBOT_KEY and TESTINGBOT_SECRET environment variables instead, so credentials never land in the repository. In CLI version 1.1.5 the variables fill in an empty value (or the <Your TestingBot key> placeholder init writes); they do not replace a value that is already filled in.
  • browsers: one entry per configuration. browserName is chrome, firefox or microsoftedge, platform an operating system such as WIN11, WIN10 or SEQUOIA, and version either a number or latest, latest-1 and so on. The Cypress browsers and platforms page lists what is available.
  • cypress_project_dir: the folder containing cypress.config.js; . when you run the CLI from the project root.
  • build_name: every session from this run is grouped under this build in the dashboard. certificate-cypress is the name the course certificate looks for.
  • cypressVersion: which Cypress runs your specs. Match the version in your package.json; the Cypress versions page lists the supported ones.
  • parallel: how many machines each browser's specs are split over. With 1, each browser runs all specs in one session.
  • start_tunnel and local_ports: only needed when the app under test runs on your machine; see below.

Add a script to package.json next to the two from chapter 1:

"cy:testingbot": "testingbot-cypress run"

testingbot.json contains "key": "TESTINGBOT_KEY" and you exported the real key as the TESTINGBOT_KEY environment variable. The CLI fails with 401 Unauthorized. Why?

The CLI only replaces an empty key or the placeholder init writes. Any other string, including the variable's name, is sent as your key. Leave the field empty and keep the real value in the environment.

Run it#

export TESTINGBOT_KEY=your-key TESTINGBOT_SECRET=your-secret
npm run cy:testingbot

The CLI uploads the project, prints a link to the build in the dashboard, and streams the familiar Cypress output for each browser:

  (Run Starting)
  │ Cypress:        14.2.0
  │ Browser:        Custom Chrome 156
  │ Specs:          7 found (assertions.cy.js, commands.cy.js, network.cy.js, ...
  Running:  assertions.cy.js                                        (1 of 7)
  ...
    √  All specs passed!                        00:20       19       19
✔ Cypress Project has finished running on TestingBot

That is the output of one of the runs we made for this course: the spec files from chapters 1 to 5, including the cy.session login and the stubbed network calls, ran unchanged on Chrome and Firefox on Windows 11. The cypress.env.json file from chapter 5 is uploaded with the project, which is why the login worked; in CI you pass those values differently (below).

Two more flags are worth knowing. --specs runs a subset, and --env sets Cypress environment variables on the remote run:

npx testingbot-cypress run --specs "cypress/e2e/secure.cy.js" --env username=tomsmith,password=SuperSecretPassword!

--group <name> overrides build_name for one run, and --parallel <n> overrides parallel. The CLI reference lists every option.

How many sessions run at once depends on your TestingBot plan. Sessions beyond that limit wait in a queue; with parallel greater than 1 and several browsers you can reach the limit quickly.

Your component tests from chapter 7 live in the same repository. What happens to them when you run testingbot-cypress run?

TestingBot's Cypress documentation lists Component Testing under limitations. Keep component tests in a separate CI step on the runner itself, and send the end-to-end specs to the grid.

Testing an app on localhost#

A virtual machine in TestingBot's data centre cannot reach http://localhost:3000 on your laptop. Set "start_tunnel": true and list the ports your app uses in local_ports, for example [3000]. The CLI then starts a TestingBot Tunnel before uploading and stops it afterwards, so the remote browser can open the local ports. The Cypress documentation walks through this with the Cypress kitchen-sink example, and the local website testing course explains how the tunnel works.

Running in GitHub Actions#

In CI the CLI is the whole test step. The job does not need browsers or even the Cypress app itself, because the specs run on TestingBot. CYPRESS_INSTALL_BINARY=0 tells npm ci to skip the Cypress download. Create .github/workflows/cypress.yml:

name: Cypress on TestingBot

on:
  push:
  pull_request:

jobs:
  e2e:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22
          cache: npm
      - name: Install dependencies
        run: npm ci
        env:
          CYPRESS_INSTALL_BINARY: 0
      - name: Run Cypress on TestingBot
        run: >
          npx testingbot-cypress run
          --group "ci-$GITHUB_RUN_ID"
          --env "username=$APP_USERNAME,password=$APP_PASSWORD"
        env:
          TESTINGBOT_KEY: ${{ secrets.TESTINGBOT_KEY }}
          TESTINGBOT_SECRET: ${{ secrets.TESTINGBOT_SECRET }}
          TESTINGBOT_CI: 1
          APP_USERNAME: ${{ secrets.APP_USERNAME }}
          APP_PASSWORD: ${{ secrets.APP_PASSWORD }}
  • Add TESTINGBOT_KEY, TESTINGBOT_SECRET, APP_USERNAME and APP_PASSWORD as repository secrets (Settings, then Secrets and variables, then Actions).
  • Secrets go into the step's env: and the command reads them as shell variables. Writing ${{ secrets.APP_PASSWORD }} inside run: would paste the value into the script text before the shell sees it, so a quote or $ in a password could break the command or run part of it.
  • cypress.env.json is in .gitignore, so the login credentials arrive through --env instead.
  • --group gives each workflow run its own build in the dashboard; GITHUB_RUN_ID is set by GitHub Actions in every job.
  • TESTINGBOT_CI=1 makes the CLI print a TestingBotSessionId=... line for every session, which TestingBot's CI plugins use to link results.
  • The CLI's exit code fails the job when a spec fails.

Component tests (chapter 7) do need the Cypress app, so give them their own job in the same file, under jobs:, whose install step downloads it as usual:

  component:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22
          cache: npm
      - name: Install dependencies and the Cypress app
        run: npm ci
      - name: Run component tests
        run: npx cypress run --component

The two jobs run in parallel, and only the e2e job skips the download. For other CI systems the same commands apply; the CI/CD integration guides cover the rest.

The workflow's install step takes minutes because it downloads the Cypress app on every run, but the tests themselves run on TestingBot. What is the fix?

The npm package stays (your editor and local runs need it), but CYPRESS_INSTALL_BINARY=0 skips downloading the app binary, which a job that only uploads specs never uses.