Run Cypress on TestingBot and in CI
On this page
Your suite runs on your machine, in the browsers you have installed. That leaves gaps: Windows and macOS when you develop on the other one, older browser versions, and a CI runner that has to install browsers and run every spec itself. This chapter runs the same specs on TestingBot's browser grid with the TestingBot Cypress CLI, first from your laptop and then from GitHub Actions.
How the TestingBot Cypress CLI works#
The CLI does not connect your local Cypress to a remote browser. Instead it:
- zips your Cypress project (without
node_modules), - uploads it to TestingBot together with the browsers you asked for,
- TestingBot runs
cypress runwith the Cypress version you chose on a virtual machine with that browser and operating system, - the CLI streams the Cypress output back to your terminal and exits with code 0 when every spec passed and 1 otherwise.
Each browser configuration becomes a session in the TestingBot dashboard, with a video and the Cypress logs. Because Cypress itself reports which tests passed, the session is marked passed or failed for you; there is no extra status call to add, unlike Selenium or Playwright.
Install and configure#
Install the CLI next to Cypress:
npm install --save-dev testingbot-cypress-cli
npx testingbot-cypress init
init writes a testingbot.json in the current folder. Replace its contents with:
{
"auth": {
"key": "",
"secret": ""
},
"browsers": [
{
"browserName": "chrome",
"platform": "WIN11",
"version": "latest"
},
{
"browserName": "firefox",
"platform": "WIN11",
"version": "latest"
}
],
"run_settings": {
"cypress_project_dir": ".",
"build_name": "certificate-cypress",
"cypressVersion": "14.2.0",
"npm_dependencies": {},
"package_config_options": {},
"start_tunnel": false,
"local_ports": [],
"realTimeLogs": true,
"parallel": 1
},
"tunnel_settings": {
"verbose": false
}
}
What each part does:
-
auth: leave the key and secret empty and set theTESTINGBOT_KEYandTESTINGBOT_SECRETenvironment variables instead, so credentials never land in the repository. In CLI version 1.1.5 the variables fill in an empty value (or the<Your TestingBot key>placeholderinitwrites); they do not replace a value that is already filled in. -
browsers: one entry per configuration.browserNameischrome,firefoxormicrosoftedge,platforman operating system such asWIN11,WIN10orSEQUOIA, andversioneither a number orlatest,latest-1and so on. The Cypress browsers and platforms page lists what is available. -
cypress_project_dir: the folder containingcypress.config.js;.when you run the CLI from the project root. -
build_name: every session from this run is grouped under this build in the dashboard.certificate-cypressis the name the course certificate looks for. -
cypressVersion: which Cypress runs your specs. Match the version in yourpackage.json; the Cypress versions page lists the supported ones. -
parallel: how many machines each browser's specs are split over. With1, each browser runs all specs in one session. -
start_tunnelandlocal_ports: only needed when the app under test runs on your machine; see below.
Add a script to package.json next to the two from chapter 1:
"cy:testingbot": "testingbot-cypress run"
testingbot.json contains "key": "TESTINGBOT_KEY" and you exported the real key as the TESTINGBOT_KEY environment variable. The CLI fails with 401 Unauthorized. Why?
The CLI only replaces an empty key or the placeholder init writes. Any other string, including the variable's name, is sent as your key. Leave the field empty and keep the real value in the environment.
Run it#
export TESTINGBOT_KEY=your-key TESTINGBOT_SECRET=your-secret
npm run cy:testingbot
The CLI uploads the project, prints a link to the build in the dashboard, and streams the familiar Cypress output for each browser:
(Run Starting)
│ Cypress: 14.2.0
│ Browser: Custom Chrome 156
│ Specs: 7 found (assertions.cy.js, commands.cy.js, network.cy.js, ...
Running: assertions.cy.js (1 of 7)
...
√ All specs passed! 00:20 19 19
✔ Cypress Project has finished running on TestingBot
That is the output of one of the runs we made for this course: the spec files from chapters 1 to 5, including the cy.session login and the stubbed network calls, ran unchanged on Chrome and Firefox on Windows 11. The cypress.env.json file from chapter 5 is uploaded with the project, which is why the login worked; in CI you pass those values differently (below).
Two more flags are worth knowing. --specs runs a subset, and --env sets Cypress environment variables on the remote run:
npx testingbot-cypress run --specs "cypress/e2e/secure.cy.js" --env username=tomsmith,password=SuperSecretPassword!
--group <name> overrides build_name for one run, and --parallel <n> overrides parallel. The CLI reference lists every option.
How many sessions run at once depends on your TestingBot plan. Sessions beyond that limit wait in a queue; with parallel greater than 1 and several browsers you can reach the limit quickly.
Your component tests from chapter 7 live in the same repository. What happens to them when you run testingbot-cypress run?
TestingBot's Cypress documentation lists Component Testing under limitations. Keep component tests in a separate CI step on the runner itself, and send the end-to-end specs to the grid.
Testing an app on localhost#
A virtual machine in TestingBot's data centre cannot reach http://localhost:3000 on your laptop. Set "start_tunnel": true and list the ports your app uses in local_ports, for example [3000]. The CLI then starts a TestingBot Tunnel before uploading and stops it afterwards, so the remote browser can open the local ports. The Cypress documentation walks through this with the Cypress kitchen-sink example, and the local website testing course explains how the tunnel works.
Running in GitHub Actions#
In CI the CLI is the whole test step. The job does not need browsers or even the Cypress app itself, because the specs run on TestingBot. CYPRESS_INSTALL_BINARY=0 tells npm ci to skip the Cypress download. Create .github/workflows/cypress.yml:
name: Cypress on TestingBot
on:
push:
pull_request:
jobs:
e2e:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 22
cache: npm
- name: Install dependencies
run: npm ci
env:
CYPRESS_INSTALL_BINARY: 0
- name: Run Cypress on TestingBot
run: >
npx testingbot-cypress run
--group "ci-$GITHUB_RUN_ID"
--env "username=$APP_USERNAME,password=$APP_PASSWORD"
env:
TESTINGBOT_KEY: ${{ secrets.TESTINGBOT_KEY }}
TESTINGBOT_SECRET: ${{ secrets.TESTINGBOT_SECRET }}
TESTINGBOT_CI: 1
APP_USERNAME: ${{ secrets.APP_USERNAME }}
APP_PASSWORD: ${{ secrets.APP_PASSWORD }}
- Add
TESTINGBOT_KEY,TESTINGBOT_SECRET,APP_USERNAMEandAPP_PASSWORDas repository secrets (Settings, then Secrets and variables, then Actions). - Secrets go into the step's
env:and the command reads them as shell variables. Writing${{ secrets.APP_PASSWORD }}insiderun:would paste the value into the script text before the shell sees it, so a quote or$in a password could break the command or run part of it. -
cypress.env.jsonis in.gitignore, so the login credentials arrive through--envinstead. -
--groupgives each workflow run its own build in the dashboard;GITHUB_RUN_IDis set by GitHub Actions in every job. -
TESTINGBOT_CI=1makes the CLI print aTestingBotSessionId=...line for every session, which TestingBot's CI plugins use to link results. - The CLI's exit code fails the job when a spec fails.
Component tests (chapter 7) do need the Cypress app, so give them their own job in the same file, under jobs:, whose install step downloads it as usual:
component:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 22
cache: npm
- name: Install dependencies and the Cypress app
run: npm ci
- name: Run component tests
run: npx cypress run --component
The two jobs run in parallel, and only the e2e job skips the download. For other CI systems the same commands apply; the CI/CD integration guides cover the rest.
The workflow's install step takes minutes because it downloads the Cypress app on every run, but the tests themselves run on TestingBot. What is the fix?
The npm package stays (your editor and local runs need it), but CYPRESS_INSTALL_BINARY=0 skips downloading the app binary, which a job that only uploads specs never uses.