Features

Cypress Testing

Cypress is an end-to-end testing framework, JavaScript-based, which does not use Selenium.


Cypress provides easy and fast testing on Chrome and Firefox.
Because Cypress does not use Selenium, we've decided to publish a CLI tool that will run your Cypress tests on the TestingBot browser grid.


Currently we are using Cypress v5.4.0. Testing is available with these capabilities:

key value possible values
platform The Operating System (OS) you want to run the tests on. WIN10, BIGSUR, CATALINA and MOJAVE
browserName The browser you want to run the tests on. chrome, firefox and edge
version The version of the browser Chrome: version 70 to latest
Firefox: version 70 to latest
Edge: version 80 to latest (WIN10 only)

Install the CLI

To get started with the CLI, please install it with npm or yarn:

$ npm install testingbot-cypress-cli

Configuration

Once installed, you'll need to provide the CLI with a testingbot.json configuration file.
This file contains various settings, including the path to your Cypress tests.


To generate a configuration file, you can use the init command:

$ testingbot-cypress init

Please add the key and secret you obtained from the TestingBot member dashboard in the testingbot.json file.

Next, please supply cypress_proj_dir with the path to the folder that contains the cypress.json file.
As an example, you can use the Cypress Kitchen Sink Example.


Before you can run your tests, you need to specify on which browser (version) and platform you'd like to test. At this time, only the following platforms are supported (we are working on adding more):

  • macOS Catalina
  • macOS Mojave
  • Windows 10

You can specify the browsers you want to use in the testingbot.json file:

"browsers": [
    {
        "browserName": "chrome",
        "platform": "MOJAVE",
        "version": 83
    },
    {
        "browserName": "chrome",
        "platform": "WIN10",
        "version": 83
    }
]

Run tests

You are now ready to run your Cypress tests on TestingBot.


Run the following command:

$ testingbot-cypress run

This will zip your Cypress tests, upload them to TestingBot and instruct TestingBot to start running the tests on the platform(s) you specified in the configuration file. The CLI will then display a link to view a live feed of your Cypress tests and will return realtime logs of the Cypress client running on TestingBot.


Once the tests have finished running, the CLI will exit with code 1 (failure) or 0 (success).

Example

As an example, you can use cypress-example-kitchensink


These are the steps you need to follow to run the example on TestingBot:

  • Clone the kitchensink repo and install the dependencies: npm install
  • Start the kitchensink example: npm start
  • Add the local port used by kitchensink (8080) to the local_ports array, this way the TestingBot VMs can access the port.
  • Change the cypress_project_dir to point to the path where you installed the example.
  • Run the tests:
    $ testingbot-cypress run
  • Test results will appear in your console and on the TestingBot dashboard.

Options

Several options are available in the testingbot.json file.


  • cypress_project_dir: this needs to be specified and is the directory (absolute path) where your cypress.json file is located.
  • build_name: this will set the build name on TestingBot. All Cypress tests will be grouped under this name.
  • npm_dependencies: you can list the NPM dependencies that your Cypress project requires here:
    "npm_dependencies": {
      "npm-package-1": "^1.0.1",
      "npm-package-2": "^1.2.3-beta",
    }
  • package_config_options: the config options for each of the npm dependencies you specified:
    "package_config_options": {
      "npm-package-1": {
        "option1": true,
        "option2": "some-value"
      }
    }
  • start_tunnel: should the CLI start a TestingBot Tunnel to access websites on your local computer/network?
  • local_ports: an array of ports that Cypress is using on your local computer. Use this in combination with start_tunnel = true to run tests from your local computer or network.
  • realTimeLogs: should the CLI display a real-time feed of the Cypress logs?
  • tunnel_settings: add additional Tunnel options to the TestingBot tunnel process.

Environment Variables

The CLI supports various Environment Variables:


  • You can use TESTINGBOT_KEY and TESTINGBOT_SECRET to pass the TestingBot credentials to the CLI.
  • If you set TESTINGBOT_CI=1 then the CLI will output the TestingBotSessionID which can be used in combination with one of our CI plugins.