Automated Test Options

Below are various options you can use for your Automated Testing.

The Selenium-Specific Settings are required to run Automated Tests and are called Desired Capabilities.

The TestingBot Options allow you to customize your test in terms of specific driver versions, privacy options, platform options, ...

Selenium-Specific Settings


TestingBot Options

Selenium Test Options

Browser Name

The name of the browser to run your automated test on. Please see our list of browsers that we support.

The browserName is a required field that needs to be passed to us via Selenium's Desired Capabilities.


Example:
"browserName" : "firefox"

Browser Version

The version of the browser to run your automated test on. Please see our list of browsers + versions that we support.

The version is a required field that needs to be passed to us via Selenium's Desired Capabilities.


Example:
"version" : "56"

Other Options:
  • "version": "*" : If you use * as version, TestingBot will pick a random version.
  • "version": "latest" : TestingBot will automatically take the latest version. You can also use latest-1, latest-2, ... to test on the next most recent versions.

    For example, if the current latest Firefox version is 52 and you use latest-2, then the test will run on Firefox 50.

Platform

Indicates on which operating system the test should run. Please see a list of platforms that we currently support.

The platform is a required field that needs to be passed to us via Selenium's Desired Capabilities.


Example:
"platform" : "Windows 7"

Make screenshots of your tests

By default we do not capture screenshots at every step of your test. If you wish to take a screenshot for every step, please add this capability (set to true) to your request.


"screenshot" : false

Value Type Default Value
boolean false

Make a video of your tests

By default we record a video of your test, which is accessible in the member area. If you do not wish to have this, you can disable it with this option.
Video should not slow down your test considerably.


"screenrecorder" : true

Value Type Default Value
boolean true

Use a specific version of Selenium

By default we use the latest Selenium version to run your test.
If you wish to use another Selenium version for your test, please specify one of the following available versions:


"selenium-version" : "2.53.1"

Value Type Default Value Possible values:
string "2.53.1"
  • 3.6.0
  • 3.5.3
  • 3.5.2
  • 3.5.1
  • 3.5.0
  • 3.4.0
  • 3.3.1
  • 3.3.0
  • 3.2.0
  • 3.1.0
  • 3.0.1
  • 3.0.0
  • 2.53.0
  • 2.52.0
  • 2.51.0
  • 2.50.0
  • 2.49.0
  • 2.48.0
  • 2.47.0
  • 2.46.0
  • 2.45.0
  • 2.44.0
  • 2.43.0
  • 2.42.0
  • 2.41.0
  • 2.40.0
  • 2.39.0
  • 2.38.0
  • 2.37.0
  • 2.36.0
  • 2.35.0
  • 2.34.0
  • 2.33.0
  • 2.32.0
  • 2.31.0
  • 2.30.0
  • 2.29.0
  • 2.28.0
  • 2.27.0
  • 2.26.0
  • 2.25.0
  • 2.24.0
  • 2.23.0
  • 2.22.0
  • 2.21.0
  • 2.20.0
  • 2.19.0
  • 2.18.0
  • 2.17.0
  • 2.16.0
  • 2.15.0
  • 2.14.0
  • 2.13.0
  • 2.12.0
  • 2.11.0
  • 2.10.0
  • 2.9.0
  • 2.8.0
  • 2.7.0
  • 2.6.0
  • 2.5.0
  • 2.4.0
  • 2.3.0
  • 2.2.0
  • 2.1.0
  • 2.0.0

Chromedriver

We support using custom ChromeDriver versions during your tests. By default the most recent Chromedriver is used according to the version of Chrome you're testing on.


"chromedriverVersion" : "2.32"

Value Type Default Value Possible values:
string "2.33" 2.33, 2.32, 2.31, 2.30, 2.29, 2.28, 2.27., 2.26, 2.25, 2.24, 2.23, 2.22, 2.21, 2.20, 2.19, 2.16, 2.15, 2.14, 2.13, 2.12, 2.10

Internet Explorer Driver (IEDriver)

We provide both 32-bit and 64-bit versions of the Internet Explorer driver. By default we use the most recent 32-bit IEDriver for your tests.
We use 32-bit because of a slow text entry bug with the 64-bit version.

For our screenshots, we use the 64-bit version, because of a screenshot problem with the 32-bit version.


"iedriverVersion" : "2.53.1"

Value Type Default Value Possible values:
string 2.53.1 2.45.0, 2.46.0, 2.47.0, 2.48.0, 2.49.0, 2.50.0, 2.51.0, 2.52.0, 2.52.1, 2.52.2, 2.53.0, 2.53.1, 3.1.0, 3.2.0, 3.3.0, 3.4.0, 3.5.0, 3.6.0, x64_2.45.0, x64_2.46.0, x64_2.47.0, x64_2.48.0, x64_2.49.0, x64_2.50.0, x64_2.51.0, x64_2.52.0, x64_2.52.1, x64_2.52.2, x64_2.53.0, x64_2.53.1, x64_3.1.0, x64_3.2.0, x64_3.3.0, x64_3.4.0, x64_3.5.0, x64_3.6.0

Edge Driver (MicrosoftWebDriver)

EdgeDriver is built by Microsoft to automate the Microsoft Edge Browser.
By default, we make sure that the most recent fully-compatible EdgeDriver is used for every test running on the Microsoft Edge Browser.

See the table below to know which EdgeDriver we use for which Edge Browser version:


Browser Version Edge Driver Version
16 16299
15 15063
14 14393
"edgedriverVersion" : "15063"

Value Type Default Value Possible values:
string 15063 "insiders", "16299", "15063", "14393"

Firefox Driver (Geckodriver)

For Firefox 47 and up, webdriver tests on Firefox need to use Mozilla's GeckoDriver.
Specify this option to choose which Geckodriver we should use. By default, we use the most recent version.


"geckodriverVersion" : "0.19.0"

Value Type Default Value Possible values:
string 0.19.0 0.19.0, 0.18.0, 0.17.0, 0.16.1, 0.16.0, 0.15.0, 0.14.0, 0.13.0, 0.12.0, 0.11.0, 0.10.0, 0.9.0, 0.8.0, 0.7.1, 0.6.2

Test Name

Add a name to this test, which will show up in our member area and API.


"name" : "My Test Name"

Value Type Default Value
string unnamed test

Screen Resolution

Will adjust the screen resolution during your test.


"screen-resolution" : "1280x1024"

Value Type Default Value Possible values:
string "1280x1024"
Platform Resolutions
Windows/Linux
  • 800x600
  • 1024x768
  • 1152x864
  • 1280x768
  • 1280x800
  • 1280x960
  • 1280x1024
  • 1400x1050
  • 1600x1200
  • 1680x1050
  • 1920x1080
  • 1920x1200
  • 2560x1440
  • 2560x1600
Mac
  • 1024x768
  • 1152x864
  • 1280x960
  • 1376x1032
  • 1400x1050
  • 1600x1200
  • 1920x1440
  • 2048x1536
  • 2360x1770

Build

A key you can use to group certain tests in the same build (for example in Jenkins).
The builds will appear in our member area.


"build" : "My First Build"

Value Type
string

Custom data

Send along custom data, for example your release, server, commit hash, ...
This will show up on the test detail page in the TestingBot member area.


"extra" : "Extra Information"

Value Type
string

Maximum test duration

The maximum duration for a single test. This is a safeguard to prevent bad tests from using up your credits.

We generally recommend to keep tests short (less than 10 minutes). It's better to split up large tests in smaller individual tests.
This keeps your tests fast and allows for more parallelization of your tests.


"maxduration" : 1800

Value Type Default Value
int (specify number of seconds) 1800 seconds (30 minutes)

Idle timeout

The maximum amount of time a browser will wait before proceeding to the next step in your test.


"idletimeout" : 130

Value Type Default Value
int (specify number of seconds) 130 seconds

Test Privacy

Make the test results for this test public so that everyone can access the results.


"public" : false

Value Type Default Value
boolean false

Custom Time Zones

Change the Time Zone of the Virtual Machine to the Time Zone you specify. You can find a list of timezones on Wikipedia. Only location names are supported (not their paths). See some examples below:


"timeZone" : "Etc/UTC"

Value Type Default Value Possible values:
string "Etc/UTC" List of timezones

Chrome/Firefox/Safari/Edge Extension

If you specify this option with the URL to a browser extension (Chrome/Firefox/Safari/Edge extension), then we will download the extension and add it automatically to either Chrome, Firefox, Microsoft Edge or Safari before your test starts.


"load-extension" : "https://..."

Browser Supported Format Example
Chrome .zip file, containing the extension source code (manifest.json, src/ directory, ...) https://.../extension.zip
Firefox (Firefox-Unbranded) .zip file, containing the extension source code (manifest.json, src/ directory, lib/ directory, ...) https://.../extension.zip
Edge .zip file, containing the extension source code (manifest.json, src/ directory, ...) https://.../extension.zip
Safari (and Safari Technology Preview) .safariextz file https://.../extension.safariextz

Grouping

Specify in which groups you want to see the test results. You can group results to have an easy overview of tests across projects.


"groups" : "group1,group2"

Value Type
string (comma-separated)

Pre-run executables

When you specify a prerun file, we will first download the file and execute it (with optional prerun-arguments).
This is useful when you want to customize/add software before your test starts.


"prerun" : "https://..."

Value Type
string (URL)

"prerun-args" : "-a -b -c ..."

Value Type
string

Upload file to VM

When you specify an URL (upload) and fileName (uploadFilepath), we will automatically download the file from the URL and save it in the downloadFilepath.


"upload" : "https://..."

Value Type
string (URL)

We recommend using these directories to save your files: For Windows ("C:\test\"), for Linux/Mac: ("/tmp/").

"uploadFilepath" : "C:\\test\\myfile.ext"

Value Type
string (URL)

Upload Multiple Files to VM

Specify an array of objects, containing keys 'url' and 'filePath'. We will automatically download these files and put them in the filePaths you specify.

We recommend using these directories to save your files: For Windows ("C:\test\"), for Linux/Mac: ("/tmp/").


"uploadMultiple" : []

Value Type
array of objects, { url: "", filePath: "" }

Autoclicker

Specify one or more of the possible options below to have TestingBot automatically click certain dialogs which are unable to be automated via Selenium WebDriver.


"autoclick" : []

Value Type Possible values:
array of options chrome_extension_add, chrome_extension_permissions_allow

Sikuli

Specify an URL to a zip-file containing one or more .sikulu projects. These will automatically start before your test runs, so that you can do certain automated tasks which Selenium cannot do (clicking native dialogs, ...)

Find more information regarding Sikuli Cloud Testing.


"sikuli" : "https://..."

Value Type
string (URL to zipped Sikuli project)

Custom hostnames

You can specify an array of objects containing the keys 'ip' and 'domain'. We will write these values to the hosts file of the VM.


"hosts" : [{ ip: '127.0.0.1', domain: 'mydomain' }]

Value Type
array of objects ({ ip: "", domain: "" })

Blacklist hostnames

The hostnames you specify will be pointed to localhost instead of their real destination. This means you can speed up tests by blocking third party content which you don't need and slows down your test.


"blacklist" : "site1.com,site2.com"

Value Type
string (comma-separated)

Disable Logging

By default, TestingBot records logs of all Selenium actions and its drivers.

Set this option to false if you don't want TestingBot to record anything (for example, if you have sensitive data).
You will not see any test logs in our member dashboard.
Set to strip-parameters to prevent the POST/GET parameters from being logged on the TestingBot test detail page (does not affect other logs like Selenium logs, Chromedriver logs, ...).


"recordLogs" : "true"

Value Type Default Value Possible values:
string "true" true, false, or strip-parameters