NodeJS Automated Testing with WebdriverIO

WebDriverIO TestingBot Service

WebDriverIO contains a TestingBot service which makes it easy to run tests with WebDriver.io on TestingBot. With this service, WebDriverIO will start a TestingBot Tunnel if required, and send back test meta-data (passed state, name, build-id, ...)

In order to use the service you need to install it:

npm install @wdio/testingbot-service --save-dev

Cucumber Example

To run Cucumber tests on TestingBot, please follow these steps:

npm install @wdio/testingbot-service --save-dev
npm install @wdio/dot-reporter --save-dev
npm install @wdio/cucumber-framework --save-dev

Now we'll make a simple Cucumber test which uses Firefox on TestingBot to go to Google.com and verify the page's title.

Please create the following files:

exports.config = {

	/**
	 * specify test files
	 */
	specs: [
		'./features/*.feature'
	],

	/**
	 * capabilities
	 */
	capabilities: [{
		browserName: 'chrome',
		browserVersion: 'latest',
		platformName: 'WIN10'
	}],

	/**
	 * test configurations
	 */
	logLevel: 'silent',
	coloredLogs: true,
	screenshotPath: 'screenshots',
	waitforTimeout: 10000,
	framework: 'cucumber',
	services: [
        ['testingbot']
    ],
	user: 'api_key',
	key: 'api_secret',

	reporters: ['dot'],
	reporterOptions: {
		outputDir: './'
	},

	cucumberOpts: {
		require: ['./step-definitions.js']
	}
};

const { Given, When, Then } = require('@cucumber/cucumber')
const assert = require('assert')

Given('I go on the website {string}', async (url) => {
    await browser.url(url)
})

Then('should the title of the page be {string}', async (expectedTitle) => {
    assert.equal(await browser.getTitle(), expectedTitle)
})

Feature: Example feature
As a user of Google
I should be able to go to Google and see its correct title

Scenario: Get title of website
Given I go on the website "https://www.google.com/"
Then should the title of the page be "Google"

Now we can run this test on TestingBot!
To start the test, please run this command:

~/node_modules/.bin/wdio run wdio.conf.js

The test will now run on TestingBot. At the end of the test, the testname and success state will be available on TestingBot, together with a video and other meta-data.

Mocha Example

To run Mocha tests on TestingBot, please follow these steps:

npm install @wdio/testingbot-service --save-dev
npm install @wdio/dot-reporter --save-dev
npm install @wdio/mocha-framework --save-dev

Now we'll make a simple Mocha test which uses Firefox on TestingBot to go to Google.com and verify the page's title.

Please create the following files:

exports.config = {

	/**
	 * specify test files
	 */
	specs: [
		'./runner-specs/mocha.test.js'
	],

	/**
	 * capabilities
	 */
	capabilities: [{
		browserName: 'chrome',
		browserVersion: 'latest',
		platformName: 'WIN10'
	}],

	/**
	 * test configurations
	 */
	logLevel: 'silent',
	coloredLogs: true,
	screenshotPath: 'screenshots',
	waitforTimeout: 10000,
	framework: 'mocha',
	services: [
        ['testingbot']
    ],
	user: 'api_key',
	key: 'api_secret',

	reporters: ['dot'],
	reporterOptions: {
		outputDir: './'
	},

	mochaOpts: {
		ui: 'bdd'
	}
};

var assert = require('assert');
describe('google page', async function() {
	it('should have the right title', async function () {
		await browser.url('https://www.google.com');
		var title = await browser.getTitle();
		assert.equal(title, 'Google');
	});
});

Now we can run this test on TestingBot.
To start the test, please run this command:

~/node_modules/.bin/wdio run wdio.conf.js

The test will now run on TestingBot. At the end of the test, the testname and success state will be available on TestingBot, together with a video and other meta-data.

Jasmine Example

To run Jasmine tests on TestingBot, please follow these steps:

npm install @wdio/testingbot-service --save-dev
npm install @wdio/dot-reporter --save-dev
npm install @wdio/jasmine-framework --save-dev

Now we'll make a simple Jasmine test which uses Firefox on TestingBot to go to Google.com and verify the page's title.

Please create the following files:

exports.config = {

	/**
	 * specify test files
	 */
	specs: [
		'./runner-specs/jasmine.spec.js'
	],

	/**
	 * capabilities
	 */
	capabilities: [{
		browserName: 'chrome',
		browserVersion: 'latest',
		platformName: 'WIN10'
	}],

	/**
	 * test configurations
	 */
	logLevel: 'silent',
	coloredLogs: true,
	screenshotPath: 'screenshots',
	waitforTimeout: 10000,
	framework: 'jasmine',
	services: [
        ['testingbot']
    ],
	user: 'api_key',
	key: 'api_secret',

	reporters: ['dot'],
	reporterOptions: {
		outputDir: './'
	},

	jasmineNodeOpts: {
		defaultTimeoutInterval: 9999999
	}
};

describe('google page', function() {
	it('should have the right title', async function () {
		await browser.url('https://www.google.com');
		var title = await browser.getTitle();
		expect(title).toBe('Google');
	});
});

Now we can run this test on TestingBot.
To start the test, please run this command:

~/node_modules/.bin/wdio run wdio.conf.js

The test will now run on TestingBot. At the end of the test, the testname and success state will be available on TestingBot, together with a video and other meta-data.

Specify Browsers & Devices

To let TestingBot know on which browser/platform/device you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field.

var capabilities = {  
	'browserName'    : '%browser%',  
	'platformName'   : '%platform%',  
	'browserVersion' : '%version%'  
};

var capabilities = {  
	'browserName' : '%browser%',  
	'platform'    : '%platform%',  
	'browserVersion'     : '%version%',  
	'platformName'     : '%platformName%',  
	'deviceName'     : '%deviceName%'
};

1. Select a Platform
Windows 10

• Windows 11
• Windows 10
• Windows 8.1
• Windows 8
• Windows 7
• macOS Sequoia
• macOS Sonoma
• macOS Ventura
• macOS Monterey
• macOS Big Sur
• macOS Catalina
• macOS Mojave
• macOS High Sierra
• macOS Sierra
• OS X El Capitan
• OS X Yosemite
• OS X Mavericks
• Linux
• iOS Simulators
• Android Emulators

2. Select a browser
Internet Explorer 11

Internet Explorer 11

Testing Internal Websites

We've built TestingBot Tunnel, to provide you with a secure way to run tests against your staged/internal webapps.
Please see our TestingBot Tunnel documentation for more information about this easy to use tunneling solution.

The example below shows how to easily run a WebDriverIO test with our Tunnel:

1. Adjust your wdio.conf.js configuration to use tbTunnel: true.
The wdio-testingbot-service will automatically download and use the tunnel for your tests.

exports.config = {
	services: [
        ['testingbot', {
            tbTunnel: true
        }]
    ],
	user: 'api_key',
	key: 'api_secret'
};

Run tests in Parallel

Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time.

You can run the same tests on all different browser configurations or run different tests all on the same browser configuration.
TestingBot has a large grid of machines and browsers, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time.

To run tests in parallel, you can use WebDriverIO's MultiRemote feature where you specify multiple desired capabilities for your tests.

Queuing

Every plan we provide comes with a limit of parallel tests.
If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available.

Mark tests as passed/failed

As TestingBot has no way to dermine whether your test passed or failed (it is determined by your business logic), we offer a way to send the test status back to TestingBot. This is useful if you want to see if a test succeeded or failed from the TestingBot member area.

If you are using the wdio-testingbot-service then your tests will automatically report back meta-data to TestingBot (like test success, name, stacktrace, ...)

Other NodeJS Framework examples

• WebDriverIO

  With WebDriverIO you can run Mocha, Jasmine and Cucumber tests.

• CodeceptJS

  Run acceptance tests with CodeceptJS on TestingBot.

• Protractor

  Protractor is an end-to-end test framework for AngularJS applications. Protractor is a NodeJS program built on top of WebDriverJS.

• Soda

  Selenium Node Adapter. A light-weight Selenium RC client for NodeJS.

• Nightwatch

  Nightwatch is an automated testing framework written in NodeJS.

• Intern

  Intern is a nodeJS framework for testing Web sites and applications.

• WD.js

  WD.js is a NodeJS client for WebDriver/Selenium.

• Hermione

  Hermione is a Test Framework similar to WebDriverIO, with automatic test retries and plugins.