Features
Start a free trial

Nightwatch Automated Browser Testing

See the TestingBot NightWatch example repository for an example on how to run NightWatch tests in parallel on TestingBot.

Install Nightwatch:

Copy code
npm install nightwatch testingbot-api --save-dev

Now, you'll need to create a settings file so that Nightwatch knows it needs to connect to the TestingBot Selenium Grid.

Create a nightwatch.conf.js file with the following contents.

Copy code
const testingBotOptions = {
  'tb:options' : {
    "build" : "nightwatch-example",
    "name" : "Nightwatch Example Test",
    "seleniumVersion" : "4.24.0",
    username: '${TESTINGBOT_KEY}' || 'YOUR_USERNAME',
    accessKey: '${TESTINGBOT_SECRET}' || 'YOUR_ACCESS_KEY',
  },
}

const testingBot = {
  webdriver: {
    start_process: false,
    timeout_options: {
      timeout: 120000,
      retry_attempts: 3
    },
    keep_alive: true
  },

  selenium: {
    host: 'hub.testingbot.com',
    port: 443
  },

  desiredCapabilities: {
    browserName: 'chrome',
    ...testingBotOptions
  }
}

const nightwatchConfigs = {
  src_folders: ['tests'],
  custom_commands_path: 'custom_commands',
  output_folder: 'reports',
  live_output: true,

  test_settings: {
    default: {
      launch_url: 'https://testingbot.com'
    },

    "testingbot.chrome": {
      ...testingBot,
      desiredCapabilities:{
        browserName: 'chrome',
        ...testingBotOptions
      }
    },
    "testingbot.firefox": {
      ...testingBot,
      desiredCapabilities:{
        browserName: 'firefox',
        ...testingBotOptions
      }
    },
    "testingbot.edge": {
      ...testingBot,
      desiredCapabilities:{
        browserName: 'Edge',
        ...testingBotOptions
      }
    }
  }
}

module.exports = nightwatchConfigs;

We will also need to add a test script, let's create a new directory called tests and add a file: googleTest.js

Copy code
const https = require('https');

module.exports = {
    '@tags': ['test'],

    'Google': function(client) {
        client
            .url('https://testingbot.com')
            .waitForElementVisible('body', 1000)
            .assert.textContains('body', 'TestingBot')
            .end();
    },

    afterEach: function(client, done) {
        client.customTestingBotEnd();
        done();
    }
};

To report back the status to TestingBot, we'll need to create a custom command in the custom_commands directory. Please create a file called customTestingBotEnd.js in this directory.

Copy code
exports.command = function(callback) {
  const TestingBot = require("testingbot-api");

  const tb = new TestingBot({
    api_key: process.env.TESTINGBOT_KEY,
    api_secret: process.env.TESTINGBOT_SECRET
  });

  const sessionid = this.capabilities['webdriver.remote.sessionid'],
    jobName = this.currentTest.name,
    passed = this.currentTest.results.testcases[jobName].failed === 0;

  console.log("TestingBotSessionId=" + sessionid);

  tb.updateTest({
    'test[success]': passed,
    'test[name]': jobName
  }, sessionid, () => {
    this.end(callback)
  });
};

To run the test, please run this command:

Copy code
TESTINGBOT_KEY=... TESTINGBOT_SECRET=... ./node_modules/.bin/nightwatch --test ./tests/googleTest.js --env testingbot.chrome

Configuring capabilities

To run your existing tests on TestingBot, your tests will need to be configured to use the TestingBot remote machines. If the test was running on your local machine or network, you can simply change your existing test like this:

Before:
Copy code
"selenium" : {
  "start_process" : false,
  "host" : "localhost",
  "port" : 4444
},
After:
Copy code
"selenium" : {
  "start_process" : false,
  "host" : "hub.testingbot.com",
  "port" : 443
},

Specify Browsers & Devices

To let TestingBot know on which browser/platform/device you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field.

1. Select a Platform
2. Select a browser
Copy code

Testing Internal Websites

We've built TestingBot Tunnel, to provide you with a secure way to run tests against your staged/internal webapps.
Please see our TestingBot Tunnel documentation for more information about this easy to use tunneling solution.

The example below shows how to easily run a NodeJS test with our Tunnel:

1. Download our tunnel and start the tunnel:

Copy code
java -jar testingbot-tunnel.jar key secret

2. Adjust your test: instead of pointing to 'hub.testingbot.com/wd/hub' like the example above - change it to point to your tunnel's IP address.
Assuming you run the tunnel on the same machine you run your tests, change to 'localhost:4445/wd/hub'. localhost is the machine running the tunnel, 4445 is the default port of the tunnel.

This way your test will go securely through the tunnel to TestingBot and back:

Copy code
{
  "src_folders" : ["tests/"],

  "selenium" : {
    "start_process" : false,
    "host" : "localhost",
    "port" : 4445
  },

  "test_settings" : {
    "default" : {
      "desiredCapabilities": {
        "browserName": "firefox",
        "platform":"VISTA",
        "version":"latest"
      }
    }
  }
}

Run tests in Parallel

Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time.

You can run the same tests on all different browser configurations or run different tests all on the same browser configuration.
TestingBot has a large grid of machines and browsers, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time.

To run tests in parallel, you can use the following command to specify multiple capabilities at once.

settings.json
Copy code
./node_modules/.bin/nightwatch --test ./tests/googleTest.js --env testingbot.chrome,testingbot.edge,testingbot.firefox

Queuing

Every plan we provide comes with a limit of parallel tests.
If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available.

Mark tests as passed/failed

As TestingBot has no way to dermine whether your test passed or failed (it is determined by your business logic), we offer a way to send the test status back to TestingBot. This is useful if you want to see if a test succeeded or failed from the TestingBot member area.

Install our NPM package to interact with the TestingBot API:

Copy code
npm install testingbot-api

After your test completed, you can send back the test success state and other meta-data:

Copy code
exports.command = function(callback) {
  const TestingBot = require("testingbot-api");

  const tb = new TestingBot({
    api_key: process.env.TB_KEY,
    api_secret: process.env.TB_SECRET
  });

  const sessionid = this.capabilities['webdriver.remote.sessionid'],
    jobName = this.currentTest.name,
    passed = this.currentTest.results.testcases[jobName].failed === 0;


  console.log("TestingBotSessionId=" + sessionid);

  tb.updateTest({
    'test[success]': passed,
    'test[name]': jobName
  }, sessionid, () => {
    this.end(callback)
  });
};

Other NodeJS Framework examples

  • WebDriverIO

    With WebDriverIO you can run Mocha, Jasmine and Cucumber tests.

  • CodeceptJS

    Run acceptance tests with CodeceptJS on TestingBot.

  • Protractor

    Protractor is an end-to-end test framework for AngularJS applications. Protractor is a NodeJS program built on top of WebDriverJS.

  • Soda

    Selenium Node Adapter. A light-weight Selenium RC client for NodeJS.

  • Nightwatch

    Nightwatch is an automated testing framework written in NodeJS.

  • Intern

    Intern is a nodeJS framework for testing Web sites and applications.

  • WD.js

    WD.js is a NodeJS client for WebDriver/Selenium.

  • Hermione

    Hermione is a Test Framework similar to WebDriverIO, with automatic test retries and plugins.