TestNG & Cucumber Automated Testing
Please consult the TestNG + Cucumber example repository to see a simple example on how to run TestNG & Cucumber tests on TestingBot.
TestNG is a powerful testing framework inspired by JUnit and NUnit but with more flexible and robust features. It is widely used by Java enthusiast for testing because it offers powerful annotations, test configuration and parallel testing.
Cucumber is a popular tool for Behavior-Driven Development (BDD), allowing test scripts to be written in a human-readable format using the Gherkin language. Because Cucumber uses this natural-like language, it enables collaboration between technical and non-technical team members.
To get started, let's go through all the necessary steps required to run your first test with TestNG and Cucumber on TestingBot.
Prerequisites
Let's start with making sure Java is available on your system.
For Windows:- You can download Java for Windows from Java.com
- Run the installer and follow the setup wizard to install Java.
Java should already be present on macOS by default.
Set up pom.xml dependencies
Now we can specify which dependencies we need to be able to run a test. Create a new file called pom.xml
and make sure to define at least the following dependencies:
-
io.cucumber:cucumber-java
io.cucumber:cucumber-testng
org.testng:testng
org.seleniumhq.selenium:selenium-java
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<artifactId>testingbot-testng-cucumber</artifactId>
<groupId>com.testingbot</groupId>
<version>1.0-SNAPSHOT</version>
<packaging>jar</packaging>
<name>testingbot_cucumber_testng</name>
<description>An example project to run Cucumber + TestNG tests on TestingBot's browser grid</description>
<dependencies>
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>cucumber-java</artifactId>
<version>7.18.1</version>
</dependency>
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>cucumber-testng</artifactId>
<version>7.18.1</version>
</dependency>
<dependency>
<groupId>org.testng</groupId>
<artifactId>testng</artifactId>
<version>7.10.2</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<version>4.24.0</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.0</version>
<configuration>
<source>1.7</source>
<target>1.7</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.12.4</version>
<configuration>
<parallel>classes</parallel>
<threadCount>40</threadCount>
<redirectTestOutputToFile>false</redirectTestOutputToFile>
</configuration>
</plugin>
</plugins>
</build>
</project>
Project Structure
Now we need to create a folder structure that will contain the .feature
files and step definitions.
Create a folder for the .feature
files:
We can also define directories for the step definitions, driver and the TestingBot runner class:
Define TestingBot driver
Now we can define a custom driver that will run the tests on the TestingBot grid:
package com.testingbotdemo.selenium.cucumber.drivers;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
import java.net.URL;
import java.net.MalformedURLException;
import java.util.HashMap;
import java.util.Map;
public class TestingBotDriver {
public static WebDriver getDriver() {
String username = System.getenv("TESTINGBOT_KEY") == null ? "TESTINGBOT_KEY" : System.getenv("TESTINGBOT_KEY");
String accesskey = System.getenv("TESTINGBOT_SECRET") == null ? "TESTINGBOT_SECRET" : System.getenv("TESTINGBOT_SECRET");
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability("browserName", "chrome");
capabilities.setCapability("browserVersion", "latest");
capabilities.setCapability("platformName", "WIN10");
Map<String, Object> testingBotOptions = new HashMap<>();
testingBotOptions.put("name", "Cucumber Example");
capabilities.setCapability("tb:options", testingBotOptions);
try {
String gridURL = "https://" + username + ":" + accesskey + "@hub.testingbot.com/wd/hub";
return new RemoteWebDriver(new URL(gridURL), capabilities);
} catch (MalformedURLException e) {
e.printStackTrace();
throw new RuntimeException("Invalid URL provided for TestingBot hub", e);
}
}
}
We will need to make sure to later specify the TESTINGBOT_KEY
and TESTINGBOT_SECRET
environment variables to authenticate and run tests on the TestingBot remote browser grid.
The DesiredCapabilities
are used to define which remote browser we would like to run this Cucumber test on.
Test Runner
Next, let's create a Test runner class that will use the CucumberOptions
annotation to let Cucumber know where to find the feature files and the step definitions.
package com.testingbotdemo.selenium.cucumber.runners;
import io.cucumber.testng.AbstractTestNGCucumberTests;
import io.cucumber.testng.CucumberOptions;
@CucumberOptions(
features = "src/test/resources/features",
glue = "com.testingbotdemo.selenium.cucumber.definitions"
)
public class TestRunner extends AbstractTestNGCucumberTests {
}
Step Definitions
Finally, we need to define the step definitions. This is the logic that converts the Gherkin syntax to actual code.
The code that we will use will interact with the Driver class we defined.
package com.testingbotdemo.selenium.cucumber.definitions;
import io.cucumber.java.en.Given;
import io.cucumber.java.en.Then;
import io.cucumber.java.en.When;
import org.openqa.selenium.By;
import org.testng.Assert;
import com.testingbotdemo.selenium.cucumber.drivers.TestingBotDriver;
import org.openqa.selenium.WebDriver;
public class StepDefinitions {
WebDriver driver = TestingBotDriver.getDriver();
@Given("I open Google")
public void i_open_google() {
driver.get("https://www.google.com");
}
@When("I search for {string}")
public void i_search_for(String query) {
driver.findElement(By.name("q")).sendKeys(query);
driver.findElement(By.name("btnK")).submit();
}
@Then("I should see results")
public void i_should_see_results() {
Assert.assertTrue(driver.getTitle().contains("Google"));
driver.quit();
}
}
Run Cucumber test
You can now run the Cucumber test:
The test will run on a remote TestingBot browser. Once the test has finished, you will be able to see the result, together with a video and logs in the TestingBot dashboard.
Specify Browsers & Devices
To let TestingBot know on which browser/platform/device you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field.
Testing Internal Websites
We've built TestingBot Tunnel, to provide you with a secure way to run tests against your staged/internal webapps.
Please see our TestingBot Tunnel documentation for more information about this easy to use tunneling solution.
The example below shows how to easily run a TestNG test with our Tunnel:
1. Download our tunnel and start the tunnel:
2. Adjust your test: instead of pointing to 'hub.testingbot.com/wd/hub'
like the example above - change it to point to your tunnel's IP address.
Assuming you run the tunnel on the same machine you run your tests, change to 'localhost:4445/wd/hub'
. localhost is the machine running the tunnel, 4445 is the default port of the tunnel.
This way your test will go securely through the tunnel to TestingBot and back:
package com.testingbotdemo.selenium.cucumber.drivers;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
import java.net.URL;
import java.net.MalformedURLException;
import java.util.HashMap;
import java.util.Map;
public class TestingBotDriver {
public static WebDriver getDriver() {
String username = System.getenv("TESTINGBOT_KEY") == null ? "TESTINGBOT_KEY" : System.getenv("TESTINGBOT_KEY");
String accesskey = System.getenv("TESTINGBOT_SECRET") == null ? "TESTINGBOT_SECRET" : System.getenv("TESTINGBOT_SECRET");
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability("browserName", "chrome");
capabilities.setCapability("browserVersion", "latest");
capabilities.setCapability("platformName", "WIN10");
Map<String, Object> testingBotOptions = new HashMap<>();
testingBotOptions.put("name", "Cucumber Example");
capabilities.setCapability("tb:options", testingBotOptions);
try {
String gridURL = "http://" + username + ":" + accesskey + "@localhost:4445/wd/hub";
return new RemoteWebDriver(new URL(gridURL), capabilities);
} catch (MalformedURLException e) {
e.printStackTrace();
throw new RuntimeException("Invalid URL provided for TestingBot hub", e);
}
}
}
Run tests in Parallel
Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time.
You can run the same tests on all different browser configurations or run different tests all on the same browser configuration.
TestingBot has a large grid of machines and browsers, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time.
To run a test on different browsers at the same time, you can run the same mvn test
command multiple times in parallel, each with a different browser/capability environment variable.
For example, assuming you define the desired capabilities with environment variables:
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability("browserName", System.getenv("CAPABILITY_BROWSERNAME"));
capabilities.setCapability("browserVersion", System.getenv("CAPABILITY_BROWSERVERSION"));
capabilities.setCapability("platformName", System.getenv("CAPABILITY_PLATFORMNAME"));
Now you can run the same test on multiple browsers simultaneously:
mvn test -DCAPABILITY_BROWSERNAME=chrome -DCAPABILITY_BROWSERVERSION=latest -DCAPABILITY_PLATFORMNAME=WIN10
mvn test -DCAPABILITY_BROWSERNAME=safari -DCAPABILITY_BROWSERVERSION=latest -DCAPABILITY_PLATFORMNAME=SONOMA
Queuing
Every plan we provide comes with a limit of parallel tests.
If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available.
Mark tests as passed/failed
To see if a test passed or not in our member area, or to send additional meta-data to TestingBot, you can use the TestingBot API.
TestingBot has a Java client to facilitate interacting with the TestingBot API.
Once you've included this client with your tests, you will be able to send back the test status and other meta-data to TestingBot:
import com.testingbot.testingbotrest.TestingbotREST;
@After
public void tearDown() throws Exception {
TestingbotREST r = new TestingbotREST("key", "secret");
Map<String, String> data = new HashMap<String, String>();
data.put("success", "1");
data.put("name", "My Test");
r.updateTest(driver.getSessionId(), data);
driver.quit();
}
Other Java Framework examples
-
JUnit
JUnit is a unit testing framework for the Java programming language.
-
Parallel JUnit
By running multiple JUnit tests at the same time you can cut down on overall test time.
-
TestNG
TestNG is a framework similar to JUnit and NUnit, which supports some additional commands and features.