TestNG Tutorial: Selenium Testing

TestNG: installation and setup

Installing TestNG is pretty straightforward.
If you are using Eclipse IDE, TestNG is already included as a plugin.

Using Eclipse

You can download the Eclipse IDE from the Eclipse website.
Once installed, you can download TestNG from the 'Eclipse Marketplace'.
After installing TestNG, you can right-click your test packages and click the TestNG option.

Using Maven

If you're not using Eclipse or another IDE, you might prefer installing TestNG via Maven:

<dependency>
    <groupId>org.testng</groupId>
    <artifactId>testng</artifactId>
    <version>7.1.0</version>
</dependency>

TestNG: annotations

Annotations are popular amongst Java developers and are used to provide more information about a method, class or snippet of code.

TestNG is able to parse the annotations and make decisions regarding test execution.
Some annotations add meta-data to the test, others are triggers that can be run during the lifecycle of a test.

To use annotations, you'll need to import the org.testng.annotations.* package.

@test annotation

This is the most popular annotation and is required when running tests via TestNG.
When TestNG is started, it will look for all methods that have the @test annotation.
By default, TestNG will run all tests with the @test annotation in alphabetical order.

@Test
public void sampleTest()
{
	System.out.println("TestNG will run this");
}

public void noAnnotationTest()
{
	System.out.println("TestNG will NOT run this - as it is lacking the @test annotation");
}

@test attributes

The @test annotation comes with some attributes that you can use:

• Description: Describe what the test does, this is useful when running tests to see what a specific test does without interpreting the code. It is also included in the test report.
  
  

  @Test(description="This tests validates the pricing page")

• Priority: Defines priority of each test. By default, tests are run in alphabetical order. You can define a different priority to modify the test order. TestNG will run the tests with the lowest priority value up to the largest.
  
  

  @Test(priority=1)

• DependsOnMethod: This instructs TestNG to run a specific method before running the test case. For example, if your test depends on being logged in, you can use this to first do the login flow.
  
  

  @Test(dependsOnMethod="loginFlow")

• Enabled: Allows you to disable a specific test. Perhaps you want to temporary disable a flaky test by setting this to false.
  
  

  @Test(enabled='true')

• Groups: Group multiple tests together. You can then instruct TestNG to only run test cases for a specific group.
  
  

  @Test(groups="Fast Suite")

other annotations

TestNG also supports other annotations, such as:

• @BeforeTest and @AfterTest:
  

  These methods will be executed either prior to the first test, or after all test cases are completed in the same TestNG file.

• @BeforeMethod and @AfterMethod:
  

  These methods will be executed prior or after each method in each test case.

• @BeforeClass and @AfterClass:
  

  These methods will be executed once before the first test case, or once after the last test case in the TestNG file.

• @BeforeSuite and @AfterSuite:
  

  These methods will be executed once before all tests in the suite, or once after all tests in this suite have completed.

The execution order for these annotations is:

TestNG: your first test

Writing your first TestNG is easy. Simple create a new file in your favorite IDE with the following contents:

import java.net.URL;

import org.openqa.selenium.By;
import org.openqa.selenium.Platform;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.testng.Assert;
import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;

public class TestingBotTest {
    private WebDriver driver;

    @BeforeClass
    public void setUp() throws Exception {
        DesiredCapabilities capabillities = DesiredCapabilities.firefox();
        capabillities.setCapability("version", "latest");
        capabillities.setCapability("platform", Platform.WINDOWS);
        capabillities.setCapability("name", "Testing Selenium");

        driver = new RemoteWebDriver(
                new URL("https://key:secret@hub.testingbot.com/wd/hub"), capabillities);
    }

    @Test(description="This will search Google for TestingBot")
    public void testSimple() throws Exception {
        driver.get("https://www.google.com/ncr");
        WebElement element = driver.findElement(By.name("q"));
        element.sendKeys("TestingBot");
        element.submit();
        Thread.sleep(5000);

        Assert.assertEquals("TestingBot - Google Search", driver.getTitle());
    }

    @AfterMethod
		public void printUrl()
		{
			System.out.println(driver.getCurrentUrl());
		}

    @AfterClass
    public void tearDown() throws Exception {
        driver.quit();
    }
}

As you can see in the example above, we're using the following annotations:

• @BeforeClass annotation to create a WebDriver connection to the TestingBot grid.
• Once that's established, TestNG will run all methods annotated with @Test.
• The @AfterMethod annotation will output the current URL of the TestingBot browser.
• Finally, TestNG will look for the @AfterClass method and stop the driver connection (close browser).

The description attribute in the @test annotation will also show up in the TestNG report and console output.

TestNG will automatically create a XML file:

<?XML version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite">
	<test name="Test">
		<classes>
			<class name=".TestingBotTest"/>
		</classes>
	</test>
</suite>

You can customize this XML file with multiple class names containing test cases.

TestNG: reports

TestNG is able to generate reports in HTML format.
By default, it will generate a HTML file with information about your test run.

Failed test cases will show in red, test cases that passed will show up in green.
Additional details, such as the description of the test, how long each test took and more will be included in the report.

ReportNG is a TestNG plugin which offers a more comprehensive HTML report.

Add logging to your reports

If you're looking to add log output to your reports, you can use the TestNG Reporter Class.
With the Reporter Class you store information with these methods:

• Reporter.log(String s);
• Reporter.log(String s, Boolean logToStandardOut);
• Reporter.log(String s, int level);
• Reporter.log(String s, int level, Boolean logToStandardOut);

For example, in your test case you can write to the report:

import java.net.URL;

import org.openqa.selenium.By;
import org.openqa.selenium.Platform;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.testng.Assert;
import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;

public class TestingBotTest {
    private WebDriver driver;

    @BeforeClass
    public void setUp() throws Exception {
        DesiredCapabilities capabillities = DesiredCapabilities.firefox();
        capabillities.setCapability("version", "latest");
        capabillities.setCapability("platform", Platform.WINDOWS);
        capabillities.setCapability("name", "Testing Selenium");

        driver = new RemoteWebDriver(
                new URL("https://key:secret@hub.testingbot.com/wd/hub"), capabillities);
    }

    @Test(description="This will search Google for TestingBot")
    public void testSimple() throws Exception {
        driver.get("https://www.google.com/ncr");
        Reporter.log("Navigate to Google");
        WebElement element = driver.findElement(By.name("q"));
        Reporter.log("Type in TestingBot");
        element.sendKeys("TestingBot");
        Reporter.log("Submit the form");
        element.submit();
        Thread.sleep(5000);

        Assert.assertEquals("TestingBot - Google Search", driver.getTitle());
    }

    @AfterMethod
		public void printUrl()
		{
			System.out.println(driver.getCurrentUrl());
		}

    @AfterClass
    public void tearDown() throws Exception {
        driver.quit();
    }
}

TestNG: assertions

Similar to JUnit, TestNG comes with multiple assertions to verify results during your test case.
To use assertions, make sure to import the class import org.testng.Assert;.

The most commonly used assertions are:

• assertTrue: This assertion verifies if something returns true.
• assertFalse: This assertion verifies if something returns false.
• assertEquals: This assertion verifies if something is equal to something else.
• assertNotEquals: This assertion verifies if something is not equal to something else.

If an assertion fails, the code after the assertion will no longer execute.

TestNG: parallel testing

TestNG will, by default, run all your tests sequentially.
To speed up your tests (lower your total test duration), you can choose to run tests in parallel.

This is where the power of TestingBot really shines: the TestingBot service allows for running multiple tests simultaneously, drastically reducing the total test duration time.

To run tests in parallel with TestNG, you'll need to modify the TestNG XML file and add a parallel attribute:

<?XML version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite" thread-count="3" parallel="methods">
	<test name="Test">
		<classes>
			<class name=".TestingBotTest"/>
		</classes>
	</test>
</suite>

The parallel attribute accepts the following values:

• methods: All methods annotated with @test will execute in parallel.
• tests: All tests inside a <test> tag will execute in parallel.
• classes: All the test cases inside a Java class will run in parallel.
• instances: Test cases which are in the same instance will execute in parallel. However, two methods of two different instances will run in separate threads.

The thread-count attribute allows you to define how many tests TestNG can run simultaneously.
Usually, this will be the same number as the number of parallel tests assigned to your TestingBot account.

TestNG: advantages

TestNG providers powerful features to make your tests more flexible and cleaner:

• Annotations are very useful and easy to understand.
• TestNG provides a clean way to do parallel testing.
• TestNG provides reports, which is useful for when a test fails.
• Giving priorities to tests and grouping tests are very useful.

If you are looking for more documentation, please see our TestNG automated testing documentation.