Run Selenium tests in parallel

A suite of 200 UI tests that each take 20 seconds needs over an hour on one thread. Spread across 10 browsers it needs a few minutes. TestNG makes the threading part a one-line change in testng.xml; making your tests safe to run on those threads is the real work, and it comes down to one rule: every thread gets its own WebDriver, and nothing else mutable is shared.

The parallel attribute#

<suite name="Course suite" parallel="methods" thread-count="3">

parallel decides what TestNG is allowed to run at the same time:

thread-count is the size of the thread pool (default 5). It caps how many things run at once regardless of how many are eligible. Parallel data providers (chapter 5) use a separate pool sized by data-provider-thread-count (default 10), so a suite with thread-count="3" and a parallel data provider can have more than three browsers open at once. Size both with that in mind.

Methods connected by dependsOnMethods are still run in dependency order; TestNG holds the dependent method back until its dependency has finished.

Why the chapter 3 BaseTest breaks#

Our BaseTest stores the driver in an instance field. With parallel="methods", TestNG runs several methods of the same instance on different threads at once. Thread A's @BeforeMethod sets driver, thread B's @BeforeMethod overwrites it, both tests now drive B's browser, and whichever finishes first quits it under the other. The symptoms look random: NoSuchSessionException, elements typed into the wrong page, assertions that fail on another test's data.

Making the field static makes it worse: then every instance of every class shares it.

One driver per thread with ThreadLocal#

A ThreadLocal holds a separate value for each thread. Each test thread sets and reads its own driver, and never sees another thread's:

package com.example;

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.firefox.FirefoxOptions;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Optional;
import org.testng.annotations.Parameters;

public abstract class BaseTest {

    private static final ThreadLocal<WebDriver> DRIVER = new ThreadLocal<>();

    public static WebDriver driver() {
        return DRIVER.get();
    }

    @BeforeMethod(alwaysRun = true)
    @Parameters("browser")
    public void startDriver(@Optional("chrome") String browser) {
        DRIVER.set(createDriver(browser));
    }

    @AfterMethod(alwaysRun = true)
    public void stopDriver() {
        WebDriver driver = DRIVER.get();
        if (driver != null) {
            try {
                driver.quit();
            } finally {
                DRIVER.remove();
            }
        }
    }

    private static WebDriver createDriver(String browser) {
        boolean headless = Boolean.getBoolean("headless");
        if ("firefox".equals(browser)) {
            FirefoxOptions options = new FirefoxOptions();
            if (headless) options.addArguments("-headless");
            return new FirefoxDriver(options);
        }
        ChromeOptions options = new ChromeOptions();
        if (headless) options.addArguments("--headless=new", "--window-size=1920,1080");
        return new ChromeDriver(options);
    }
}

Three details make this robust:

1. The ThreadLocal is static, the driver inside it is per thread. One ThreadLocal object for the whole JVM, one value per thread.
2. @BeforeMethod and @AfterMethod run on the same thread as the test method. That is what guarantees the test reads the driver its own setup created. (TestNG runs a method's configuration methods on the thread that runs the method.)
3. DRIVER.remove() after quit(). TestNG reuses pool threads. Without remove(), a thread keeps pointing at the previous test's dead session. If the next setup on that thread fails before it calls set(), teardown and listeners act on that stale driver and bury the real setup error under a NoSuchSessionException.

driver() is now public static, which the screenshot listener in the next chapter will need. Because every test already calls driver() (chapter 3), none of the test classes change. If you kept a waitUpTo helper, have it call driver() instead of reading a field.

Try it#

Update testng.xml:

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Course suite" parallel="methods" thread-count="3" data-provider-thread-count="2">
  <test name="Chrome">
    <parameter name="browser" value="chrome"/>
    <classes>
      <class name="com.example.FirstTest"/>
      <class name="com.example.FormDefaultsTest"/>
      <class name="com.example.WebFormDataTest"/>
    </classes>
  </test>
</suite>

Switch the textInputs provider from chapter 5 to parallel = true, then run mvn test -Dheadless=true. Several Chrome processes start at once and the total time drops. Add System.out.println(Thread.currentThread().getName()) to a test if you want to watch the threads.

Everything else that must not be shared#

The driver is the obvious shared resource. These are the less obvious ones that cause the "only fails in parallel" bugs:

• SoftAssert fields. Create one per test method (chapter 4).
• Page objects stored in fields. A page object wraps a driver. Create it inside the test from driver(), or store it in a ThreadLocal too.
• Test data. Two threads logging in as the same user can log each other out, and two threads editing the same record race each other. Give each test its own data, for example a user created through an API in setup or a unique suffix such as "user-" + UUID.randomUUID().
• Static helpers with state. A static WebDriverWait, SimpleDateFormat or counter is shared by every thread.
• Files. Screenshots or downloads written to a fixed file name overwrite each other; include the test name and a timestamp in the path.

Choosing thread-count#

More threads is not always faster.

• Locally, each Chrome instance needs CPU and memory. Past a point, extra browsers make every test slower, explicit waits begin to time out, and the suite becomes flaky instead of fast. Start near the number of CPU cores and measure.
• On a remote grid, the browsers run elsewhere, so your machine only coordinates. The limit becomes the number of parallel sessions your grid account allows. On TestingBot, the docs state that sessions beyond your plan's limit are queued (for up to 6 minutes) and start as soon as a slot frees up, so setting thread-count slightly above your limit is harmless, but far above it only produces queue time. Our parallel testing overview and the parallel calculator help estimate how many slots a suite actually needs.

Parallel by tests for cross-browser runs#

parallel="tests" is the natural fit when each <test> block targets a different browser:

<suite name="Cross-browser" parallel="tests" thread-count="2">
  <test name="Chrome">
    <parameter name="browser" value="chrome"/>
    <packages><package name="com.example"/></packages>
  </test>
  <test name="Firefox">
    <parameter name="browser" value="firefox"/>
    <packages><package name="com.example"/></packages>
  </test>
</suite>

Chrome and Firefox run side by side, while the tests inside each block run one after another. Our ThreadLocal base class works unchanged in every mode, which is exactly why it is worth setting up even before you need parallelism.

If your team uses JUnit 5 rather than TestNG, the same driver-per-thread idea applies; the parallel JUnit example shows it.