Hard assertions vs SoftAssert

An assertion is where a test stops describing steps and starts checking outcomes. TestNG gives you two styles: hard assertions that stop the test at the first failure, and soft assertions that collect failures and report them together. Using the right one in the right place makes failures much faster to diagnose.

Hard assertions with org.testng.Assert#

org.testng.Assert has static methods that throw AssertionError as soon as a check fails. The rest of the test method does not run.

import org.testng.Assert;

Assert.assertEquals(driver().getTitle(), "Web form");
Assert.assertNotEquals(cartCount, 0);
Assert.assertTrue(button.isEnabled(), "Submit should be enabled once the form is valid");
Assert.assertFalse(error.isDisplayed());
Assert.assertNull(cookie);
Assert.assertNotNull(driver().findElement(By.id("message")));
Assert.assertEqualsNoOrder(actualTags, List.of("new", "sale"));   // collections, any order

Argument order: actual first#

TestNG's signature is assertEquals(actual, expected), the reverse of JUnit's assertEquals(expected, actual). TestNG builds its message from that order:

expected [Received!] but found [Error]

If you pass the arguments the JUnit way, the test still fails at the right moment, but the message tells you the page showed your expected value and you expected what the page actually showed. Under pressure at 2am, that costs real time. If your team moves between JUnit and TestNG, consider AssertJ (assertThat(actual).isEqualTo(expected)), which reads the same in both.

TestNG also ships org.testng.AssertJUnit with the JUnit argument order, intended for migrating JUnit 3 tests. Don't mix both classes in one codebase.

Always add a message to boolean checks#

assertTrue(x) fails with expected [true] but found [false], which tells you nothing. The overload with a message is free:

Assert.assertTrue(checkbox.isSelected(), "Newsletter checkbox should be pre-selected");

Checking for exceptions#

assertThrows and expectThrows take a lambda:

import org.openqa.selenium.NoSuchElementException;

Assert.assertThrows(NoSuchElementException.class,
        () -> driver().findElement(By.id("does-not-exist")));

For UI tests, prefer findElements(...).isEmpty() to assert absence. It is cheaper and does not depend on exception behaviour.

Soft assertions with SoftAssert#

Some tests verify many independent facts about one page: the default state of a form, every field of an order summary, the labels of a navigation menu. With hard assertions, the first wrong value hides all the others, and you fix and rerun one at a time. org.testng.asserts.SoftAssert records every failure and throws them together when you call assertAll():

package com.example;

import org.openqa.selenium.By;
import org.testng.annotations.Test;
import org.testng.asserts.SoftAssert;

public class FormDefaultsTest extends BaseTest {

    @Test
    public void formDefaults() {
        driver().get("https://www.selenium.dev/selenium/web/web-form.html");

        SoftAssert softly = new SoftAssert();
        softly.assertTrue(driver().findElement(By.id("my-check-1")).isSelected(),
                "first checkbox should be checked");
        softly.assertFalse(driver().findElement(By.id("my-check-2")).isSelected(),
                "second checkbox should be unchecked");
        softly.assertFalse(driver().findElement(By.name("my-disabled")).isEnabled(),
                "disabled input should be disabled");
        softly.assertEquals(
                driver().findElement(By.name("my-readonly")).getDomAttribute("value"),
                "Readonly input", "read-only input value");
        softly.assertAll();
    }
}

If two checks fail, the test fails once with both listed:

The following asserts failed:
	first checkbox should be checked expected [true] but found [false],
	disabled input should be disabled expected [false] but found [true]

The three SoftAssert mistakes#

1. Forgetting assertAll(). Without it, failures are recorded and then silently discarded, and the test passes. Make assertAll() the last line of every method that creates a SoftAssert, and grep for it in code review.
2. Sharing one SoftAssert as a class field. A SoftAssert accumulates every failure it has ever seen. As a field it carries failures from one test into the next, and under parallel execution several threads write into it at once. Create a new instance inside each test method.
3. Using soft assertions for preconditions. If the login failed, there is no point soft-checking ten things on the dashboard; each of them will fail with a confusing NoSuchElementException that is not even recorded as an assertion. Hard-assert the precondition, then soft-assert the details:

Assert.assertEquals(driver().getTitle(), "Web form", "precondition: on the form page");

SoftAssert softly = new SoftAssert();
// ... independent checks ...
softly.assertAll();

A useful rule of thumb: hard assertions for "can the test continue?", soft assertions for "what exactly is wrong with this page?"

Waiting and asserting are different steps#

A frequent source of flaky failures is asserting on something that has not finished rendering:

// Flaky: the message may not be there yet
Assert.assertEquals(driver().findElement(By.id("message")).getText(), "Received!");

Wait for the state first, then assert on it:

import org.openqa.selenium.support.ui.ExpectedConditions;

String text = waitUpTo(10)
        .until(ExpectedConditions.visibilityOfElementLocated(By.id("message")))
        .getText();
Assert.assertEquals(text, "Received!");

When the text itself changes asynchronously (a counter, a status line), wait for the expected text with ExpectedConditions.textToBePresentInElementLocated(...), and keep a plain assertion after it so the report shows a readable expected [...] but found [...] rather than a timeout.

Which value to assert on#

Selenium gives you several ways to read an element, and they differ:

For an input the user has typed into, getDomProperty("value") is the one that reflects the typed text; getDomAttribute("value") still returns the original HTML attribute.

What to assert, and what not to#

Assert on what a user would notice: visible text, URL changes, enabled or disabled controls, items in a list. Avoid asserting on CSS class names, inline styles or exact pixel positions unless they are the feature under test; they change with every redesign and produce failures that are not bugs. For visual regressions, screenshot comparison does a better job than assertions.

The TestNG with Selenium article lists the full set of Assert methods, and "hard vs soft assertions" is a staple of TestNG interview questions if you are preparing for one.

Next, we will stop hard-coding input values and let a data provider feed the same test many times.