Exit course
Chapter 4 of 11 Intermediate 8 min read 2 quick checks Updated

Writing Tests That Survive Engine Differences

On this page
  1. Keyboard shortcuts belong to the operating system
  2. Native form controls accept input differently
  3. Assert on meaning, not on pixels
  4. Wait for the state, not for the clock
  5. Pin the locale and the time zone
  6. When a browser really cannot do it
  7. Summary

Most failures that appear in only one browser are not bugs in the application. They are tests that quietly depended on one browser's behaviour: a keyboard shortcut, a pixel width, the way a native control accepts typing, or how fast the page happens to be. This chapter goes through the habits that keep a suite honest across engines, with examples that we ran on Chromium, Firefox and WebKit.

The examples use Playwright, because it can run all three engines locally with one command. Every point applies equally to Selenium; chapter 8 debugs the Selenium version of the first one.

Keyboard shortcuts belong to the operating system#

Here is a test step that looks harmless: select everything in a text field and type over it.

await editor.press('Control+A');
await editor.pressSequentially('new');

On Windows and Linux, Control+A selects all. On macOS it does not: in a text field it moves the cursor to the start of the line, a keybinding inherited from Emacs. We ran this step on a Mac against a field containing old text:

Engine (on macOS) Value after Control+A, then typing new
Chromium newold text
WebKit newold text
Firefox old textnew

Three engines, two different wrong answers, and the same test passes on every Windows machine. Playwright has a modifier for exactly this case: ControlOrMeta presses Control on Windows and Linux and Command (Meta) on macOS. This is the test from the course project, run against the TodoMVC demo:

test('edits a todo with select-all', async ({ page }) => {
  const input = page.getByPlaceholder('What needs to be done?');
  await input.fill('Buy milk');
  await input.press('Enter');

  await page.getByTestId('todo-title').dblclick();
  const editor = page.getByRole('textbox', { name: 'Edit' });
  await editor.press('ControlOrMeta+A');
  await editor.pressSequentially('Buy oat milk');
  await editor.press('Enter');

  await expect(page.getByTestId('todo-title')).toHaveText('Buy oat milk');
});

It passes on Chromium, Firefox and WebKit on a Mac, and on Chrome, Edge and Firefox on Windows 11 and Safari on macOS on the grid (chapter 5). Better still, ask whether you need the shortcut at all: if the intent is "replace the text", editor.fill('Buy oat milk') says that directly and does not depend on any keybinding.

A test uses press('Control+A') and then types. It passes in CI on Linux and fails on a developer's Mac in every browser. What is the best fix?

The failure is deterministic: Control+A is not select-all on macOS. Replacing text is what fill() is for, and ControlOrMeta picks the right modifier per platform when the shortcut matters. Skipping hides it from your Mac users.

Native form controls accept input differently#

Date pickers, colour pickers, file inputs and select boxes are drawn by the browser, and each engine decides how keystrokes reach them. We typed the digits 11122026 into an empty <input type="date"> on all three engines:

Engine Resulting value
Chromium empty
WebKit empty
Firefox 2026-11-12

A test that types into a date field is testing the browser's date picker, not your form. Set the value the way the HTML standard defines it instead: fill() on a date input takes the ISO format yyyy-mm-dd on every engine.

test('fills a native date input', async ({ page }) => {
  await page.setContent(`
    <label>Departure <input type="date" name="departure"></label>
    <output id="echo"></output>
    <script>
      document.querySelector('input').addEventListener('change', e => {
        document.querySelector('#echo').textContent = e.target.value;
      });
    </script>`);

  // fill() takes the ISO value on every engine; typing digits does not.
  await page.getByLabel('Departure').fill('2026-11-12');

  await expect(page.getByLabel('Departure')).toHaveValue('2026-11-12');
  await expect(page.locator('#echo')).toHaveText('2026-11-12');
});

The second assertion matters: it proves the page received a change event, not only that the value was set. In Selenium there is no fill() for dates, so set the value with JavaScript and dispatch the events your app listens for, or, if your users type dates, test your own date component rather than the native one.

Assert on meaning, not on pixels#

Chapter 1 measured a 15-pixel scrollbar in Chrome on Windows, none in Firefox on Windows 11 and 17 pixels in Safari on a Mac. Fonts differ too, so the same text wraps at different widths. Assertions like these break across browsers without anything being wrong:

  • the sidebar is exactly 280 pixels wide,
  • the button's y coordinate is 412,
  • the heading's text height is one line.

Ask what the user actually needs and assert that: the button is visible and enabled, the menu is in the viewport, the error message has the right text, the element is not covered by another one (Playwright's actions and Selenium's click() already fail if something else would receive the click). For "it looks right", use visual comparisons with one baseline per browser, which chapter 9 introduces.

Locators follow the same rule. Role and label locators (getByRole, getByLabel, or Selenium's By.ID on stable ids) describe the page the same way in every engine. Long XPath expressions that walk the DOM tree can break when an engine inserts an extra text node or a browser-generated wrapper.

A test asserts that a card is exactly 320 pixels wide. It passes in Chrome on Windows and fails in Firefox on Windows 11 at 335 pixels. What is the most likely cause?

Firefox on Windows 11 uses overlay scrollbars and reserves no width, while Chrome reserves 15 pixels, so content widths inside a scrolling container differ by about that much. The test should assert on what the user needs, not on an exact width.

Wait for the state, not for the clock#

Browsers are not equally fast, and the same browser is slower on a phone than on a desktop. A test that looks for an element immediately after a click passes when the page happens to be fast and fails when it is not. In our runs, a version of the login test without an explicit wait passed on desktop browsers and failed on a real iPhone with NoSuchElementException.

The fix is never a longer sleep. In Playwright, web-first assertions (await expect(locator).toHaveText(...)) retry until the state appears. In Selenium, wait for the condition you need:

flash = WebDriverWait(driver, 10).until(
    EC.visibility_of_element_located((By.ID, "flash")))
assert "You logged into a secure area!" in flash.text

Pin the locale and the time zone#

Remote browsers do not run in your time zone. Every desktop browser in chapter 1's probe reported UTC. A test that checks a formatted date ("Thursday, 10:00") passes on your laptop in Berlin and fails on the grid. Make the environment explicit instead of inheriting it:

  • In Playwright, set locale and timezoneId in the project's use block.
  • On TestingBot, the timezone option sets the time zone of the remote machine for Selenium sessions.
  • Or assert on values that do not depend on either, such as the ISO date in a datetime attribute.

When a browser really cannot do it#

Sometimes an engine genuinely lacks a feature, and the application handles that with a fallback. Then the test should say so explicitly, with a reason, instead of hiding a browser check inside an assertion:

test('shares the page with the Web Share API', async ({ page, browserName }) => {
  test.skip(browserName === 'firefox', 'Web Share API is not available in desktop Firefox; fallback tested separately');
  // ...
});

Rules of thumb for these conditions:

  • Branch on the feature, in the application, with feature detection ('share' in navigator), never on the user agent.
  • Branch on the browser only in the test, only with a written reason, and keep a separate test for the fallback path.
  • For a known engine bug you are waiting on, test.fixme() with a link to the bug report keeps it visible instead of deleted.

Summary#

  • Keyboard shortcuts differ by OS: prefer fill(), use ControlOrMeta when the shortcut matters.
  • Native controls take input differently: set ISO values instead of typing into them.
  • Assert on what the user needs, not on pixel positions or sizes.
  • Wait for states with retrying assertions or explicit waits; never sleep.
  • Make locale and time zone explicit; remote browsers run in UTC unless told otherwise.
  • Skip per browser only with a reason, and test the fallback.