Exit course
Chapter 8 of 12 Intermediate 6 min read 2 quick checks Updated

Shadow DOM and iframes

On this page
  1. Shadow DOM
  2. iframes
  3. Locating the frame itself
  4. Summary

Two kinds of boundary stop a normal locator from finding an element that is plainly visible on the page: a shadow root, used by web components to keep their internals private, and an iframe, which embeds a separate document. Selenium makes you cross both explicitly; Playwright crosses shadow roots for you and needs one extra call for frames.

Shadow DOM#

A web component attaches a shadow root to its host element and renders its markup inside it. The Selenium project's test page https://www.selenium.dev/selenium/web/webComponents.html has one:

<custom-checkbox-element>
  #shadow-root (open)
    <div><input type="checkbox"/>
    <a href="simpleTest.html">Example Link</a>
    </div>
</custom-checkbox-element>

document.querySelectorAll('input') on that page returns nothing, and so do Selenium's find_element and the DevTools $$ helper. The checkbox lives in a separate tree.

Selenium: get the shadow root, then search inside it#

The Selenium tests for this chapter go in tests/test_shadow_and_frames.py, the Playwright ones in tests/shadow-and-frames.spec.ts.

import pytest
from selenium.common.exceptions import InvalidArgumentException, InvalidSelectorException
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait

COMPONENTS = "https://www.selenium.dev/selenium/web/webComponents.html"
IFRAMES = "https://www.selenium.dev/selenium/web/iframes.html"
NESTED = "https://the-internet.herokuapp.com/nested_frames"


def test_shadow_root(driver):
    driver.get(COMPONENTS)
    assert driver.find_elements(By.CSS_SELECTOR, "input[type='checkbox']") == []

    host = driver.find_element(By.CSS_SELECTOR, "custom-checkbox-element")
    root = host.shadow_root
    checkbox = root.find_element(By.CSS_SELECTOR, "input[type='checkbox']")
    checkbox.click()
    assert checkbox.is_selected()


def test_no_xpath_inside_a_shadow_root(driver):
    driver.get(COMPONENTS)
    root = driver.find_element(By.CSS_SELECTOR, "custom-checkbox-element").shadow_root
    with pytest.raises((InvalidArgumentException, InvalidSelectorException)):
        root.find_element(By.XPATH, ".//input")

shadow_root (Selenium 4, getShadowRoot() in Java) returns an object with find_element and find_elements. Inside it, use CSS: the shadow root is a document fragment, and XPath queries are rejected. Chrome answers with InvalidArgumentException ("invalid locator") and Firefox with InvalidSelectorException, hence the two exception types in the test. By.TAG_NAME failed inside the shadow root in both browsers in our runs too; write By.CSS_SELECTOR, "a" instead. Selenium sends By.ID, By.NAME and By.CLASS_NAME as CSS, so they work.

For a component nested inside another component, repeat the step: find the inner host inside the outer root, then take its shadow_root. A small helper that walks a list of host selectors keeps tests readable when a design system nests components several levels deep.

Closed shadow roots (mode: 'closed') are not reachable through shadow_root at all. If a component you need to test uses one, ask for test hooks on the host element or test the component on its own.

Playwright: CSS and role locators pierce open shadow roots#

import { test, expect } from './fixtures';

test('CSS and role locators pierce open shadow roots', async ({ page }) => {
  await page.goto('https://www.selenium.dev/selenium/web/webComponents.html');
  await page.getByRole('checkbox').check();
  await expect(page.locator('custom-checkbox-element input[type="checkbox"]')).toBeChecked();
  await expect(page.getByRole('link', { name: 'Example Link' })).toBeVisible();
});

test('XPath does not pierce shadow roots', async ({ page }) => {
  await page.goto('https://www.selenium.dev/selenium/web/webComponents.html');
  await expect(page.locator('xpath=//input')).toHaveCount(0);
  await expect(page.locator('input')).toHaveCount(1);
});

Playwright's CSS engine and its getBy... locators search open shadow roots automatically, so the test does not need to know a component is involved. Its XPath engine does not, which is one more reason to keep XPath for the cases in chapter 4.

A Selenium test fails with NoSuchElementException for input#search, but DevTools shows the input inside <app-header> under #shadow-root (open). What fixes it?

The input is in the shadow tree, which document-level searches do not enter, however long you wait. XPath cannot cross the boundary either. A shadow root is not a frame, so switch_to.frame does not apply.

iframes#

An iframe holds a complete second document. A WebDriver session looks at one document at a time, so Selenium has to switch into the frame, and back out again.

def test_iframe(driver):
    driver.get(IFRAMES)
    WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it("iframe1"))
    driver.find_element(By.ID, "email").send_keys("me@example.com")
    driver.switch_to.default_content()
    assert driver.find_element(By.ID, "iframe_page_heading").text == "This is the heading"
  • frame_to_be_available_and_switch_to_it accepts the frame's id or name, a locator tuple, or an element, and waits until the frame has loaded before switching. Prefer it over a bare switch_to.frame(), which fails if the iframe is not there yet.
  • After switching, every find_element searches inside the frame, and elements of the outer page cannot be found until you switch back with default_content().
  • Element references do not cross frames. An element found inside the frame cannot be used after switching out.

Nested frames are entered one level at a time. the-internet.herokuapp.com/nested_frames is a frameset with a frameset inside it:

def test_nested_frames(driver):
    driver.get(NESTED)
    driver.switch_to.frame("frame-top")
    driver.switch_to.frame("frame-middle")
    assert driver.find_element(By.ID, "content").text == "MIDDLE"
    driver.switch_to.parent_frame()
    driver.switch_to.frame("frame-left")
    assert "LEFT" in driver.find_element(By.TAG_NAME, "body").text

parent_frame() goes up one level; default_content() goes all the way to the top. A test that forgets to switch back leaves every later lookup searching the wrong document, so put the switching inside a page object method or a context manager rather than scattering it through tests.

Playwright does not switch at all. A frame locator scopes everything after it:

test('locators inside an iframe', async ({ page }) => {
  await page.goto('https://www.selenium.dev/selenium/web/iframes.html');
  const form = page.frameLocator('#iframe1');
  await form.locator('#email').fill('me@example.com');
  await expect(form.locator('#email')).toHaveValue('me@example.com');
  await expect(page.locator('#email')).toHaveCount(0);
});

test('nested frames', async ({ page }) => {
  await page.goto('https://the-internet.herokuapp.com/nested_frames');
  const middle = page
    .frameLocator('frame[name="frame-top"]')
    .frameLocator('frame[name="frame-middle"]');
  await expect(middle.locator('#content')).toHaveText('MIDDLE');
});

page.locator('#email') still searches only the top document, which is why it finds nothing: the frame locator is explicit, but you never have to remember to switch back. All getBy... methods are available on a frame locator, so form.getByRole('button', { name: 'Submit' }) works as you would expect.

A Selenium test fills a payment form inside an iframe, then fails to find the "Order summary" heading on the main page, although it is visible. What is missing?

After switching into a frame, every search runs inside that frame's document until you switch back. Waiting longer does not help, and refreshing loses the form you just filled.

Locating the frame itself#

The frame element needs a locator too. Prefer, in order: a name or id you control, a title attribute (which iframes should have for accessibility anyway, and which Playwright can use via page.frameLocator('iframe[title="Payment"]')), and only then a src match such as iframe[src*="checkout"]. Avoid "the second iframe on the page": third-party scripts such as chat widgets and analytics often inject their own iframes.

Summary#

  • Selenium: element.shadow_root and CSS inside it; no XPath in shadow trees; closed roots are out of reach.
  • Playwright: CSS and getBy... pierce open shadow roots; XPath does not.
  • Selenium: switch into frames with a wait, one level at a time, and always switch back.
  • Playwright: frameLocator() scopes locators to a frame without any switching.