Shadow DOM and iframes
On this page
Two kinds of boundary stop a normal locator from finding an element that is plainly visible on the page: a shadow root, used by web components to keep their internals private, and an iframe, which embeds a separate document. Selenium makes you cross both explicitly; Playwright crosses shadow roots for you and needs one extra call for frames.
Shadow DOM#
A web component attaches a shadow root to its host element and renders its markup inside it. The Selenium project's test page https://www.selenium.dev/selenium/web/webComponents.html has one:
<custom-checkbox-element>
#shadow-root (open)
<div><input type="checkbox"/>
<a href="simpleTest.html">Example Link</a>
</div>
</custom-checkbox-element>
document.querySelectorAll('input') on that page returns nothing, and so do Selenium's find_element and the DevTools $$ helper. The checkbox lives in a separate tree.
Selenium: get the shadow root, then search inside it#
The Selenium tests for this chapter go in tests/test_shadow_and_frames.py, the Playwright ones in tests/shadow-and-frames.spec.ts.
import pytest
from selenium.common.exceptions import InvalidArgumentException, InvalidSelectorException
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
COMPONENTS = "https://www.selenium.dev/selenium/web/webComponents.html"
IFRAMES = "https://www.selenium.dev/selenium/web/iframes.html"
NESTED = "https://the-internet.herokuapp.com/nested_frames"
def test_shadow_root(driver):
driver.get(COMPONENTS)
assert driver.find_elements(By.CSS_SELECTOR, "input[type='checkbox']") == []
host = driver.find_element(By.CSS_SELECTOR, "custom-checkbox-element")
root = host.shadow_root
checkbox = root.find_element(By.CSS_SELECTOR, "input[type='checkbox']")
checkbox.click()
assert checkbox.is_selected()
def test_no_xpath_inside_a_shadow_root(driver):
driver.get(COMPONENTS)
root = driver.find_element(By.CSS_SELECTOR, "custom-checkbox-element").shadow_root
with pytest.raises((InvalidArgumentException, InvalidSelectorException)):
root.find_element(By.XPATH, ".//input")
shadow_root (Selenium 4, getShadowRoot() in Java) returns an object with find_element and find_elements. Inside it, use CSS: the shadow root is a document fragment, and XPath queries are rejected. Chrome answers with InvalidArgumentException ("invalid locator") and Firefox with InvalidSelectorException, hence the two exception types in the test. By.TAG_NAME failed inside the shadow root in both browsers in our runs too; write By.CSS_SELECTOR, "a" instead. Selenium sends By.ID, By.NAME and By.CLASS_NAME as CSS, so they work.
For a component nested inside another component, repeat the step: find the inner host inside the outer root, then take its shadow_root. A small helper that walks a list of host selectors keeps tests readable when a design system nests components several levels deep.
Closed shadow roots (mode: 'closed') are not reachable through shadow_root at all. If a component you need to test uses one, ask for test hooks on the host element or test the component on its own.
Playwright: CSS and role locators pierce open shadow roots#
import { test, expect } from './fixtures';
test('CSS and role locators pierce open shadow roots', async ({ page }) => {
await page.goto('https://www.selenium.dev/selenium/web/webComponents.html');
await page.getByRole('checkbox').check();
await expect(page.locator('custom-checkbox-element input[type="checkbox"]')).toBeChecked();
await expect(page.getByRole('link', { name: 'Example Link' })).toBeVisible();
});
test('XPath does not pierce shadow roots', async ({ page }) => {
await page.goto('https://www.selenium.dev/selenium/web/webComponents.html');
await expect(page.locator('xpath=//input')).toHaveCount(0);
await expect(page.locator('input')).toHaveCount(1);
});
Playwright's CSS engine and its getBy... locators search open shadow roots automatically, so the test does not need to know a component is involved. Its XPath engine does not, which is one more reason to keep XPath for the cases in chapter 4.
A Selenium test fails with NoSuchElementException for input#search, but DevTools shows the input inside <app-header> under #shadow-root (open). What fixes it?
The input is in the shadow tree, which document-level searches do not enter, however long you wait. XPath cannot cross the boundary either. A shadow root is not a frame, so switch_to.frame does not apply.
iframes#
An iframe holds a complete second document. A WebDriver session looks at one document at a time, so Selenium has to switch into the frame, and back out again.
def test_iframe(driver):
driver.get(IFRAMES)
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it("iframe1"))
driver.find_element(By.ID, "email").send_keys("me@example.com")
driver.switch_to.default_content()
assert driver.find_element(By.ID, "iframe_page_heading").text == "This is the heading"
-
frame_to_be_available_and_switch_to_itaccepts the frame'sidorname, a locator tuple, or an element, and waits until the frame has loaded before switching. Prefer it over a bareswitch_to.frame(), which fails if the iframe is not there yet. - After switching, every
find_elementsearches inside the frame, and elements of the outer page cannot be found until you switch back withdefault_content(). - Element references do not cross frames. An element found inside the frame cannot be used after switching out.
Nested frames are entered one level at a time. the-internet.herokuapp.com/nested_frames is a frameset with a frameset inside it:
def test_nested_frames(driver):
driver.get(NESTED)
driver.switch_to.frame("frame-top")
driver.switch_to.frame("frame-middle")
assert driver.find_element(By.ID, "content").text == "MIDDLE"
driver.switch_to.parent_frame()
driver.switch_to.frame("frame-left")
assert "LEFT" in driver.find_element(By.TAG_NAME, "body").text
parent_frame() goes up one level; default_content() goes all the way to the top. A test that forgets to switch back leaves every later lookup searching the wrong document, so put the switching inside a page object method or a context manager rather than scattering it through tests.
Playwright does not switch at all. A frame locator scopes everything after it:
test('locators inside an iframe', async ({ page }) => {
await page.goto('https://www.selenium.dev/selenium/web/iframes.html');
const form = page.frameLocator('#iframe1');
await form.locator('#email').fill('me@example.com');
await expect(form.locator('#email')).toHaveValue('me@example.com');
await expect(page.locator('#email')).toHaveCount(0);
});
test('nested frames', async ({ page }) => {
await page.goto('https://the-internet.herokuapp.com/nested_frames');
const middle = page
.frameLocator('frame[name="frame-top"]')
.frameLocator('frame[name="frame-middle"]');
await expect(middle.locator('#content')).toHaveText('MIDDLE');
});
page.locator('#email') still searches only the top document, which is why it finds nothing: the frame locator is explicit, but you never have to remember to switch back. All getBy... methods are available on a frame locator, so form.getByRole('button', { name: 'Submit' }) works as you would expect.
A Selenium test fills a payment form inside an iframe, then fails to find the "Order summary" heading on the main page, although it is visible. What is missing?
After switching into a frame, every search runs inside that frame's document until you switch back. Waiting longer does not help, and refreshing loses the form you just filled.
Locating the frame itself#
The frame element needs a locator too. Prefer, in order: a name or id you control, a title attribute (which iframes should have for accessibility anyway, and which Playwright can use via page.frameLocator('iframe[title="Payment"]')), and only then a src match such as iframe[src*="checkout"]. Avoid "the second iframe on the page": third-party scripts such as chat widgets and analytics often inject their own iframes.
Summary#
- Selenium:
element.shadow_rootand CSS inside it; no XPath in shadow trees; closed roots are out of reach. - Playwright: CSS and
getBy...pierce open shadow roots; XPath does not. - Selenium: switch into frames with a wait, one level at a time, and always switch back.
- Playwright:
frameLocator()scopes locators to a frame without any switching.