Skip to main content

Robot Framework & Playwright

Browser Library allows you to run Playwright tests with Robot Framework. By using TestingBot, you will be able to run Robot tests using Playwright on TestingBot's browser grid. The advantage is that you do not have to set up and maintain a fleet of different browsers, versions and operating systems. Another advantage is that you can run multiple tests, in parallel, on TestingBot's grid.

Example

Below is an example on how to connect to the TestingBot browser grid, using Browser Library's Connect To Browser. The connection is made from a small Python library, so your TestingBot key and secret never end up in Robot Framework's output.xml or log.html (see Keep your credentials out of the logs).

pip install robotframework robotframework-browser
rfbrowser init chromium

Save the code below in a TestingBot.py file:

import json
import os
from datetime import timedelta
from urllib.parse import quote

from Browser.utils.data_types import SupportedBrowsers
from robot.libraries.BuiltIn import BuiltIn

# Playwright engine that drives each TestingBot browser
ENGINES = {"chrome": "chromium", "edge": "chromium", "firefox": "firefox", "safari": "webkit"}


def connect_to_testingbot(browser, platform, version="latest", name="", build=""):
    """Connects the Browser library to a TestingBot browser.

    The key and secret are read from the TB_KEY and TB_SECRET environment
    variables and only exist inside Python, so they are never logged.
    """
    capabilities = {
        "browserName": browser.lower(),
        "browserVersion": version,
        "platform": platform,
        "tb:options": {
            "key": os.environ["TB_KEY"],
            "secret": os.environ["TB_SECRET"],
            "name": name,
            "build": build,
        },
    }
    endpoint = ("wss://cloud.testingbot.com/playwright?capabilities="
                + quote(json.dumps(capabilities)))
    BuiltIn().get_library_instance("Browser").connect_to_browser(
        endpoint, SupportedBrowsers[ENGINES[browser.lower()]], timeout=timedelta(minutes=2))

Then save the test in a file, for example test.robot:

*** Settings ***
Library    Browser    enable_playwright_debug=disabled
Library    TestingBot.py
Test Setup    Connect To TestingBot    chrome    WIN11    latest
...    name=${TEST NAME}    build=robot-playwright
Test Teardown    Report Status And Close

*** Test Cases ***
Playwright Test
    New Page    https://playwright.dev/
    Get Title    matches    Playwright
    Get Attribute    "Get started"    href    ==    /docs/intro
    Click    "Get started"
    Get Url    matches    .*intro

*** Keywords ***
Report Status And Close
    ${passed}=    Evaluate    $TEST_STATUS == 'PASS'
    &{arguments}=    Create Dictionary    passed=${passed}    reason=${TEST MESSAGE}
    &{command}=    Create Dictionary    action=setSessionStatus    arguments=${arguments}
    ${json}=    Evaluate    json.dumps($command)    modules=json
    Evaluate JavaScript    ${None}    () => {}    arg=testingbot_executor: ${json}
    Close Browser

Run the test with your TestingBot credentials in the environment:

export TB_KEY=add-your-TestingBot-key
export TB_SECRET=add-your-TestingBot-secret
robot --outputdir results test.robot

This example will open a Chrome browser on Windows 11 using Playwright on TestingBot, navigate to the Playwright website and do some verifications. The teardown sends the result to TestingBot with the testingbot_executor: setSessionStatus command, so the test shows up as passed or failed in your dashboard.

  • The client-side engine must match the remote browser: chromium for Chrome and Edge, firefox for Firefox and webkit for Safari. The ENGINES lookup takes care of that.
  • The connect timeout is raised to two minutes, because a remote browser (Firefox in particular) can take longer than Browser Library's 30 second default to start, and a session may wait in the queue for a free slot.

Keep your credentials out of the logs

Robot Framework logs every variable assignment. If you build the capabilities in Robot Framework syntax (for example &{caps}= Create Dictionary key=... secret=...), your TestingBot secret is written to output.xml and log.html, which CI systems often publish as build artifacts. Building the endpoint in Python, as above, avoids that.

Browser Library also writes its own playwright-log.txt next to the Robot Framework output. That file logs the full endpoint URL, including the secret, so the example imports the library with enable_playwright_debug=disabled. Running the example above on TestingBot with that setting left 0 occurrences of the secret in the output directory.

Was this page helpful?
Last updated