Skip to main content

Robot Framework Automated WebDriver Testing

See our Robot Framework example repository for a first example on how to run Robot Framework tests on TestingBot.

With SeleniumLibrary you can run WebDriver tests with Robot Framework.
Robot Framework is a test automation framework to run acceptance tests.

The Robot test framework allows you to build easy to read test cases, using keyword driven and behavior-driven approaches.
The framework provides reports, in HTML format.

Setting up Robot Framework

Make sure you've installed pip, then install the necessary libraries:

pip install robotframework-seleniumlibrary requests

Running your first test

Below is an example on how to run a simple test on Chrome. When the test has finished, the test name and success state is sent to TestingBot so you can see the test success/failures in our dashboard.

Save the example below in a test.robot file:

*** Settings ***

Library  SeleniumLibrary
Library  TestingBot.py

Test Setup  Open test browser
Test Teardown  Close test browser

*** Variables ***

${BROWSER}  chrome
${VERSION}  latest
${PLATFORM}  WIN11

*** Test Cases ***

Simple Test
    Go to  https://testingbot.com
    ${title}=  Get Title
    Should Contain  ${title}  TestingBot

*** Keywords ***

Open test browser
    ${options}=  TestingBot Options  ${BROWSER}  ${PLATFORM}  ${VERSION}
    ...  name=${TEST_NAME}  build=MyTestBuild
    Open Browser  remote_url=https://hub.testingbot.com/wd/hub  options=${options}

Close test browser
    Report TestingBot status  ${SUITE_NAME} | ${TEST_NAME}  ${TEST_STATUS}
    Close all browsers

Save the code below in a TestingBot.py file, next to test.robot. It builds the browser options, including the tb:options with your TestingBot key and secret, and reports the test result back to TestingBot:

import os

import requests
from robot.libraries.BuiltIn import BuiltIn
from selenium import webdriver

OPTIONS = {
    "chrome": webdriver.ChromeOptions,
    "firefox": webdriver.FirefoxOptions,
    "edge": webdriver.EdgeOptions,
    "safari": webdriver.SafariOptions,
}


def testingbot_options(browser, platform, version="latest", **tb_options):
    """Returns Selenium options for a TestingBot browser.

    The key and secret are read from the TB_KEY and TB_SECRET environment
    variables and only exist inside Python, so they are never logged.
    Extra named arguments (name, build, realDevice, ...) go into tb:options.
    """
    options = OPTIONS[browser.lower()]()
    options.browser_version = version
    options.platform_name = platform
    options.set_capability("tb:options", {
        "key": os.environ["TB_KEY"],
        "secret": os.environ["TB_SECRET"],
        **tb_options,
    })
    return options


def report_testingbot_status(name, status):
    selenium = BuiltIn().get_library_instance("SeleniumLibrary")
    session_id = selenium.driver.session_id

    payload = {"test[name]": name, "test[success]": int(status == "PASS")}
    url = f"https://api.testingbot.com/v1/tests/{session_id}"
    response = requests.put(url, data=payload,
                            auth=(os.environ["TB_KEY"], os.environ["TB_SECRET"]))
    assert response.status_code == 200, response.text

To run the test, put your TestingBot credentials in the environment and run robot:

export TB_KEY=API_KEY
export TB_SECRET=API_SECRET
robot --variable BROWSER:chrome --variable VERSION:latest --variable PLATFORM:WIN11 test.robot

Once the test finished running, you should see it in the member dashboard. Use the variables to run the test on different platforms/browsers/versions.

Keep your secret out of the logs. Robot Framework logs every variable assignment, so building tb:options with Create Dictionary key=... secret=... (or putting the credentials in remote_url, which Open Browser logs) writes your secret to output.xml and log.html. Keep the credentials in Python as shown above, and do not run with --loglevel DEBUG or TRACE in CI: at those levels the session request, including tb:options, is logged too.

Specifying the operating system, browser and version

To let TestingBot know on which browser/platform you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field.

${options}=    TestingBot Options    ${BROWSER}    ${PLATFORM}    ${VERSION}

To see how to do this, please select a combination of browser, version and platform in the drop-down menus below.

Testing Internal Websites

We've built TestingBot Tunnel, to provide you with a secure way to run tests against your staged/internal webapps.
Please see our TestingBot Tunnel documentation for more information about this easy to use tunneling solution.

The example below shows how to easily run a Robot Framework test with our Tunnel:

1. Download our tunnel and start the tunnel:

export TESTINGBOT_KEY=key
export TESTINGBOT_SECRET=secret
java -jar testingbot-tunnel.jar

The tunnel reads your credentials from the TESTINGBOT_KEY and TESTINGBOT_SECRET environment variables (or a ~/.testingbot file), which keeps the secret out of the process list. See Tunnel configuration.

2. Adjust your test: instead of pointing to 'hub.testingbot.com/wd/hub' like the example above - change it to point to your tunnel's IP address.
Assuming you run the tunnel on the same machine you run your tests, change to 'localhost:4445/wd/hub'. localhost is the machine running the tunnel, 4445 is the default port of the tunnel.

This way your test will go securely through the tunnel to TestingBot and back.

Parallel Testing

You can run multiple Robot Framework tests in parallel by using the Pabot library.

First install Pabot using pip:

pip install -U robotframework-pabot

Then run your tests using the pabot command instead of robot:

export TB_KEY=API_KEY
export TB_SECRET=API_SECRET
pabot --processes 2 --variable BROWSER:chrome --variable VERSION:latest --variable PLATFORM:WIN11 test.robot

Some recommendations when running tests in parallel:

  • Make sure each test case is independent and does not rely on shared state.
  • Specify the --processes with the same amount as your TestingBot plan allows (parallel test sessions).
  • Consider using --processtimeout to set a timeout for each process.

Queuing

Every plan we provide comes with a limit of parallel tests.
If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available.

Mark tests as passed/failed

As TestingBot has no way to determine whether your test passed or failed (it is determined by your business logic), we offer a way to send the test status back to TestingBot. This is useful if you want to see if a test succeeded or failed from the TestingBot member area.

You can use our Python API client to report back test results.

pip install "testingbotclient>=0.2"
from testingbotclient import TestingBotClient

client = TestingBotClient('API_KEY', 'API_SECRET')
client.tests.update_test(
    driver.session_id,
    name='My Test',
    passed=True,
    build='build-123'
)

Other Python Framework examples

  • PyTest

    PyTest makes it easy to run Selenium tests with Python.

  • Behave

    Behave is behaviour-driven development, Python style.

  • Lettuce

    Lettuce is a Python BDD plugin based on Ruby's Cucumber, offering Gherkin stories.

  • PyUnit

    PyUnit is the standard unit testing framework module for Python, described as a Python version of JUnit.

  • Helium

    Helium is a tool that makes it easy to test websites and automate browsers.

  • Pylenium

    Pylenium brings the best of Selenium, Cypress and Python into one package.

  • SeleniumBase

    SeleniumBase is a Python package that simplifies Selenium testing.

Was this page helpful?
Last updated