Robot Framework Automated WebDriver Testing
See our Robot Framework example repository for a first example on how to run Robot Framework tests on TestingBot.
With SeleniumLibrary you can run WebDriver tests with Robot Framework.
Robot Framework is a test automation framework to run acceptance tests.
The Robot test framework allows you to build easy to read test cases, using keyword driven and behavior-driven approaches.
The framework provides reports, in HTML format.
Setting up Robot Framework
Make sure you've installed pip, then install the necessary libraries:
pip install robotframework-seleniumlibrary requests
Running your first test
Below is an example on how to run a simple test on Chrome. When the test has finished, the test name and success state is sent to TestingBot so you can see the test success/failures in our dashboard.
Save the example below in a test.robot file:
*** Settings ***
Library SeleniumLibrary
Library TestingBot.py
Test Setup Open test browser
Test Teardown Close test browser
*** Variables ***
${BROWSER} chrome
${VERSION} latest
${PLATFORM} WIN11
*** Test Cases ***
Simple Test
Go to https://testingbot.com
${title}= Get Title
Should Contain ${title} TestingBot
*** Keywords ***
Open test browser
${options}= TestingBot Options ${BROWSER} ${PLATFORM} ${VERSION}
... name=${TEST_NAME} build=MyTestBuild
Open Browser remote_url=https://hub.testingbot.com/wd/hub options=${options}
Close test browser
Report TestingBot status ${SUITE_NAME} | ${TEST_NAME} ${TEST_STATUS}
Close all browsers
Save the code below in a TestingBot.py file, next to test.robot. It builds the browser options, including the tb:options with your TestingBot key and secret, and reports the test result back to TestingBot:
import os
import requests
from robot.libraries.BuiltIn import BuiltIn
from selenium import webdriver
OPTIONS = {
"chrome": webdriver.ChromeOptions,
"firefox": webdriver.FirefoxOptions,
"edge": webdriver.EdgeOptions,
"safari": webdriver.SafariOptions,
}
def testingbot_options(browser, platform, version="latest", **tb_options):
"""Returns Selenium options for a TestingBot browser.
The key and secret are read from the TB_KEY and TB_SECRET environment
variables and only exist inside Python, so they are never logged.
Extra named arguments (name, build, realDevice, ...) go into tb:options.
"""
options = OPTIONS[browser.lower()]()
options.browser_version = version
options.platform_name = platform
options.set_capability("tb:options", {
"key": os.environ["TB_KEY"],
"secret": os.environ["TB_SECRET"],
**tb_options,
})
return options
def report_testingbot_status(name, status):
selenium = BuiltIn().get_library_instance("SeleniumLibrary")
session_id = selenium.driver.session_id
payload = {"test[name]": name, "test[success]": int(status == "PASS")}
url = f"https://api.testingbot.com/v1/tests/{session_id}"
response = requests.put(url, data=payload,
auth=(os.environ["TB_KEY"], os.environ["TB_SECRET"]))
assert response.status_code == 200, response.text
To run the test, put your TestingBot credentials in the environment and run robot:
export TB_KEY=API_KEY
export TB_SECRET=API_SECRET
robot --variable BROWSER:chrome --variable VERSION:latest --variable PLATFORM:WIN11 test.robot
Once the test finished running, you should see it in the member dashboard. Use the variables to run the test on different platforms/browsers/versions.
Keep your secret out of the logs. Robot Framework logs every variable assignment, so building tb:options with Create Dictionary key=... secret=... (or putting the credentials in remote_url, which Open Browser logs) writes your secret to output.xml and log.html. Keep the credentials in Python as shown above, and do not run with --loglevel DEBUG or TRACE in CI: at those levels the session request, including tb:options, is logged too.
Specifying the operating system, browser and version
To let TestingBot know on which browser/platform you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field.
${options}= TestingBot Options ${BROWSER} ${PLATFORM} ${VERSION}
To see how to do this, please select a combination of browser, version and platform in the drop-down menus below.
Testing Internal Websites
We've built TestingBot Tunnel, to provide you with a secure way to run tests against your staged/internal webapps.
Please see our TestingBot Tunnel documentation for more information about this easy to use tunneling solution.
The example below shows how to easily run a Robot Framework test with our Tunnel:
1. Download our tunnel and start the tunnel:
export TESTINGBOT_KEY=key
export TESTINGBOT_SECRET=secret
java -jar testingbot-tunnel.jar
The tunnel reads your credentials from the TESTINGBOT_KEY and TESTINGBOT_SECRET environment variables (or a ~/.testingbot file), which keeps the secret out of the process list. See Tunnel configuration.
2. Adjust your test: instead of pointing to 'hub.testingbot.com/wd/hub' like the example above - change it to point to your tunnel's IP address.
Assuming you run the tunnel on the same machine you run your tests, change to 'localhost:4445/wd/hub'. localhost is the machine running the tunnel, 4445 is the default port of the tunnel.
This way your test will go securely through the tunnel to TestingBot and back.
Parallel Testing
You can run multiple Robot Framework tests in parallel by using the Pabot library.
First install Pabot using pip:
pip install -U robotframework-pabot
Then run your tests using the pabot command instead of robot:
export TB_KEY=API_KEY
export TB_SECRET=API_SECRET
pabot --processes 2 --variable BROWSER:chrome --variable VERSION:latest --variable PLATFORM:WIN11 test.robot
Some recommendations when running tests in parallel:
- Make sure each test case is independent and does not rely on shared state.
- Specify the
--processeswith the same amount as your TestingBot plan allows (parallel test sessions). - Consider using
--processtimeoutto set a timeout for each process.
Queuing
Every plan we provide comes with a limit of parallel tests.
If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available.
Mark tests as passed/failed
As TestingBot has no way to determine whether your test passed or failed (it is determined by your business logic), we offer a way to send the test status back to TestingBot. This is useful if you want to see if a test succeeded or failed from the TestingBot member area.
You can use our Python API client to report back test results.
pip install "testingbotclient>=0.2"
from testingbotclient import TestingBotClient
client = TestingBotClient('API_KEY', 'API_SECRET')
client.tests.update_test(
driver.session_id,
name='My Test',
passed=True,
build='build-123'
)
Other Python Framework examples
-
PyTest
PyTest makes it easy to run Selenium tests with Python.
-
Behave
Behave is behaviour-driven development, Python style.
-
Lettuce
Lettuce is a Python BDD plugin based on Ruby's Cucumber, offering Gherkin stories.
-
PyUnit
PyUnit is the standard unit testing framework module for Python, described as a Python version of JUnit.
-
Helium
Helium is a tool that makes it easy to test websites and automate browsers.
-
Pylenium
Pylenium brings the best of Selenium, Cypress and Python into one package.
-
SeleniumBase
SeleniumBase is a Python package that simplifies Selenium testing.