Data-driven tests with Robot Framework templates
On this page
Many tests are the same steps with different data: a login form with a wrong user, a wrong password, an empty field. Writing each one out repeats the steps; looping over the data in one test hides which case failed. Robot Framework's test templates give you one keyword for the steps and one table row per case, and every row is its own test in the report.
The examples use the practice login page of The Internet. Its valid credentials, tomsmith and SuperSecretPassword!, are printed on the page itself; anything else shows a red message saying whether the username or the password was wrong.
The keyword that does the work#
resources/login.resource:
*** Settings ***
Documentation Keywords for the practice login page.
Resource browser.resource
*** Variables ***
${LOGIN URL} https://the-internet.herokuapp.com/login
*** Keywords ***
Log In As
[Arguments] ${username} ${password}
Input Text id:username ${username}
Input Password id:password ${password}
Click Button css:button[type=submit]
Login Should Fail With
[Arguments] ${username} ${password} ${message}
Log In As ${username} ${password}
Wait Until Element Contains id:flash ${message}
Input Password does the same as Input Text but does not write the value to the log, so a real password never ends up in log.html. Wait Until Element Contains is the fix from chapter 2: the message only exists after the page has reloaded.
The template#
tests/login.robot:
*** Settings ***
Documentation Data-driven login checks with a test template.
Resource ../resources/login.resource
Test Setup Open Test Browser ${LOGIN URL}
Test Teardown Close Test Browser
Test Template Login Should Fail With
*** Test Cases *** USERNAME PASSWORD MESSAGE
Unknown User nobody SuperSecretPassword! Your username is invalid!
Wrong Password tomsmith wrong Your password is invalid!
Empty Username ${EMPTY} SuperSecretPassword! Your username is invalid!
Username Is Case Sensitive TOMSMITH SuperSecretPassword! Your username is invalid!
*** Test Cases ***
Valid Login Reaches The Secure Area
[Template] NONE
Log In As tomsmith SuperSecretPassword!
Wait Until Element Contains id:flash You logged into a secure area!
How it works:
-
Test Template Login Should Fail Withmakes every test in the file call that keyword, with the test's cells as its arguments. A row is therefore a complete test: name, then username, password and expected message. -
The extra words in the header line (
USERNAME,PASSWORD,MESSAGE) are ignored by Robot Framework. They are column titles for the human reader, and they are what makes the table readable. -
${EMPTY}passes an empty string. A blank cell would be ignored rather than passed as an argument, which shifts every value one column to the left. -
[Template] NONEswitches the template off for a test that needs ordinary steps. A file may have more than one Test Cases section, which keeps the happy path visually apart from the table.
Setup and teardown still apply to every row, so each case gets its own fresh browser: one case can never leave the next one logged in.
robot --outputdir results tests/login.robot
Unknown User | PASS |
------------------------------------------------------------------------------
Wrong Password | PASS |
------------------------------------------------------------------------------
Empty Username | PASS |
------------------------------------------------------------------------------
Username Is Case Sensitive | PASS |
------------------------------------------------------------------------------
Valid Login Reaches The Secure Area | PASS |
------------------------------------------------------------------------------
Login :: Data-driven login checks with a test template. | PASS |
5 tests, 5 passed, 0 failed
The same file ran unchanged on TestingBot, all five tests passing on Safari on macOS Sequoia (--variable GRID:True --variable BROWSER:safari --variable PLATFORM:SEQUOIA) and on Edge on Windows 11.
A new row Empty Password tomsmith Your password is invalid! (with an empty cell for the password) fails with "expected 3 arguments, got 2". Why?
Robot Framework separates cells by runs of spaces, so an empty cell simply disappears. ${EMPTY} is the explicit empty string. Header words are only labels and have no types.
Template or FOR loop?#
You could write the same checks as one test with a FOR loop over a list. The difference shows up when something fails:
| Template, one row per test | One test with a FOR loop | |
|---|---|---|
| Report | one line per case, named after it | one line for all cases |
| After a failing case | the other cases still run | the loop stops at the first failure |
| Rerun one case | robot --test "Wrong Password" |
not possible |
| Parallel runs | each row can run in its own process (chapter 7) | the loop runs as one unit |
Use a loop for steps inside a test (fill five fields, check five rows); use a template when every data set is a separate case you want reported on its own.
A suite checks 40 discount codes in one test with a FOR loop. Code 7 is broken, and the report only says the test failed. What change gives you a result per code?
Continuing on failure would at least run all codes, but the report would still have one test with a list of errors, and a single code could not be rerun or run in parallel. One row per code makes each its own test.
When the data lives outside the suite#
A table in the .robot file is right for a handful of cases that change with the tests. When the data comes from a spreadsheet maintained by someone else, or there are hundreds of rows, the community library DataDriver (robotframework-datadriver) generates one test per row of a CSV or Excel file using the same template keyword. Keep the keyword exactly as it is here; only where the rows come from changes.