---
title: Robot Framework reports, CI and test status on TestingBot
description: A suite earns its keep when it runs on every change and its failures
  are quick to understand. This chapter covers the files every run produces, the exit
  code…
source_url:
  html: https://testingbot.com/resources/courses/robot-framework/reports-ci-and-test-status
  md: https://testingbot.com/resources/courses/robot-framework/reports-ci-and-test-status.md
---

Course

[Robot Framework Tutorial for Web Testing](https://testingbot.com/resources/courses/robot-framework)

Chapter 9 of 10

1. [1 Robot Framework Tutorial for Beginners](https://testingbot.com/resources/courses/robot-framework)
2. [2 Install Robot Framework in a Python virtual environment](https://testingbot.com/resources/courses/robot-framework/install-robot-framework)
3. [3 Your first Robot Framework test with SeleniumLibrary](https://testingbot.com/resources/courses/robot-framework/first-test-with-seleniumlibrary)
4. [4 Variables, resource files and user keywords](https://testingbot.com/resources/courses/robot-framework/variables-resources-and-keywords)
5. [5 Run Robot Framework tests on TestingBot](https://testingbot.com/resources/courses/robot-framework/run-robot-framework-on-testingbot)
6. [6 Robot Framework Browser library: Playwright on TestingBot](https://testingbot.com/resources/courses/robot-framework/browser-library-playwright)
7. [7 Data-driven tests with Robot Framework templates](https://testingbot.com/resources/courses/robot-framework/data-driven-tests-with-templates)
8. [8 Tags and parallel test runs with Pabot](https://testingbot.com/resources/courses/robot-framework/tags-and-parallel-runs-with-pabot)
9. [9 Robot Framework reports, CI and test status on TestingBot](https://testingbot.com/resources/courses/robot-framework/reports-ci-and-test-status)
10. [10 Knowledge check](https://testingbot.com/resources/courses/robot-framework/knowledge-check)

[Exit course](https://testingbot.com/resources/courses)

 Chapter 9 of 10 Beginner 6 min read 2 quick checks Updated Nov 2026 

# Robot Framework reports, CI and test status on TestingBot
On this page
1. [The three output files](https://testingbot.com#the-three-output-files)
2. [The exit code](https://testingbot.com#the-exit-code)
3. [Combining and post-processing results](https://testingbot.com#combining-and-post-processing-results)
4. [Run the suite in GitHub Actions](https://testingbot.com#run-the-suite-in-github-actions)
5. [Pass and fail on TestingBot, recapped](https://testingbot.com#pass-and-fail-on-testingbot-recapped)

A suite earns its keep when it runs on every change and its failures are quick to understand. This chapter covers the files every run produces, the exit code your CI reads, rerunning failures, and a GitHub Actions workflow that runs the suite on TestingBot.

## The three output files[#](https://testingbot.com#the-three-output-files)

Every `robot` run writes three files into `--outputdir`:

| File | What it is | Who reads it |
| --- | --- | --- |
| `output.xml` | the complete machine-readable result | `rebot`, CI plugins, Pabot's merge step |
| `log.html` | every suite, test and keyword with arguments, timings, log messages and screenshots | you, when something failed |
| `report.html` | a summary with statistics by tag and by suite, linking into the log | anyone who wants the overview |

When a test fails, open `log.html` and expand the failing test. The failing keyword is highlighted and SeleniumLibrary's failure screenshot (`selenium-screenshot-1.png`, also in the output folder) is embedded right below it. If your keywords are well named, the path to the failure reads like a sentence: _Open The Web Form, Fill In The Form, Submit The Form, Confirmation Should Be Shown_ failed on _Title Should Be_.

The log shows the arguments of every keyword, so be careful what goes into it. `Input Password` hides the value; your key and secret are only inside `tb:options` built in Python, not passed as keyword arguments. Adding `--loglevel TRACE` logs even more, including return values, which helps debugging but also shows more data.

## The exit code[#](https://testingbot.com#the-exit-code)

`robot` exits with the number of failed tests (capped at 250), so 0 means everything passed. CI systems treat any non-zero exit code as a failed step, so you do not need to parse anything to fail the build. A run of the web form suite together with the deliberately failing test from chapter 4 exited with code 1.

## Combining and post-processing results[#](https://testingbot.com#combining-and-post-processing-results)

`rebot` turns one or more `output.xml` files into a log and report without running anything. Two uses come up constantly.

**One report for several browser runs.** Run each browser with its own output file and a suite name, then combine them:

```bash
robot --outputdir results --output chrome.xml --name Chrome --variable BROWSER:chrome --include smoke tests
robot --outputdir results --output firefox.xml --name Firefox --variable BROWSER:firefox --include smoke tests
rebot --outputdir results --name "Smoke on two browsers" --xunit xunit.xml results/chrome.xml results/firefox.xml
```

The combined report has one top-level suite per browser. `--xunit xunit.xml` also writes a JUnit-style XML file, the format most CI systems display as test results. Both `robot` and `pabot` accept `--xunit` directly too.

**Rerunning only the failures.**

```bash
robot --outputdir results tests
robot --outputdir results --output rerun.xml --rerunfailed results/output.xml tests
rebot --outputdir results --merge results/output.xml results/rerun.xml
```

`--rerunfailed` selects the tests that failed in the given output, and `--merge` writes a report where the rerun results replace the original ones, with a note that the test was rerun. Both runs must use the same root suite (here the `tests` folder); `rebot` refuses to merge outputs from different ones.

Use this to separate a genuine bug from a hiccup, not to make a red build green. A test that passes only on the rerun is a flaky test that needs fixing; the [flaky tests course](https://testingbot.com/resources/courses/flaky-tests) shows how to find out why.

The nightly job reruns failures automatically, and the merged report has been green for weeks. The same three tests appear as "rerun" almost every night. What should happen?

 Nothing: the merged report is green, so the suite is healthy Investigate those three tests; passing only on rerun means they are flaky or hiding an intermittent bug Rerun failures twice to be safe

The merge records which tests needed a rerun precisely so you can see this pattern. A test that regularly fails first time either has a timing problem or catches a real intermittent defect; more reruns only hide it better.

## Run the suite in GitHub Actions[#](https://testingbot.com#run-the-suite-in-github-actions)

Store your TestingBot key and secret as repository secrets named `TB_KEY` and `TB_SECRET` (Settings, Secrets and variables, Actions). Then add `.github/workflows/robot.yml`:

```yaml
name: Robot Framework tests
on: [push, pull_request]

jobs:
  robot:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.13"
          cache: pip
      - uses: actions/setup-node@v4
        with:
          node-version: 22
      - run: pip install -r requirements.txt
      - run: rfbrowser init --skip-browsers
      - name: Run the suite on TestingBot
        env:
          TB_KEY: ${{ secrets.TB_KEY }}
          TB_SECRET: ${{ secrets.TB_SECRET }}
          TB_BUILD: github-${{ github.run_id }}-${{ github.run_attempt }}
        run: >
          pabot --processes 2 --testlevelsplit --outputdir results --xunit xunit.xml
          --variable GRID:True --variable BROWSER:chrome --variable PLATFORM:WIN11
          tests
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: robot-results
          path: results/
```

Notes on the choices:

- **`rfbrowser init --skip-browsers`** installs the Browser library's Node dependencies but no browsers: every browser runs on TestingBot, so the job does not need any.
- **`TB_BUILD`** built from the run ID and attempt groups every session of one workflow run into one TestingBot build; a re-run of the workflow gets its own build. The resource files read it with `%{TB_BUILD=robot-local}`.
- **`--processes 2`** : match your plan's parallel limit, as in chapter 7.
- **`if: always()`** uploads the log, report and screenshots even when tests fail, which is exactly when you need them. Without it, a failing test step skips the upload.

For a site that only exists inside the CI job (started in an earlier step), add the TestingBot Tunnel action and point the suite at the tunnel; the [GitHub Actions guide](https://testingbot.com/support/integrations/ci-cd/github-actions) shows the setup.

Robot tests fail in the GitHub Actions job, but the `robot-results` artifact with the log is missing, so nobody can see why. What is the likely cause?

 `log.html` is only written when all tests pass The `--outputdir` folder is deleted on failure Artifacts cannot contain HTML files The upload step has no `if: always()`, so it is skipped after the failing test step

GitHub Actions skips later steps after a failed step unless they say otherwise. Robot always writes the log and report; the upload step must run regardless of the test result.

## Pass and fail on TestingBot, recapped[#](https://testingbot.com#pass-and-fail-on-testingbot-recapped)

Your CI reads Robot's exit code; the TestingBot dashboard needs its own signal. Both resource files send it in the teardown, where `${TEST STATUS}` is known:

- **SeleniumLibrary** (chapter 4): `Execute Javascript tb:test-result=passed` or `failed`, before the browser closes.
- **Browser library** (chapter 5): `Evaluate JavaScript` with a `testingbot_executor:` `setSessionStatus` command.

With both in place, the build in the dashboard matches the Robot report test for test: the same names, the same pass and fail, plus a video for every session. When a test fails in CI, the session ID in the console output or the test name takes you straight to the recording of what the browser did.

That completes the suite. The last chapter is a short knowledge check.

[Previous · Chapter 8 Tags and parallel test runs with Pabot](https://testingbot.com/resources/courses/robot-framework/tags-and-parallel-runs-with-pabot) [Next · Chapter 10 Knowledge check](https://testingbot.com/resources/courses/robot-framework/knowledge-check)

On this page

1. [The three output files](https://testingbot.com#the-three-output-files)
2. [The exit code](https://testingbot.com#the-exit-code)
3. [Combining and post-processing results](https://testingbot.com#combining-and-post-processing-results)
4. [Run the suite in GitHub Actions](https://testingbot.com#run-the-suite-in-github-actions)
5. [Pass and fail on TestingBot, recapped](https://testingbot.com#pass-and-fail-on-testingbot-recapped)

Certification

Earn the free, resume-worthy TestingBot Robot Framework Certification for this course.

[How it works](https://testingbot.com/resources/courses/robot-framework/certification)
