Exit course
Chapter 9 of 10 Intermediate 6 min read 2 quick checks Updated

Selenium C# tests in CI

On this page
  1. Store the credentials
  2. The GitHub Actions workflow
  3. The same in Azure Pipelines
  4. What we ran
  5. Where to go from here

A suite that only runs when someone remembers to run it decays quickly. This chapter runs it on every push and pull request, first with GitHub Actions and then with Azure Pipelines, keeping the test results and failure screenshots of every run.

The plan is the same on any CI system:

  1. Install the .NET SDK.
  2. Pass the TestingBot credentials in as secret environment variables.
  3. Run dotnet test with GRID=testingbot, a build name unique to the run and a TRX logger.
  4. Upload the TestResults folder even when tests fail, because that is when you need it.

A CI agent usually has no desktop and only a few cores, so running the browsers on the grid rather than on the agent keeps the job small and gives you browsers the agent does not have, such as Safari.

Store the credentials#

In GitHub, open the repository's Settings, then Secrets and variables, then Actions, and add TB_KEY and TB_SECRET. Never put them in the workflow file, in appsettings.json or in a .runsettings file that is committed: anything in the repository is readable by everyone who can clone it, and stays in its history.

The GitHub Actions workflow#

Add .github/workflows/ui-tests.yml:

name: UI tests
on: [push, pull_request]

jobs:
  selenium:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        browser: [chrome, firefox]
    steps:
      - uses: actions/checkout@v7
      - uses: actions/setup-dotnet@v6
        with:
          dotnet-version: "10.0.x"
      - run: dotnet restore
      - run: dotnet build --no-restore
      - name: Run the suite on TestingBot
        env:
          GRID: testingbot
          BROWSER: ${{ matrix.browser }}
          TB_KEY: ${{ secrets.TB_KEY }}
          TB_SECRET: ${{ secrets.TB_SECRET }}
          TB_BUILD: github-${{ github.run_id }}-${{ github.run_attempt }}
        run: >
          dotnet test --no-build
          --logger "trx;LogFileName=${{ matrix.browser }}.trx"
          --logger "console;verbosity=normal"
          -- NUnit.NumberOfTestWorkers=2
      - uses: actions/upload-artifact@v7
        if: always()
        with:
          name: test-results-${{ matrix.browser }}
          path: "**/TestResults/"

What each part does:

  • The matrix runs the job once per browser, in parallel jobs. fail-fast: false keeps the Firefox job running when the Chrome job fails, so one run tells you whether a failure is browser-specific.
  • TB_BUILD from the run ID and attempt groups every session of one workflow run, from both matrix jobs, into one TestingBot build. A re-run gets a new build.
  • NUnit.NumberOfTestWorkers=2 per job. Two jobs times two workers is four sessions at once; keep the product at or below your plan's parallel limit.
  • --no-build reuses the build from the previous step, so a compile error fails the Build step with a clear message instead of appearing in the middle of the test output.
  • if: always() uploads the TRX files and screenshots even when the test step failed.

dotnet test exits with a non-zero code when any test fails, which fails the job and, on a pull request, the check. GitHub does not display TRX files itself: download the artifact and open it in Visual Studio or Rider, or add a test-report action from the Marketplace that renders TRX as a check summary. The GitHub Actions guide adds the TestingBot Tunnel action, for testing a site that runs inside the job, and pull request status checks.

Tests fail in CI, but the uploaded artifact is missing from the run. The upload step has no if: condition. Why?

GitHub Actions stops running later steps once one fails. The TRX logger writes its file whatever the outcome; the step that would have uploaded it was simply skipped.

The same in Azure Pipelines#

Teams on Azure DevOps get one more thing for free: the test results tab. Add a pipeline variable group or pipeline variables TB_KEY and TB_SECRET, marked secret, and an azure-pipelines.yml:

trigger: [main]
pr: [main]

pool:
  vmImage: ubuntu-latest

steps:
  - task: UseDotNet@2
    inputs:
      version: "10.0.x"

  - task: DotNetCoreCLI@2
    displayName: Run the suite on TestingBot
    inputs:
      command: test
      arguments: "-- NUnit.NumberOfTestWorkers=2"
      publishTestResults: true
    env:
      GRID: testingbot
      BROWSER: chrome
      TB_KEY: $(TB_KEY)
      TB_SECRET: $(TB_SECRET)
      TB_BUILD: azure-$(Build.BuildId)

trigger runs the pipeline on pushes to main and pr on pull requests into it. The pr key works for repositories hosted on GitHub; for Azure Repos Git it is ignored, and pull request runs are set up with a build validation branch policy on main instead. The DotNetCoreCLI@2 test command adds a TRX logger itself and, with publishTestResults: true, publishes the results to the run's Tests tab. The env: block matters: Azure Pipelines does not expose secret variables to scripts and tasks as environment variables unless you map them explicitly like this. Without it, DriverFactory.Required fails with "Set the TB_KEY environment variable". The Azure DevOps guide covers TestingBot's Marketplace extension, which can set TB_KEY and TB_SECRET from a service connection and start the tunnel for a build.

What we ran#

The workflow files above were checked for valid YAML, and their commands were run locally with the same environment variables (dotnet restore, dotnet build --no-restore, then dotnet test --no-build with both loggers and GRID=testingbot, using one worker because the account was shared). They were not run inside GitHub Actions or Azure Pipelines for this course; the action and task versions are the current major releases as of November 2026.

A new Azure pipeline fails every test in SetUp with "Set the TB_KEY environment variable", although TB_KEY is defined as a secret pipeline variable. What is missing?

Azure keeps secrets out of the environment unless a step maps them in. The explicit mapping is the fix; committing the key in a .runsettings file would leak it to everyone with repository access.

Where to go from here#

The suite now runs locally, in parallel, on a cloud grid and in CI. From here, the work is in what you test rather than how: the Page Object Model course for structure as the suite grows, cross-browser testing for choosing the browser matrix, and the flaky tests course for keeping it trustworthy. The last chapter is a quiz on the decisions this course made.