---
title: 'Visual Tests in CI: check pull request previews'
description: Run a hosted visual test from GitHub Actions against a pull request's
  preview deployment and report the result as a GitHub check.
source_url:
  html: https://testingbot.com/support/visual-testing/ci
  md: https://testingbot.com/support/visual-testing/ci.md
---

# Run Visual Tests from CI

A [hosted visual test](https://testingbot.com/support/visual-testing/codeless) normally runs on a schedule against its own URL. From CI you can run it on demand, in all of its browsers, against a pull request's preview deployment, and get the result back as a GitHub check on the pull request.

1. Your workflow deploys a preview of the pull request.
2. It calls [`POST /v1/visual/tests/{id}/run`](https://testingbot.com/support/api/tests#post-visual-tests-id-run) with the preview URL.
3. TestingBot screenshots the preview in every browser of the test and compares each screenshot with the test's baseline for that browser.
4. The result appears as a `TestingBot / visual #{id}` check on the commit, or your workflow polls [`GET /v1/visual/checks/{check}`](https://testingbot.com/support/api/tests#get-visual-checks-check).

## Allow preview hosts

The preview URL replaces the test's URL for that one run. It may be on the same domain as the test's URL (`pr-42.staging.example.com` for a test of `www.example.com`). Preview deployments on another domain, such as `*.vercel.app`, must be listed under **Preview hosts** in the test's settings. A `*` matches within one part of the host name.

The preview URL must resolve to a public address. To check a deployment that is only reachable inside your network, start a [TestingBot Tunnel](https://testingbot.com/support/tunnel) first: hosted visual tests use a running tunnel automatically.

## GitHub Actions example

Install the [TestingBot GitHub App](https://testingbot.com/support/integrations/ci-cd/github-pr-checks#install) on the repository owner so TestingBot can post the check. Store your API key and secret as the repository secrets `TESTINGBOT_KEY` and `TESTINGBOT_SECRET`, then add a job after the one that deploys the preview. Replace `1234` with the ID of your visual test (it is in the test's URL in the dashboard).

```yaml
name: Visual check
on: pull_request

jobs:
  deploy:
    runs-on: ubuntu-latest
    outputs:
      preview_url: ${{ steps.deploy.outputs.url }}
    steps:
      # Your existing preview deployment, exposing the preview URL as an output.
      - id: deploy
        run: echo "url=https://pr-${{ github.event.number }}.preview.example.com/" >> "$GITHUB_OUTPUT"

  visual:
    needs: deploy
    runs-on: ubuntu-latest
    steps:
      - name: Run the visual test against the preview
        env:
          TB_KEY: ${{ secrets.TESTINGBOT_KEY }}
          TB_SECRET: ${{ secrets.TESTINGBOT_SECRET }}
          PREVIEW_URL: ${{ needs.deploy.outputs.preview_url }}
          HEAD_SHA: ${{ github.event.pull_request.head.sha }}
          PR: ${{ github.event.pull_request.number }}
        run: |
          curl -sSf -u "$TB_KEY:$TB_SECRET" -X POST \
            "https://api.testingbot.com/v1/visual/tests/1234/run" \
            --data-urlencode "url=$PREVIEW_URL" \
            -d "repo=$GITHUB_REPOSITORY" \
            -d "commit=$HEAD_SHA" \
            -d "pr=$PR"
```

The call returns as soon as the browsers are queued. The `TestingBot / visual #1234` check shows as in progress and concludes when the last browser has been compared. Use the `head.sha` of the pull request, not the merge commit, so the check lands on the commit branch protection looks at. To block merging on a visual change, add the check under **Require status checks to pass** in the branch protection rule.

## Poll instead of a check

Leave out `repo` and `commit` when you do not use the GitHub App, and let the job wait for the result itself:

```bash
CHECK=$(curl -sSf -u "$TB_KEY:$TB_SECRET" -X POST \
  "https://api.testingbot.com/v1/visual/tests/1234/run" \
  --data-urlencode "url=$PREVIEW_URL" | jq -r .check)

for i in $(seq 1 90); do
  STATUS=$(curl -sSf -u "$TB_KEY:$TB_SECRET" "https://api.testingbot.com/v1/visual/checks/$CHECK")
  if ["$(echo "$STATUS" | jq -r .state)" = "done"]; then
    echo "$STATUS" | jq '.browsers[] | {browser, state, passed, mismatch, run_url}'
    ["$(echo "$STATUS" | jq -r .passed)" = "true"]
    exit $?
  fi
  sleep 20
done
echo "Visual check did not finish in time"
exit 1
```

## How pass/fail is decided

Each browser of the test is compared with its own baseline, using the test's mismatch threshold, masks and match levels, exactly as a scheduled run would. The check passes only when every browser was compared and passed. It fails when:

- a browser's screenshot differs from its baseline by more than the threshold;
- a browser has no baseline yet (see below);
- a browser could not be run, for example because the preview did not load.

The check lists every browser with its mismatch percentage and a link to the run page, where you can review the difference and approve it.

## Baselines

A run against a preview URL is compared with the test's normal baselines and never replaces them on its own, so scheduled runs of the test are not affected. A preview run also does not send the test's alerts: its result is reported on the check.

A browser that has no baseline yet is not compared. Run the test once against its own URL, on its schedule or with **Run now** , to record one. When a change is intended, approve the run on its run page and set it as the new baseline, the same way as for a scheduled run.

## Looking for more help?

Have questions or need more information? Reach out via email or Slack.

[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)