CI/CD Integration

Q: Can I check on an async Maestro run later from CI?

Yes. Start the run with --async, then use testingbot status --id <projectId> --wait in a later job to block until it finishes, and testingbot artifacts to fetch reports and videos. See the Async runs section .

The TestingBot CLI integrates seamlessly with any CI/CD environment. It runs all your Maestro flows, streams the output in real-time and exits with the appropriate exit code for pipeline automation.

We recommend using npx to run the CLI without installing it globally. This ensures you're always using the latest version.

Quick Start

Run Maestro tests in any CI/CD service with a single command:

npx --yes @testingbot/cli@latest maestro app.apk ./flows \
  --device "Pixel 8" \
  --deviceVersion "14" \
  --api-key $TB_KEY \
  --api-secret $TB_SECRET

This command will:

Upload your mobile app (.apk, .aab, .ipa or .app)
Upload and run all Maestro flow files from the ./flows directory
Stream test progress to your CI logs
Exit with code 0 on success, non-zero on failure

You can pass credentials as arguments (--api-key and --api-secret) or set them as environment variables. Environment variables are the recommended approach for CI/CD.

Environment Variables

Instead of passing credentials as command arguments, you can set them as environment variables. This is the most secure approach for CI/CD pipelines since credentials are never exposed in logs or command history.

Variable	Description
`TB_KEY`	Your TestingBot API key
`TB_SECRET`	Your TestingBot API secret

When these environment variables are set, you can omit the credential arguments entirely:

export TB_KEY="your_api_key"
export TB_SECRET="your_api_secret"

npx --yes @testingbot/cli@latest maestro app.apk ./flows \
  --device "Pixel 8" \
  --deviceVersion "14"

Never hardcode credentials in your CI configuration files. Always use your CI provider's secrets management feature to store TB_KEY and TB_SECRET.

Exit Codes

The CLI distinguishes a red test run from a broken pipeline, so a job can retry on infrastructure errors but fail the build on real test failures:

Exit Code	Description
`0`	All flows passed. Also returned for `--async`, `--dry-run`, and for failed flows when `--json-file` is used (the file is the contract then)
`1`	CLI or infrastructure error: invalid arguments, missing credentials, upload failure, timeout
`2`	One or more flows failed

testingbot status follows the same rule and additionally exits 0 while a project is still running.

Test Reports

Generate test reports for CI/CD result decoration and test analytics. The CLI supports JUnit XML, HTML and Allure report formats, plus a machine-readable JSON summary.

JUnit Report

Generate JUnit XML reports for integration with CI/CD test result visualization:

npx --yes @testingbot/cli@latest maestro app.apk ./flows \
  --device "Pixel 8" \
  --deviceVersion "14" \
  --report junit \
  --report-output-dir ./test-results

Option	Description
`--report <format>`	Report format: `junit`
`--report-output-dir <path>`	Directory to save test reports (required when `--report` is used)

JUnit reports can be used with most CI/CD platforms for:

Test result visualization in your pipeline
Tracking test trends over time
Annotating pull requests with test results
Integration with test management tools

You can also fetch reports via the REST API if you need to retrieve them separately.

Allure Report

--report allure converts each run's results into Allure result files under <report-output-dir>/allure-results/, one JSON per flow with its steps, timings, status and failure details. Render them with the Allure CLI; results from several runs or shards accumulate in the same directory.

npx --yes @testingbot/cli@latest maestro app.apk ./flows --report allure --report-output-dir ./reports
allure serve ./reports/allure-results

JSON Output

--json prints a single JSON document on stdout and moves all log lines to stderr, so it pipes straight into jq. --json-file writes the same document to <projectId>_testingbot.json (or the path given with --json-file-name) while keeping the normal console output. Both imply --quiet.

npx --yes @testingbot/cli@latest maestro app.apk ./flows --json | jq '.runs[].flows[] | {name, passed, durationSeconds}'

{
  "provider": "maestro",
  "outcome": "failed",
  "success": false,
  "appId": 1234,
  "url": "https://testingbot.com/members/maestro/1234",
  "runs": [
    {
      "id": 5678,
      "status": "DONE",
      "passed": false,
      "device": { "name": "Pixel 9", "platform": "Android", "version": "14" },
      "url": "https://testingbot.com/members/maestro/1234/runs/5678",
      "flows": [
        { "id": 1, "runId": 5678, "name": "login", "status": "DONE", "passed": true, "attempt": 1, "latest": true, "durationSeconds": 30, "errors": [] }
      ]
    }
  ]
}

outcome is one of passed, failed, started (async), dry-run or error. flows lists every attempt, including --retry re-runs; latest marks the attempt whose verdict counts. When the CLI fails before a verdict, the document still arrives with outcome: "error" and an error message, so a consumer never has to handle missing output.

CI/CD Metadata

Pass repository and commit information to link test runs with your source control. This enables better traceability and integration with your development workflow.

Option	Description
`--commit-sha <sha>`	The commit SHA of this upload
`--pull-request-id <id>`	The ID of the pull request this upload originated from
`--repo-name <name>`	Repository name (e.g., GitHub repo slug)
`--repo-owner <owner>`	Repository owner (e.g., GitHub organization or user)
`--branch <name>`	Git branch this run was built from
`--pr-url <url>`	URL of the pull request this run originated from
`-m, --metadata <KEY=VALUE>`	Free-form metadata attached to the run (repeatable), for example `-m team=mobile -m env=staging`

Example with metadata:

npx --yes @testingbot/cli@latest maestro app.apk ./flows \
  --device "Pixel 8" \
  --deviceVersion "14" \
  --commit-sha "$COMMIT_SHA" \
  --branch "$BRANCH" \
  --pull-request-id "$PR_NUMBER" \
  --repo-owner "your-org" \
  --repo-name "your-repo" \
  -m team=mobile

With the TestingBot GitHub App installed, a run that carries the repository and the full head commit SHA also posts a TestingBot / tests status check on the pull request, so you can require green Maestro flows before merging. No extra step is needed.

Async runs, status and artifacts

Long suites do not have to hold a CI runner. Start the run with --async, capture the project ID, and come back for the result in a later job or from a different machine.

# Start and capture the project id
PROJECT_ID=$(npx --yes @testingbot/cli@latest maestro app.apk ./flows --async --json | jq -r .appId)

# Later: block until it finishes (exit 2 if any flow failed), with the live flow table
npx --yes @testingbot/cli@latest status --id "$PROJECT_ID" --wait

# Fetch reports and artifacts once it finished
npx --yes @testingbot/cli@latest artifacts --id "$PROJECT_ID" --report junit --report-output-dir ./reports
npx --yes @testingbot/cli@latest artifacts --id "$PROJECT_ID" --download-artifacts failed --artifacts-output-dir ./artifacts

# Browse recent projects
npx --yes @testingbot/cli@latest list --count 20

status --wait only watches the project; interrupting it does not cancel the run. status without --wait prints a snapshot and exits 0 while the project is still running, so it also works for polling.

Expo / EAS Build

Pass an eas build download URL with --app-url, or run inside EAS Workflows with the @testingbot/eas-workflow wrapper. Both are covered on the Expo / EAS page.

Upload the app once, run many suites

Every run prints its project ID after the app upload. Pass it to a later run with --app-binary-id to skip the upload entirely; each run still gets its own project and results. testingbot upload does the upload on its own and prints the ID.

APP_ID=$(npx --yes @testingbot/cli@latest upload app.apk --json | jq -r .appId)
npx --yes @testingbot/cli@latest maestro --app-binary-id "$APP_ID" ./flows/smoke --device "Pixel 9"
npx --yes @testingbot/cli@latest maestro --app-binary-id "$APP_ID" ./flows/regression --device-matrix "Pixel 9:14,Samsung Galaxy S24:14:real"

View Result Details

A link to the current test results will be printed to your CI logs during execution.

The test details page provides detailed test result information, including:

Real-time test execution status
Video recordings of each test run
Screenshots captured during tests
Device logs and Maestro output
Test artifacts for debugging

GitHub Actions

The TestingBot Maestro GitHub Action wraps the CLI, fills in branch, commit, repository and pull request from the workflow context, writes a results table to the job summary and exposes the results as outputs:

- uses: testingbot/maestro-action@v1
  with:
    api-key: ${{ secrets.TB_KEY }}
    api-secret: ${{ secrets.TB_SECRET }}
    app-file: app/build/outputs/apk/debug/app-debug.apk
    workspace: .maestro

Inputs, outputs, a device-matrix example and the raw-CLI alternative are on the GitHub Actions page.

GitHub PR status checks

With the TestingBot GitHub App installed, every Maestro run started from a pull request posts a TestingBot / tests check listing each flow on each device, and can be made a required check so a pull request cannot be merged while a flow is failing. Nothing to configure beyond installing the app. See GitHub PR Checks for Maestro.

GitLab CI

Add Maestro testing to your .gitlab-ci.yml. Store your credentials in Settings → CI/CD → Variables and mark them as protected and masked.

stages:
  - build
  - test

build:
  stage: build
  image: gradle:8-jdk17
  script:
    - ./gradlew assembleDebug
  artifacts:
    paths:
      - app/build/outputs/apk/debug/

maestro-tests:
  stage: test
  image: node:20
  dependencies:
    - build
  script:
    - npx --yes @testingbot/cli@latest maestro
        app/build/outputs/apk/debug/app-debug.apk
        ./maestro-flows
        --device "Pixel 8"
        --deviceVersion "14"
        --report junit
        --report-output-dir ./test-results
        --commit-sha $CI_COMMIT_SHA
        --pull-request-id $CI_MERGE_REQUEST_IID
        --repo-owner $CI_PROJECT_NAMESPACE
        --repo-name $CI_PROJECT_NAME
  variables:
    TB_KEY: $TB_KEY
    TB_SECRET: $TB_SECRET
  artifacts:
    when: always
    paths:
      - test-results/
    reports:
      junit: test-results/*.xml

Add TB_KEY and TB_SECRET as CI/CD variables in your GitLab project settings. Enable "Mask variable" to prevent them from appearing in job logs.

Jenkins

Add Maestro testing to your Jenkins pipeline (Jenkinsfile). Store your credentials using Manage Jenkins → Manage Credentials as secret text.

pipeline {
    agent any

    environment {
        TB_KEY = credentials('testingbot-api-key')
        TB_SECRET = credentials('testingbot-api-secret')
    }

    stages {
        stage('Build') {
            steps {
                sh './gradlew assembleDebug'
            }
        }

        stage('Maestro Tests') {
            steps {
                sh """
                    npx --yes @testingbot/cli@latest maestro \\
                        app/build/outputs/apk/debug/app-debug.apk \\
                        ./maestro-flows \\
                        --device "Pixel 8" \\
                        --deviceVersion "14" \\
                        --report junit \\
                        --report-output-dir test-results \\
                        --commit-sha ${env.GIT_COMMIT} \\
                        --pull-request-id ${env.CHANGE_ID ?: ''} \\
                        --repo-owner "your-org" \\
                        --repo-name "your-repo"
                """
            }
            post {
                always {
                    junit 'test-results/*.xml'
                    archiveArtifacts artifacts: 'test-results/**', allowEmptyArchive: true
                }
            }
        }
    }
}

Create credentials of type "Secret text" in Jenkins for TB_KEY and TB_SECRET. Reference them using the credentials() helper in your pipeline.

CircleCI

Add Maestro testing to your .circleci/config.yml. Store your credentials in Project Settings → Environment Variables.

version: 2.1

jobs:
  maestro-tests:
    docker:
      - image: cimg/node:20
    steps:
      - checkout

      - run:
          name: Run Maestro Tests
          command: |
            npx --yes @testingbot/cli@latest maestro \
              ./app.apk \
              ./maestro-flows \
              --device "Pixel 8" \
              --deviceVersion "14" \
              --report junit \
              --report-output-dir test-results \
              --commit-sha $CIRCLE_SHA1 \
              --pull-request-id $CIRCLE_PR_NUMBER \
              --repo-owner $CIRCLE_PROJECT_USERNAME \
              --repo-name $CIRCLE_PROJECT_REPONAME

      - store_test_results:
          path: test-results

      - store_artifacts:
          path: test-results
          destination: maestro-results

workflows:
  test:
    jobs:
      - maestro-tests

Add TB_KEY and TB_SECRET as environment variables in your CircleCI project settings. CircleCI automatically masks these values in build output.

Bitbucket Pipelines

Add Maestro testing to your bitbucket-pipelines.yml. Store your credentials in Repository settings → Pipelines → Repository variables and enable "Secured".

image: node:20

pipelines:
  default:
    - step:
        name: Build and Test
        caches:
          - node
        script:
          - ./gradlew assembleDebug
          - npx --yes @testingbot/cli@latest maestro
              app/build/outputs/apk/debug/app-debug.apk
              ./maestro-flows
              --device "Pixel 8"
              --deviceVersion "14"
              --report junit
              --report-output-dir test-results
              --commit-sha $BITBUCKET_COMMIT
              --pull-request-id $BITBUCKET_PR_ID
              --repo-owner $BITBUCKET_WORKSPACE
              --repo-name $BITBUCKET_REPO_SLUG
        artifacts:
          - test-results/**

Add TB_KEY and TB_SECRET as secured repository variables in Bitbucket. Secured variables are encrypted and masked in build logs.

Frequently asked questions

Which CI/CD systems can run Maestro tests on TestingBot?

Any CI runner that can execute a shell command. The page below has copy-paste-ready examples for GitHub Actions, GitLab CI, Jenkins, CircleCI and Bitbucket Pipelines. The same TestingBot CLI works on any other CI runner with a shell.

How do I store TestingBot credentials in CI securely?

Set TB_KEY and TB_SECRET as encrypted CI secrets. GitHub Actions uses repository or environment secrets, GitLab CI uses protected variables, Jenkins uses credentials, CircleCI uses environment variables in contexts, and Bitbucket uses secured repository variables. Never commit the key or secret in plain text.

What exit codes does the TestingBot CLI return?

Exit code 0 when every flow passed, 2 when one or more flows failed, and 1 for a CLI or infrastructure error such as a failed upload or missing credentials. See the Exit Codes section for the full mapping so you can wire conditional follow-up steps in CI.

How do I publish Maestro test reports in CI?

The CLI writes JUnit, HTML or Allure reports that most CI runners can parse to show per-flow results, and prints a machine-readable JSON document with --json. See the Test Reports section for the file path and configuration in each CI system.

How do I link CI metadata (branch, commit, pull request) to TestingBot test runs?

Pass --repo-owner, --repo-name, --branch, --commit-sha, --pull-request-id and --pr-url flags to the CLI, or use the TestingBot Maestro GitHub Action which fills them in from the workflow context. The values appear in the TestingBot dashboard, and with the TestingBot GitHub App installed they also post a status check on the pull request. See the CI/CD Metadata section.

Can I check on an async Maestro run later from CI?

Yes. Start the run with --async, then use testingbot status --id <projectId> --wait in a later job to block until it finishes, and testingbot artifacts to fetch reports and videos. See the Async runs section.