--- title: Maestro CI/CD Integration | TestingBot description: Run Maestro mobile tests on TestingBot from CI/CD using the TestingBot CLI. GitHub Actions, GitLab CI, Jenkins. source_url: html: https://testingbot.com/support/app-automate/maestro/ci-cd md: https://testingbot.com/support/app-automate/maestro/ci-cd.md --- # CI/CD Integration The [TestingBot CLI](https://testingbot.com/support/app-automate/maestro#cli) integrates seamlessly with any CI/CD environment. It runs all your Maestro flows, streams the output in real-time and exits with the appropriate exit code for pipeline automation. We recommend using `npx` to run the CLI without installing it globally. This ensures you're always using the latest version. ## Quick Start Run Maestro tests in any CI/CD service with a single command: ```bash npx --yes @testingbot/cli@latest maestro app.apk ./flows \ --device "Pixel 8" \ --deviceVersion "14" \ --api-key $TB_KEY \ --api-secret $TB_SECRET ``` This command will: - Upload your mobile app (`.apk`, `.aab`, `.ipa` or `.app`) - Upload and run all Maestro flow files from the `./flows` directory - Stream test progress to your CI logs - Exit with code `0` on success, non-zero on failure You can pass credentials as arguments (`--api-key` and `--api-secret`) or set them as environment variables. Environment variables are the recommended approach for CI/CD. ## Environment Variables Instead of passing credentials as command arguments, you can set them as environment variables. This is the most secure approach for CI/CD pipelines since credentials are never exposed in logs or command history. | Variable | Description | | --- | --- | | `TB_KEY` | Your TestingBot API key | | `TB_SECRET` | Your TestingBot API secret | When these environment variables are set, you can omit the credential arguments entirely: ```bash export TB_KEY="your_api_key" export TB_SECRET="your_api_secret" npx --yes @testingbot/cli@latest maestro app.apk ./flows \ --device "Pixel 8" \ --deviceVersion "14" ``` Never hardcode credentials in your CI configuration files. Always use your CI provider's secrets management feature to store `TB_KEY` and `TB_SECRET`. ## Exit Codes The CLI distinguishes a red test run from a broken pipeline, so a job can retry on infrastructure errors but fail the build on real test failures: | Exit Code | Description | | --- | --- | | `0` | All flows passed. Also returned for `--async`, `--dry-run`, and for failed flows when `--json-file` is used (the file is the contract then) | | `1` | CLI or infrastructure error: invalid arguments, missing credentials, upload failure, timeout | | `2` | One or more flows failed | `testingbot status` follows the same rule and additionally exits `0` while a project is still running. ## Test Reports Generate test reports for CI/CD result decoration and test analytics. The CLI supports JUnit XML, HTML and Allure report formats, plus a machine-readable JSON summary. ### JUnit Report Generate JUnit XML reports for integration with CI/CD test result visualization: ```bash npx --yes @testingbot/cli@latest maestro app.apk ./flows \ --device "Pixel 8" \ --deviceVersion "14" \ --report junit \ --report-output-dir ./test-results ``` | Option | Description | | --- | --- | | `--report ` | Report format: `junit` | | `--report-output-dir ` | Directory to save test reports (required when `--report` is used) | JUnit reports can be used with most CI/CD platforms for: - Test result visualization in your pipeline - Tracking test trends over time - Annotating pull requests with test results - Integration with test management tools You can also fetch reports via the [REST API](https://testingbot.com/support/app-automate/maestro/api#report) if you need to retrieve them separately. ### Allure Report `--report allure` converts each run's results into Allure result files under `/allure-results/`, one JSON per flow with its steps, timings, status and failure details. Render them with the [Allure CLI](https://allurereport.org/docs/install/); results from several runs or shards accumulate in the same directory. ```bash npx --yes @testingbot/cli@latest maestro app.apk ./flows --report allure --report-output-dir ./reports allure serve ./reports/allure-results ``` ### JSON Output `--json` prints a single JSON document on stdout and moves all log lines to stderr, so it pipes straight into `jq`. `--json-file` writes the same document to `_testingbot.json` (or the path given with `--json-file-name`) while keeping the normal console output. Both imply `--quiet`. ```bash npx --yes @testingbot/cli@latest maestro app.apk ./flows --json | jq '.runs[].flows[] | {name, passed, durationSeconds}' ``` ```json { "provider": "maestro", "outcome": "failed", "success": false, "appId": 1234, "url": "https://testingbot.com/members/maestro/1234", "runs": [ { "id": 5678, "status": "DONE", "passed": false, "device": { "name": "Pixel 9", "platform": "Android", "version": "14" }, "url": "https://testingbot.com/members/maestro/1234/runs/5678", "flows": [ { "id": 1, "runId": 5678, "name": "login", "status": "DONE", "passed": true, "attempt": 1, "latest": true, "durationSeconds": 30, "errors": [] } ] } ] } ``` `outcome` is one of `passed`, `failed`, `started` (async), `dry-run` or `error`. `flows` lists every attempt, including `--retry` re-runs; `latest` marks the attempt whose verdict counts. When the CLI fails before a verdict, the document still arrives with `outcome: "error"` and an `error` message, so a consumer never has to handle missing output. ## CI/CD Metadata Pass repository and commit information to link test runs with your source control. This enables better traceability and integration with your development workflow. | Option | Description | | --- | --- | | `--commit-sha ` | The commit SHA of this upload | | `--pull-request-id ` | The ID of the pull request this upload originated from | | `--repo-name ` | Repository name (e.g., GitHub repo slug) | | `--repo-owner ` | Repository owner (e.g., GitHub organization or user) | | `--branch ` | Git branch this run was built from | | `--pr-url ` | URL of the pull request this run originated from | | `-m, --metadata ` | Free-form metadata attached to the run (repeatable), for example `-m team=mobile -m env=staging` | Example with metadata: ```bash npx --yes @testingbot/cli@latest maestro app.apk ./flows \ --device "Pixel 8" \ --deviceVersion "14" \ --commit-sha "$COMMIT_SHA" \ --branch "$BRANCH" \ --pull-request-id "$PR_NUMBER" \ --repo-owner "your-org" \ --repo-name "your-repo" \ -m team=mobile ``` With the [TestingBot GitHub App](https://testingbot.com/support/app-automate/maestro/ci-cd/github-pr-checks) installed, a run that carries the repository and the full head commit SHA also posts a `TestingBot / tests` status check on the pull request, so you can require green Maestro flows before merging. No extra step is needed. ## Async runs, status and artifacts Long suites do not have to hold a CI runner. Start the run with `--async`, capture the project ID, and come back for the result in a later job or from a different machine. ```bash # Start and capture the project id PROJECT_ID=$(npx --yes @testingbot/cli@latest maestro app.apk ./flows --async --json | jq -r .appId) # Later: block until it finishes (exit 2 if any flow failed), with the live flow table npx --yes @testingbot/cli@latest status --id "$PROJECT_ID" --wait # Fetch reports and artifacts once it finished npx --yes @testingbot/cli@latest artifacts --id "$PROJECT_ID" --report junit --report-output-dir ./reports npx --yes @testingbot/cli@latest artifacts --id "$PROJECT_ID" --download-artifacts failed --artifacts-output-dir ./artifacts # Browse recent projects npx --yes @testingbot/cli@latest list --count 20 ``` `status --wait` only watches the project; interrupting it does not cancel the run. `status` without `--wait` prints a snapshot and exits `0` while the project is still running, so it also works for polling. ### Expo / EAS Build Pass an `eas build` download URL with `--app-url`, or run inside EAS Workflows with the `@testingbot/eas-workflow` wrapper. Both are covered on the [Expo / EAS page](https://testingbot.com/support/app-automate/maestro/ci-cd/expo-eas). ### Upload the app once, run many suites Every run prints its project ID after the app upload. Pass it to a later run with `--app-binary-id` to skip the upload entirely; each run still gets its own project and results. `testingbot upload` does the upload on its own and prints the ID. ```bash APP_ID=$(npx --yes @testingbot/cli@latest upload app.apk --json | jq -r .appId) npx --yes @testingbot/cli@latest maestro --app-binary-id "$APP_ID" ./flows/smoke --device "Pixel 9" npx --yes @testingbot/cli@latest maestro --app-binary-id "$APP_ID" ./flows/regression --device-matrix "Pixel 9:14,Samsung Galaxy S24:14:real" ``` ## View Result Details A link to the current test results will be printed to your CI logs during execution. The test details page provides detailed test result information, including: - Real-time test execution status - Video recordings of each test run - Screenshots captured during tests - Device logs and Maestro output - Test artifacts for debugging ## GitHub Actions The [TestingBot Maestro GitHub Action](https://testingbot.com/support/app-automate/maestro/ci-cd/github-actions) wraps the CLI, fills in branch, commit, repository and pull request from the workflow context, writes a results table to the job summary and exposes the results as outputs: ```yaml - uses: testingbot/maestro-action@v1 with: api-key: ${{ secrets.TB_KEY }} api-secret: ${{ secrets.TB_SECRET }} app-file: app/build/outputs/apk/debug/app-debug.apk workspace: .maestro ``` Inputs, outputs, a device-matrix example and the raw-CLI alternative are on the [GitHub Actions page](https://testingbot.com/support/app-automate/maestro/ci-cd/github-actions). ## GitHub PR status checks With the TestingBot GitHub App installed, every Maestro run started from a pull request posts a `TestingBot / tests` check listing each flow on each device, and can be made a required check so a pull request cannot be merged while a flow is failing. Nothing to configure beyond installing the app. See [GitHub PR Checks for Maestro](https://testingbot.com/support/app-automate/maestro/ci-cd/github-pr-checks). ## GitLab CI Add Maestro testing to your `.gitlab-ci.yml`. Store your credentials in **Settings → CI/CD → Variables** and mark them as protected and masked. ```yaml stages: - build - test build: stage: build image: gradle:8-jdk17 script: - ./gradlew assembleDebug artifacts: paths: - app/build/outputs/apk/debug/ maestro-tests: stage: test image: node:20 dependencies: - build script: - npx --yes @testingbot/cli@latest maestro app/build/outputs/apk/debug/app-debug.apk ./maestro-flows --device "Pixel 8" --deviceVersion "14" --report junit --report-output-dir ./test-results --commit-sha $CI_COMMIT_SHA --pull-request-id $CI_MERGE_REQUEST_IID --repo-owner $CI_PROJECT_NAMESPACE --repo-name $CI_PROJECT_NAME variables: TB_KEY: $TB_KEY TB_SECRET: $TB_SECRET artifacts: when: always paths: - test-results/ reports: junit: test-results/*.xml ``` Add `TB_KEY` and `TB_SECRET` as CI/CD variables in your GitLab project settings. Enable "Mask variable" to prevent them from appearing in job logs. ## Jenkins Add Maestro testing to your Jenkins pipeline (`Jenkinsfile`). Store your credentials using **Manage Jenkins → Manage Credentials** as secret text. ```groovy pipeline { agent any environment { TB_KEY = credentials('testingbot-api-key') TB_SECRET = credentials('testingbot-api-secret') } stages { stage('Build') { steps { sh './gradlew assembleDebug' } } stage('Maestro Tests') { steps { sh """ npx --yes @testingbot/cli@latest maestro \\ app/build/outputs/apk/debug/app-debug.apk \\ ./maestro-flows \\ --device "Pixel 8" \\ --deviceVersion "14" \\ --report junit \\ --report-output-dir test-results \\ --commit-sha ${env.GIT_COMMIT} \\ --pull-request-id ${env.CHANGE_ID ?: ''} \\ --repo-owner "your-org" \\ --repo-name "your-repo" """ } post { always { junit 'test-results/*.xml' archiveArtifacts artifacts: 'test-results/**', allowEmptyArchive: true } } } } } ``` Create credentials of type "Secret text" in Jenkins for `TB_KEY` and `TB_SECRET`. Reference them using the `credentials()` helper in your pipeline. ## CircleCI Add Maestro testing to your `.circleci/config.yml`. Store your credentials in **Project Settings → Environment Variables**. ```yaml version: 2.1 jobs: maestro-tests: docker: - image: cimg/node:20 steps: - checkout - run: name: Run Maestro Tests command: | npx --yes @testingbot/cli@latest maestro \ ./app.apk \ ./maestro-flows \ --device "Pixel 8" \ --deviceVersion "14" \ --report junit \ --report-output-dir test-results \ --commit-sha $CIRCLE_SHA1 \ --pull-request-id $CIRCLE_PR_NUMBER \ --repo-owner $CIRCLE_PROJECT_USERNAME \ --repo-name $CIRCLE_PROJECT_REPONAME - store_test_results: path: test-results - store_artifacts: path: test-results destination: maestro-results workflows: test: jobs: - maestro-tests ``` Add `TB_KEY` and `TB_SECRET` as environment variables in your CircleCI project settings. CircleCI automatically masks these values in build output. ## Bitbucket Pipelines Add Maestro testing to your `bitbucket-pipelines.yml`. Store your credentials in **Repository settings → Pipelines → Repository variables** and enable "Secured". ```yaml image: node:20 pipelines: default: - step: name: Build and Test caches: - node script: - ./gradlew assembleDebug - npx --yes @testingbot/cli@latest maestro app/build/outputs/apk/debug/app-debug.apk ./maestro-flows --device "Pixel 8" --deviceVersion "14" --report junit --report-output-dir test-results --commit-sha $BITBUCKET_COMMIT --pull-request-id $BITBUCKET_PR_ID --repo-owner $BITBUCKET_WORKSPACE --repo-name $BITBUCKET_REPO_SLUG artifacts: - test-results/** ``` Add `TB_KEY` and `TB_SECRET` as secured repository variables in Bitbucket. Secured variables are encrypted and masked in build logs. ## Frequently asked questions Which CI/CD systems can run Maestro tests on TestingBot? Any CI runner that can execute a shell command. The page below has copy-paste-ready examples for [GitHub Actions](https://testingbot.com#github), [GitLab CI](https://testingbot.com#gitlab), [Jenkins](https://testingbot.com#jenkins), [CircleCI](https://testingbot.com#circleci) and [Bitbucket Pipelines](https://testingbot.com#bitbucket). The same TestingBot CLI works on any other CI runner with a shell. How do I store TestingBot credentials in CI securely? Set TB\_KEY and TB\_SECRET as encrypted CI secrets. GitHub Actions uses repository or environment secrets, GitLab CI uses protected variables, Jenkins uses credentials, CircleCI uses environment variables in contexts, and Bitbucket uses secured repository variables. Never commit the key or secret in plain text. What exit codes does the TestingBot CLI return? Exit code 0 when every flow passed, 2 when one or more flows failed, and 1 for a CLI or infrastructure error such as a failed upload or missing credentials. See the [Exit Codes section](https://testingbot.com#exitcodes) for the full mapping so you can wire conditional follow-up steps in CI. How do I publish Maestro test reports in CI? The CLI writes JUnit, HTML or Allure reports that most CI runners can parse to show per-flow results, and prints a machine-readable JSON document with --json. See the [Test Reports section](https://testingbot.com#reports) for the file path and configuration in each CI system. How do I link CI metadata (branch, commit, pull request) to TestingBot test runs? Pass --repo-owner, --repo-name, --branch, --commit-sha, --pull-request-id and --pr-url flags to the CLI, or use the TestingBot Maestro GitHub Action which fills them in from the workflow context. The values appear in the TestingBot dashboard, and with the TestingBot GitHub App installed they also post a status check on the pull request. See the [CI/CD Metadata section](https://testingbot.com#metadata). Can I check on an async Maestro run later from CI? Yes. Start the run with --async, then use `testingbot status --id --wait` in a later job to block until it finishes, and `testingbot artifacts` to fetch reports and videos. See the [Async runs section](https://testingbot.com#async).