Running Maestro in CI Pipelines
On this page
A suite that only runs on someone's laptop stops being trusted within weeks. This chapter puts the course's workspace into CI in two ways: running Maestro itself on an Android emulator inside the CI job, and handing the run to a device cloud so the CI job only uploads and waits. Both use GitHub Actions; the second approach also works from GitLab, Jenkins, CircleCI or any shell.
Option A: Maestro on an emulator inside the job#
GitHub's Linux runners can run a hardware-accelerated Android emulator once KVM is enabled. The community action reactivecircus/android-emulator-runner creates and boots the emulator and runs your script against it:
name: Maestro (emulator)
on:
pull_request:
jobs:
maestro:
runs-on: ubuntu-latest
env:
MAESTRO_DRIVER_STARTUP_TIMEOUT: 180000
steps:
- uses: actions/checkout@v4
- uses: actions/setup-java@v4
with:
distribution: temurin
java-version: '17'
- name: Enable KVM
run: |
echo 'KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"' | sudo tee /etc/udev/rules.d/99-kvm4all.rules
sudo udevadm control --reload-rules
sudo udevadm trigger --name-match=kvm
- name: Install Maestro
run: |
export MAESTRO_VERSION=2.12.1
curl -fsSL "https://get.maestro.mobile.dev" | bash
echo "$HOME/.maestro/bin" >> "$GITHUB_PATH"
- name: Get the app
run: curl -LO https://testingbot.com/appium/sample.apk
- name: Run flows on an emulator
uses: reactivecircus/android-emulator-runner@v2
with:
api-level: 34
target: google_apis
arch: x86_64
script: |
adb install sample.apk
maestro test -e APP_ID=com.sampleapp --exclude-tags=flaky --format junit --output report.xml --test-output-dir maestro-results .maestro/
- uses: actions/upload-artifact@v4
if: always()
with:
name: maestro-results
path: |
report.xml
maestro-results
Points worth understanding rather than copying:
-
Pin the Maestro version with
MAESTRO_VERSION. The install script otherwise installs whatever was released last night. -
MAESTRO_DRIVER_STARTUP_TIMEOUTraises the time Maestro waits for its on-device driver to start (the default is 15 seconds on Android, 120 on iOS). Cold CI emulators are slow; this avoids a common false failure. -
Upload artifacts with
if: always(), otherwise the screenshots you need disappear exactly when a flow fails. - In a real project, replace the download step with your build (
./gradlew assembleDebug) and install the APK it produces.
The tradeoffs are real. You get one emulator per job, so the suite runs serially unless you build a job matrix. There are no physical devices, and an iOS equivalent needs macOS runners with simulators. Emulator boot time and flakiness become part of your test results. For a small Android-only smoke suite this is a reasonable start; beyond that, most teams move the devices out of the CI job.
On a GitHub-hosted runner with a freshly booted emulator, Maestro fails before the first command because its on-device driver did not start in time. What helps?
Cold CI emulators are slow, and the default driver start-up wait on Android is short. Raising it avoids a common false failure; without KVM the emulator would be slower still.
Option B: hand the run to a device cloud#
Here the CI job builds the app and calls the TestingBot CLI (chapter 7), which uploads the app and flows and waits for the result. The job needs no emulator, no KVM and no macOS runner, even for iOS: the flows run on TestingBot's simulators and devices, not on the runner.
TestingBot publishes a GitHub Action that wraps the CLI and fills in the branch, commit, repository and pull request from the workflow context. Store TB_KEY and TB_SECRET as repository secrets first.
name: Maestro Tests
on:
push:
branches: [main]
pull_request:
branches: [main]
jobs:
maestro-tests:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Build Android App
run: ./gradlew assembleDebug
- name: Run Maestro Tests on TestingBot
id: maestro
uses: testingbot/maestro-action@v1
with:
api-key: ${{ secrets.TB_KEY }}
api-secret: ${{ secrets.TB_SECRET }}
app-file: app/build/outputs/apk/debug/app-debug.apk
workspace: .maestro
device: Pixel 9
device-version: '15'
exclude-tags: flaky
report: junit
download-artifacts: failed
- name: Upload Test Results
uses: actions/upload-artifact@v4
if: always()
with:
name: maestro-results
path: |
testingbot-reports
testingbot-artifacts
Every CLI option is available as an input (device-matrix, real-device, include-tags, env, retry, shard-split, maestro-version and more), and the step exposes outputs such as outcome, console-url and app-id. Pass app-id as app-binary-id to a later step to run a second suite against the same upload without uploading again. The GitHub Actions page lists every input and output.
Without the action#
The CLI runs anywhere Node.js does, which is all you need for GitLab CI, Jenkins or CircleCI:
# .gitlab-ci.yml (test stage only)
maestro-tests:
stage: test
image: node:20
script:
- npx --yes @testingbot/cli@latest maestro
app/build/outputs/apk/debug/app-debug.apk
./.maestro
--device "Pixel 8"
--deviceVersion "14"
--report junit
--report-output-dir ./test-results
artifacts:
when: always
reports:
junit: test-results/*.xml
Set TB_KEY and TB_SECRET as masked CI/CD variables. Inside GitHub Actions, GitLab CI, Bitbucket Pipelines, Azure Pipelines and CircleCI the CLI detects the repository, branch, commit and pull request on its own. In production, consider pinning the CLI version rather than using @latest.
Your team needs iOS coverage in CI but only has Linux runners. With the device-cloud approach, what does the CI job need?
The job only uploads the app and flows and waits for the result, so it needs no emulator, KVM or macOS, even for iOS.
Exit codes are a contract#
The TestingBot CLI exits with 0 when all flows passed, 2 when one or more flows failed, and 1 for CLI or infrastructure errors (bad arguments, missing credentials, upload failure, timeout). That lets a pipeline treat a red test differently from a broken pipeline, for example retrying the job on 1 but failing the build on 2. If you would rather gate on a results file, --json-file writes the results to a JSON file and keeps the exit code at 0 when flows fail.
Long suites without holding a runner#
For a nightly regression run, there is no need to keep a CI machine waiting. Start asynchronously, then check back:
PROJECT_ID=$(npx --yes @testingbot/cli@latest maestro app.apk ./.maestro --async --json | jq -r .appId)
npx --yes @testingbot/cli@latest status --id "$PROJECT_ID" --wait
npx --yes @testingbot/cli@latest artifacts --id "$PROJECT_ID" --report junit --report-output-dir ./reports
Gating merges#
With the TestingBot GitHub App installed, a run that carries the repository and the full commit SHA posts a TestingBot / tests status check on the pull request, which you can make a required check. If iOS and Android run as separate jobs, give each a check-name so they post separate checks. See GitHub PR checks for Maestro.
Separate Android and iOS jobs both post a TestingBot / tests check, and the second result overwrites the first on the pull request. What is the fix?
A distinct check-name makes each job post its own status check, so both can be required.
A pipeline shape that works#
-
On every pull request:
--include-tags=smoke, one Android and one iOS target, a few minutes at most. -
On merge to main: the full
regressionset on a device matrix that mirrors your users' devices. -
Always:
flakyexcluded, reports and artifacts uploaded, and a named owner for every quarantined flow.
Expo and EAS users can pass an EAS build URL straight to the CLI; the Maestro CI/CD overview links the Expo, Jenkins, CircleCI and Bitbucket examples.