Espresso Testing
TestingBot provides access to Android physical devices and Android emulators, capable of running your Android Espresso tests.
Espresso is a mobile automation framework from Google, designed to run UI tests for Android applications.
Using Espresso in combination with TestingBot's Android cloud service is very straightforward:
- Upload your native mobile Android app, together with the Espresso test apk file. These are created with either Java or Kotlin.
- Specify on which devices you want to run the Espresso tests.
- Retrieve the results of the Espresso tests.
Getting Started
To get started, please follow the steps below:
-
You will need a
keyandsecretto connect to the TestingBot grid.
These credentials are available in the TestingBot member area. -
Espresso tests require a separate
apkfile for testing. So you will need to upload both the app (.apkor.aabfile) and the test file (.apk).
You can also use the TestingBot Espresso Demo App to give the Espresso Cloud Testing functionality a try.
The 1.0.0 release of the demo has one test, AdvancedCalculatorTest.ensureLargeNumbersWork, that fails because of a typo in its expected value. Expect 5 of 6 tests to pass with it.
TestingBot CLI
The easiest way to run Espresso tests on TestingBot is using our CLI tool. It handles uploading your app, test APK, and running tests in a single command.
Installation
npm install -g @testingbot/cli
Authentication
Authenticate using this command:
testingbot login
This opens your browser for authentication. After logging in, your credentials are saved to ~/.testingbot and you can start using the CLI.
Run testingbot whoami to see which account the CLI is using, your plan, minutes left and concurrency, and where the credentials come from (options, TB_KEY/TB_SECRET or ~/.testingbot). testingbot logout deletes ~/.testingbot.
Other Authentication Methods
-
Command-line options:
--api-keyand--api-secret -
Environment variables:
TB_KEYandTB_SECRET -
Config file: Create
~/.testingbotwith contentkey:secret
Quick Start
Run Espresso tests with a single command:
testingbot espresso app.apk app-test.apk --device "Pixel 8" --platform-version "14"
testingbot espresso app.apk app-test.apk --device "Galaxy S24" --real-device
To see the device names and OS versions that --device and --platform-version accept, run testingbot devices --framework espresso. Add --real to list real devices with their current availability, or --json for machine-readable output.
The CLI will:
- Upload your app APK to TestingBot
- Upload your test APK
- Start the test run
- Show real-time progress and results
If you build your Android app with Gradle, our Gradle plugin can upload your app and run Espresso tests straight from your build, no separate CLI step required.
Upload Android App
To run Android Espresso tests on TestingBot, you will first have to upload your .apk or .aab file to TestingBot Storage.
When using the CLI, the app upload is handled automatically as part of the testingbot espresso command. Simply specify your app APK file as the first argument:
testingbot espresso /path/to/app/Application-debug.apk /path/to/test/Application-debug-test.apk --device "Pixel 8"
Alternatively, you can use the --app option:
testingbot espresso --app /path/to/app/Application-debug.apk --test-app /path/to/test/Application-debug-test.apk --device "Pixel 8"
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/espresso/app" \
-F "file=@/path/to/app/file/Application-debug.apk"
The API response will include an id response, which you can use in the other API calls.
{
"id": 4012
}
Upload Espresso Test Suite
You can now upload the .apk file containing the Espresso tests.
The upload process is similar to the app upload process.
When using the CLI, the test APK upload is handled automatically. Specify your test APK as the second argument:
testingbot espresso app.apk app-test.apk --device "Pixel 8"
Or use the --test-app option:
testingbot espresso --app app.apk --test-app app-test.apk --device "Pixel 8"
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/espresso/:id/tests" \
-F "file=@/path/to/app/file/Application-debug-test.apk"
Replace the :id with the identifier you received during the app upload call.
Specify devices
You can specify one or more Android device capabilities with your API call.
These capabilities can target either Android emulators, or physical Android devices.
We offer special parameters which you can use to allocate a device:
| Regex Input | Result |
|---|---|
"Pixel.*" |
This will allocate any available Pixel device (phone) |
"*" |
This will allocate a random available Android device |
"Pixel [8-9]" |
This will allocate either an Pixel 8 or 9 |
Some Examples:
# find any Samsung Galaxy, except the S23 family
-d '{"capabilities":[{"deviceName":"^Galaxy(?!.*23).*$", "platformName":"Android", "realDevice": true}]}'
# find any device which name starts with Pixel
-d '{"capabilities":[{"deviceName":"Pixel.*", "platformName":"Android", "realDevice": true}]}'
Tablet Only
You can specify the tabletOnly capability when you only want to allocate a tablet device.
Phone Only
You can specify the phoneOnly capability when you only want to allocate a phone device.
-d '{"capabilities":[{"deviceName":"*", "tabletOnly": true, "platformName":"Android", "realDevice": true}]}'
-d '{"capabilities":[{"deviceName":"*", "phoneOnly": true, "platformName":"Android", "realDevice": true}]}'
Run Espresso Tests
Now that you have uploaded both your Android application and the Espresso test suite, you can run your first Espresso test on the TestingBot cloud.
Use the CLI to run tests with device selection options:
# Basic usage with device selection
testingbot espresso app.apk app-test.apk --device "Pixel 8" --platform-version "14"
# Real device
testingbot espresso app.apk app-test.apk --device "Galaxy S24" --real-device
# With test name and build identifier
testingbot espresso app.apk app-test.apk \
--device "Pixel 8" \
--platform-version "14" \
--name "Smoke Tests" \
--build "v2.1.0"
Device Options
| Option | Description |
|---|---|
--device <name> |
Device name (e.g., "Pixel 8", "Samsung.*") |
--platform-version <version> |
Android version (e.g., "12", "13", "14") |
--real-device |
Use a real device instead of emulator |
--tablet-only |
Only allocate tablet devices |
--phone-only |
Only allocate phone devices |
Test Filtering
# Run specific test classes
testingbot espresso app.apk app-test.apk \
--device "Pixel 8" \
--class "com.example.LoginTest,com.example.HomeTest"
# Run tests with specific annotations
testingbot espresso app.apk app-test.apk \
--device "Pixel 8" \
--annotation "com.example.SmokeTest" \
--size "small,medium"
# Exclude specific test classes
testingbot espresso app.apk app-test.apk \
--device "Pixel 8" \
--not-class "com.example.SlowTest"
Test Filtering Options
| Option | Description |
|---|---|
--class <classes> |
Run tests in specific classes (comma-separated fully qualified names) |
--not-class <classes> |
Exclude tests in specific classes |
--package <packages> |
Run tests in specific packages (comma-separated) |
--not-package <packages> |
Exclude tests in specific packages |
--annotation <annotations> |
Run tests with specific annotations (comma-separated) |
--not-annotation <annotations> |
Exclude tests with specific annotations |
--size <sizes> |
Run tests by size: small, medium, large (comma-separated) |
Network & Location
# With network throttling and geolocation
testingbot espresso app.apk app-test.apk \
--device "Pixel 8" \
--throttle-network "3G" \
--geo-country-code "DE" \
--language "de"
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/espresso/:id/run" \
-d '{"capabilities":[{"version":"12", "deviceName":"Pixel 6", "platformName":"Android"}]}' \
-H "Content-Type: application/json"
Replace the :id with the identifier you received during the app upload call.
The API response will include a success response, indicating if the tests will start or if an error occurred.
{
"success": true
}
View Espresso Test Results
You will now see a test appear in the TestingBot dashboard, together with a video, test logs and other metadata.
You can also use the CLI or an API call to get the results from the run(s):
By default, the CLI waits for test completion and displays real-time progress:
testingbot espresso app.apk app-test.apk --device "Pixel 8"
The CLI will show:
- Upload progress for app and test APK
- Device allocation status
- Live output from Espresso tests
- Final pass/fail status
Async Mode
Use --async to start tests without waiting for results:
testingbot espresso app.apk app-test.apk --device "Pixel 8" --async
Download JUnit Report
Download the JUnit report after test completion:
testingbot espresso app.apk app-test.apk --device "Pixel 8" \
--report junit \
--report-output-dir ./reports
Espresso runs only produce a JUnit report, so --report html is rejected before anything is uploaded.
Debugging
Add --debug to log the API responses the CLI receives, which helps when a run does not behave as expected.
curl -u api_key:api_secret \
"https://api.testingbot.com/v1/app-automate/espresso/:id"
Replace the :id with the identifier you received during the app upload call.
The API response will include a response indicating the current state of the Espresso test results.
{
"runs": [
{
"status": "DONE",
"capabilities": {
"version": "12",
"deviceName": "Pixel 6",
"platformName": "Android"
},
"success": false,
"test": {
"sessionId": "30f642d082c3-804f3c0ec2df-a18b2d431ffb-169920992163-48455547",
"environment": {
"name": "chrome",
"os": "Pixel 6 - 12.0",
"version": "12.0"
}
}
}
],
"version": "1.0"
}
Retry Failed Tests
Pass --retry <count> (0-2) to start a failed Espresso run again on the app you already uploaded, up to that many times.
Only the last attempt counts: it decides the exit code, the --report and the downloaded artifacts.
testingbot espresso app.apk app-test.apk --device "Pixel 8" --retry 2
--retry cannot be combined with --async, since retrying needs the result of the run.
Timeout
The CLI waits up to 60 minutes for the results. Pass --timeout <minutes> to change that.
The clock starts when the tests start, after the uploads, and includes every --retry attempt.
When the time is up, the CLI stops the runs that are still queued or running, prints the results so far and fails with exit code 2.
testingbot espresso app.apk app-test.apk --device "Pixel 8" --timeout 30
Queue Priority
When all your parallel slots are busy, new runs wait for one to free up.
Pass --priority high to start these runs before your other waiting runs, for example a release candidate or a hotfix, or --priority low for a nightly run that can wait.
Runs with the same priority start oldest first; the default is normal.
testingbot espresso app.apk app-test.apk --device "Pixel 8" --priority high
Priority only orders your own waiting runs. It does not take slots from other accounts or raise your parallel limit.
Download Artifacts
Use --download-artifacts to download the video, logs and screenshots of each run once the tests have finished.
They are saved as one zip, with a folder per run, named after --name (or timestamped).
--download-artifacts failed only downloads the runs that failed.
testingbot espresso app.apk app-test.apk --device "Pixel 8" \
--download-artifacts failed \
--artifacts-output-dir ./artifacts
Job Summary and Annotations
In GitHub Actions the CLI adds the results to the job summary, with a link to each run, and annotates every failed run so it shows on the workflow run and the pull request.
In Azure Pipelines the same table is attached to the build and failures are logged as errors.
Set TESTINGBOT_CI_SUMMARY=0 to turn this off.
CI/CD Metadata
Attach the branch, commit, pull request and repository a run was built from. The run page in the dashboard then links to them on your Git host: GitHub, GitLab, Bitbucket, Azure Repos or a self-hosted server.
Inside GitHub Actions, GitLab CI, Bitbucket Pipelines, Azure Pipelines and CircleCI, the CLI reads the repository, branch, commit and pull request from the CI environment, so no flags are needed. Outside those, or to override what was detected, pass them yourself:
testingbot espresso app.apk app-test.apk --device "Pixel 8" \
--branch "feature/login" \
--commit-sha "4e976b0ce06454ca61bf5d3c8b20ac0fdc457021" \
--pr-url "https://github.com/acme/app/pull/42" \
-m team=mobile -m env=staging
| Option | Description |
|---|---|
--branch <name> |
Git branch this run was built from |
--commit-sha <sha> |
Git commit the run tested |
--pull-request-id <id> |
Pull request (or merge request) number |
--pr-url <url> |
Pull request URL |
--repo-owner <owner>, --repo-name <name>
|
A GitHub repository (organization or user, and repository name) |
-m, --metadata <KEY=VALUE> |
Free-form key/value pairs stored with the run (repeatable) |
Send a metadata object with the run call. Every field is optional; repoUrl is the repository's web URL on any Git host.
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/espresso/:id/run" \
-d '{
"capabilities": [{ "deviceName": "Pixel 8" }],
"metadata": {
"repoUrl": "https://dev.azure.com/acme/mobile/_git/app",
"branch": "feature/login",
"commitSha": "4e976b0ce06454ca61bf5d3c8b20ac0fdc457021",
"pullRequestId": "42",
"custom": { "team": "mobile" }
}
}' \
-H "Content-Type: application/json"