Skip to main content

Espresso Testing

TestingBot provides access to Android physical devices and Android emulators, capable of running your Android Espresso tests.

Espresso is a mobile automation framework from Google, designed to run UI tests for Android applications.

Using Espresso in combination with TestingBot's Android cloud service is very straightforward:

  • Upload your native mobile Android app, together with the Espresso test apk file. These are created with either Java or Kotlin.
  • Specify on which devices you want to run the Espresso tests.
  • Retrieve the results of the Espresso tests.

Getting Started

To get started, please follow the steps below:

  1. You will need a key and secret to connect to the TestingBot grid.
    These credentials are available in the TestingBot member area.

  2. Espresso tests require a separate apk file for testing. So you will need to upload both the app (.apk or .aab file) and the test file (.apk).

You can also use the TestingBot Espresso Demo App to give the Espresso Cloud Testing functionality a try.

The 1.0.0 release of the demo has one test, AdvancedCalculatorTest.ensureLargeNumbersWork, that fails because of a typo in its expected value. Expect 5 of 6 tests to pass with it.

TestingBot CLI

The easiest way to run Espresso tests on TestingBot is using our CLI tool. It handles uploading your app, test APK, and running tests in a single command.

Installation

npm install -g @testingbot/cli

Authentication

Authenticate using this command:

testingbot login

This opens your browser for authentication. After logging in, your credentials are saved to ~/.testingbot and you can start using the CLI.

Run testingbot whoami to see which account the CLI is using, your plan, minutes left and concurrency, and where the credentials come from (options, TB_KEY/TB_SECRET or ~/.testingbot). testingbot logout deletes ~/.testingbot.

Other Authentication Methods

  • Command-line options: --api-key and --api-secret
  • Environment variables: TB_KEY and TB_SECRET
  • Config file: Create ~/.testingbot with content key:secret

Quick Start

Run Espresso tests with a single command:

testingbot espresso app.apk app-test.apk --device "Pixel 8" --platform-version "14"
testingbot espresso app.apk app-test.apk --device "Galaxy S24" --real-device

To see the device names and OS versions that --device and --platform-version accept, run testingbot devices --framework espresso. Add --real to list real devices with their current availability, or --json for machine-readable output.

The CLI will:

  1. Upload your app APK to TestingBot
  2. Upload your test APK
  3. Start the test run
  4. Show real-time progress and results

If you build your Android app with Gradle, our Gradle plugin can upload your app and run Espresso tests straight from your build, no separate CLI step required.

Upload Android App

To run Android Espresso tests on TestingBot, you will first have to upload your .apk or .aab file to TestingBot Storage.

When using the CLI, the app upload is handled automatically as part of the testingbot espresso command. Simply specify your app APK file as the first argument:

testingbot espresso /path/to/app/Application-debug.apk /path/to/test/Application-debug-test.apk --device "Pixel 8"

Alternatively, you can use the --app option:

testingbot espresso --app /path/to/app/Application-debug.apk --test-app /path/to/test/Application-debug-test.apk --device "Pixel 8"
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/espresso/app" \
-F "file=@/path/to/app/file/Application-debug.apk"

The API response will include an id response, which you can use in the other API calls.

{
    "id": 4012
}

Upload Espresso Test Suite

You can now upload the .apk file containing the Espresso tests.
The upload process is similar to the app upload process.

When using the CLI, the test APK upload is handled automatically. Specify your test APK as the second argument:

testingbot espresso app.apk app-test.apk --device "Pixel 8"

Or use the --test-app option:

testingbot espresso --app app.apk --test-app app-test.apk --device "Pixel 8"
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/espresso/:id/tests" \
-F "file=@/path/to/app/file/Application-debug-test.apk"

Replace the :id with the identifier you received during the app upload call.

Specify devices

You can specify one or more Android device capabilities with your API call.
These capabilities can target either Android emulators, or physical Android devices.

We offer special parameters which you can use to allocate a device:

Regex Input Result
"Pixel.*" This will allocate any available Pixel device (phone)
"*" This will allocate a random available Android device
"Pixel [8-9]" This will allocate either an Pixel 8 or 9

Some Examples:

# find any Samsung Galaxy, except the S23 family
-d '{"capabilities":[{"deviceName":"^Galaxy(?!.*23).*$", "platformName":"Android", "realDevice": true}]}'

# find any device which name starts with Pixel
-d '{"capabilities":[{"deviceName":"Pixel.*", "platformName":"Android", "realDevice": true}]}'

Tablet Only

You can specify the tabletOnly capability when you only want to allocate a tablet device.

Phone Only

You can specify the phoneOnly capability when you only want to allocate a phone device.


-d '{"capabilities":[{"deviceName":"*", "tabletOnly": true, "platformName":"Android", "realDevice": true}]}'

-d '{"capabilities":[{"deviceName":"*", "phoneOnly": true, "platformName":"Android", "realDevice": true}]}'

Run Espresso Tests

Now that you have uploaded both your Android application and the Espresso test suite, you can run your first Espresso test on the TestingBot cloud.

Use the CLI to run tests with device selection options:

# Basic usage with device selection
testingbot espresso app.apk app-test.apk --device "Pixel 8" --platform-version "14"

# Real device
testingbot espresso app.apk app-test.apk --device "Galaxy S24" --real-device

# With test name and build identifier
testingbot espresso app.apk app-test.apk \
  --device "Pixel 8" \
  --platform-version "14" \
  --name "Smoke Tests" \
  --build "v2.1.0"

Device Options

Option Description
--device <name> Device name (e.g., "Pixel 8", "Samsung.*")
--platform-version <version> Android version (e.g., "12", "13", "14")
--real-device Use a real device instead of emulator
--tablet-only Only allocate tablet devices
--phone-only Only allocate phone devices

Test Filtering

# Run specific test classes
testingbot espresso app.apk app-test.apk \
  --device "Pixel 8" \
  --class "com.example.LoginTest,com.example.HomeTest"

# Run tests with specific annotations
testingbot espresso app.apk app-test.apk \
  --device "Pixel 8" \
  --annotation "com.example.SmokeTest" \
  --size "small,medium"

# Exclude specific test classes
testingbot espresso app.apk app-test.apk \
  --device "Pixel 8" \
  --not-class "com.example.SlowTest"

Test Filtering Options

Option Description
--class <classes> Run tests in specific classes (comma-separated fully qualified names)
--not-class <classes> Exclude tests in specific classes
--package <packages> Run tests in specific packages (comma-separated)
--not-package <packages> Exclude tests in specific packages
--annotation <annotations> Run tests with specific annotations (comma-separated)
--not-annotation <annotations> Exclude tests with specific annotations
--size <sizes> Run tests by size: small, medium, large (comma-separated)

Network & Location

# With network throttling and geolocation
testingbot espresso app.apk app-test.apk \
  --device "Pixel 8" \
  --throttle-network "3G" \
  --geo-country-code "DE" \
  --language "de"
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/espresso/:id/run" \
-d '{"capabilities":[{"version":"12", "deviceName":"Pixel 6", "platformName":"Android"}]}' \
-H "Content-Type: application/json"

Replace the :id with the identifier you received during the app upload call.

The API response will include a success response, indicating if the tests will start or if an error occurred.

{
    "success": true
}

View Espresso Test Results

You will now see a test appear in the TestingBot dashboard, together with a video, test logs and other metadata.

You can also use the CLI or an API call to get the results from the run(s):

By default, the CLI waits for test completion and displays real-time progress:

testingbot espresso app.apk app-test.apk --device "Pixel 8"

The CLI will show:

  • Upload progress for app and test APK
  • Device allocation status
  • Live output from Espresso tests
  • Final pass/fail status

Async Mode

Use --async to start tests without waiting for results:

testingbot espresso app.apk app-test.apk --device "Pixel 8" --async

Download JUnit Report

Download the JUnit report after test completion:

testingbot espresso app.apk app-test.apk --device "Pixel 8" \
  --report junit \
  --report-output-dir ./reports

Espresso runs only produce a JUnit report, so --report html is rejected before anything is uploaded.

Debugging

Add --debug to log the API responses the CLI receives, which helps when a run does not behave as expected.

curl -u api_key:api_secret \
"https://api.testingbot.com/v1/app-automate/espresso/:id"

Replace the :id with the identifier you received during the app upload call.

The API response will include a response indicating the current state of the Espresso test results.

{
  "runs": [
      {
          "status": "DONE",
          "capabilities": {
              "version": "12",
              "deviceName": "Pixel 6",
              "platformName": "Android"
          },
          "success": false,
          "test": {
              "sessionId": "30f642d082c3-804f3c0ec2df-a18b2d431ffb-169920992163-48455547",
              "environment": {
                  "name": "chrome",
                  "os": "Pixel 6 - 12.0",
                  "version": "12.0"
              }
          }
      }
  ],
  "version": "1.0"
}

Retry Failed Tests

Pass --retry <count> (0-2) to start a failed Espresso run again on the app you already uploaded, up to that many times. Only the last attempt counts: it decides the exit code, the --report and the downloaded artifacts.

testingbot espresso app.apk app-test.apk --device "Pixel 8" --retry 2

--retry cannot be combined with --async, since retrying needs the result of the run.

Timeout

The CLI waits up to 60 minutes for the results. Pass --timeout <minutes> to change that. The clock starts when the tests start, after the uploads, and includes every --retry attempt. When the time is up, the CLI stops the runs that are still queued or running, prints the results so far and fails with exit code 2.

testingbot espresso app.apk app-test.apk --device "Pixel 8" --timeout 30

Queue Priority

When all your parallel slots are busy, new runs wait for one to free up. Pass --priority high to start these runs before your other waiting runs, for example a release candidate or a hotfix, or --priority low for a nightly run that can wait. Runs with the same priority start oldest first; the default is normal.

testingbot espresso app.apk app-test.apk --device "Pixel 8" --priority high

Priority only orders your own waiting runs. It does not take slots from other accounts or raise your parallel limit.

Download Artifacts

Use --download-artifacts to download the video, logs and screenshots of each run once the tests have finished. They are saved as one zip, with a folder per run, named after --name (or timestamped). --download-artifacts failed only downloads the runs that failed.

testingbot espresso app.apk app-test.apk --device "Pixel 8" \
  --download-artifacts failed \
  --artifacts-output-dir ./artifacts

Job Summary and Annotations

In GitHub Actions the CLI adds the results to the job summary, with a link to each run, and annotates every failed run so it shows on the workflow run and the pull request. In Azure Pipelines the same table is attached to the build and failures are logged as errors. Set TESTINGBOT_CI_SUMMARY=0 to turn this off.

CI/CD Metadata

Attach the branch, commit, pull request and repository a run was built from. The run page in the dashboard then links to them on your Git host: GitHub, GitLab, Bitbucket, Azure Repos or a self-hosted server.

Inside GitHub Actions, GitLab CI, Bitbucket Pipelines, Azure Pipelines and CircleCI, the CLI reads the repository, branch, commit and pull request from the CI environment, so no flags are needed. Outside those, or to override what was detected, pass them yourself:

testingbot espresso app.apk app-test.apk --device "Pixel 8" \
  --branch "feature/login" \
  --commit-sha "4e976b0ce06454ca61bf5d3c8b20ac0fdc457021" \
  --pr-url "https://github.com/acme/app/pull/42" \
  -m team=mobile -m env=staging
Option Description
--branch <name> Git branch this run was built from
--commit-sha <sha> Git commit the run tested
--pull-request-id <id> Pull request (or merge request) number
--pr-url <url> Pull request URL
--repo-owner <owner>, --repo-name <name> A GitHub repository (organization or user, and repository name)
-m, --metadata <KEY=VALUE> Free-form key/value pairs stored with the run (repeatable)

Send a metadata object with the run call. Every field is optional; repoUrl is the repository's web URL on any Git host.

curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/espresso/:id/run" \
-d '{
  "capabilities": [{ "deviceName": "Pixel 8" }],
  "metadata": {
    "repoUrl": "https://dev.azure.com/acme/mobile/_git/app",
    "branch": "feature/login",
    "commitSha": "4e976b0ce06454ca61bf5d3c8b20ac0fdc457021",
    "pullRequestId": "42",
    "custom": { "team": "mobile" }
  }
}' \
-H "Content-Type: application/json"