Mobilewright Testing

Quickstart

The fastest way to start is the scaffolder. It asks a few questions and writes a mobilewright.config.ts plus a sample test into your project. Add --yes to accept the defaults.

npx @testingbot/mobilewright-driver init

To set things up by hand instead, install the packages as dev dependencies:

npm install --save-dev mobilewright @mobilewright/test @testingbot/mobilewright-driver

Credentials

The driver reads your TestingBot key and secret from the environment. You can find them on your member edit page.

export TESTINGBOT_KEY=your_key
export TESTINGBOT_SECRET=your_secret

TB_KEY and TB_SECRET are accepted as aliases. If you would rather not use environment variables, pass key and secret to the driver directly. Do not commit them to your repository.

Configuration

Point Mobilewright at TestingBot by passing a TestingBotDriver instance in your mobilewright.config.ts. Because TestingBot sessions start with the app already installed, the app under test has to be declared up front in the apps option.

import { defineConfig } from 'mobilewright';
import { TestingBotDriver } from '@testingbot/mobilewright-driver';

export default defineConfig({
  testDir: '.',
  bundleId: 'com.example.MyApp',
  driver: new TestingBotDriver({
    apps: {
      android: './build/app.apk',
      'ios-simulator': './build/app-sim.zip',
      'ios-real': './build/app.ipa', // must be test-signed for real devices
    },
  }),
  projects: [
    { name: 'android', use: { platform: 'android', deviceType: 'emulator' } },
    { name: 'ios', use: { platform: 'ios', deviceType: 'real', osVersion: '>=17' } },
  ],
});

A few things worth knowing about apps:

• Local .apk, .ipa and simulator .zip paths are uploaded to TestingBot Storage automatically. Already-uploaded builds can be referenced with their tb:// URL.
• The keys are android, android-emulator, android-real, ios, ios-simulator and ios-real. The most specific one wins: ios-real overrides ios.
• A value can be an array. The first entry is the app under test, the rest are installed alongside it, which is how you ship a mock server or a test harness with your build.

A Mobilewright test looks the same whether it runs locally or on TestingBot:

import { test, expect } from '@mobilewright/test';

test('shows the welcome screen', async ({ device, screen, bundleId }) => {
  await device.launchApp(bundleId);
  await expect(screen.getByText('Welcome')).toBeVisible();
});

Selecting devices

Device selection comes from the Mobilewright project config, in the use block. Regular expressions and version ranges work on both real and virtual devices.

Real devices resolve against the live physical catalog, where idle devices are preferred. If the device you asked for is busy, TestingBot queues your session until it frees up rather than failing. Virtual devices resolve against the simulator and emulator catalog, where the newest matching OS version wins. See the supported device list for what is currently available.

Running tests

npx mobilewright test

Each device slot maps to a TestingBot session on your dashboard, with video and a pass or fail verdict reported automatically. Prefix the command with DEBUG=testingbot:* to print every allocation, hub command, upload and result the driver performs.

DEBUG=testingbot:* npx mobilewright test

Driver options

Everything the driver accepts. Only apps is required; the values below are examples, and the comments call out the real defaults. Anything left unset is simply not sent, in which case the TestingBot server-side default applies.

new TestingBotDriver({
  apps: { android: './build/app.apk' }, // app under test, most specific key wins
                                        // ('ios-real' beats 'ios'); tb:// URLs allowed.
                                        // Array = app under test first, helper apps after:
                                        //   android: ['./app.apk', './mock-server.apk']
  key: '...',                 // default: TESTINGBOT_KEY (or TB_KEY) env
  secret: '...',              // default: TESTINGBOT_SECRET (or TB_SECRET) env
  sessionPerTest: false,      // true = fresh session (and video) per test
  name: 'checkout flow',      // session name (default 'mobilewright')
  hubUrl: '...',              // default: https://hub.testingbot.com/wd/hub
  apiUrl: '...',              // default: https://api.testingbot.com/v1
  build: 'ci-1234',           // default: auto-detected from CI env / TESTINGBOT_BUILD
  testResults: 'on',          // 'off' disables pass/fail reporting

  allocationTimeout: 395_000, // ms to wait for a device (default 395_000)
  commandTimeout: 60_000,     // ms per WebDriver command (default 60_000)
  idleTimeout: 230,           // seconds before TestingBot reaps an idle session (default 230)
  maxDuration: 1800,          // max session length in seconds (unset by default)
  appiumVersion: '2.11.2',    // pin the Appium version (unset = TestingBot's default)

  timeZone: 'Europe/Brussels',    // device timezone (tz database name)
  geoCountryCode: 'DE',           // route device traffic via a proxy in this country
  throttleNetwork: '3G',          // or { downloadSpeed, uploadSpeed, latency, loss }
  autoGrantPermissions: true,     // Android: auto-grant app permission dialogs
  autoAcceptAlerts: true,         // iOS: auto-accept system permission alerts
  screenshots: true,              // screenshot at every step (default false)
  video: true,                    // session video (default true)
  recordLogs: 'strip-parameters', // command logs: true | false | 'strip-parameters'
  public: false,                  // make results publicly accessible
  extra: 'commit=abc123',         // custom metadata on the test detail page
  tabletOnly: false,              // restrict allocation to tablets
  phoneOnly: false,               // ...or to phones

  capabilities: {},           // extra Appium capabilities (escape hatch)
  tbOptions: {},              // extra tb:options entries (escape hatch)
});

capabilities and tbOptions are the escape hatches for anything the driver does not expose directly. See the Appium capabilities reference for what you can put in them.

TestingBot commands inside tests

Some TestingBot features are runtime commands rather than session capabilities. Import the testingbot helper and call it from a test body; it targets the device session Mobilewright connected for that test.

import { test, expect } from '@mobilewright/test';
import { testingbot } from '@testingbot/mobilewright-driver';

test('checkout survives a slow network', async ({ device, screen, bundleId }) => {
  await device.launchApp(bundleId);

  await testingbot.annotate('starting checkout');   // shows in the session timeline
  await testingbot.throttle('3G');
  await expect(screen.getByText('Order placed')).toBeVisible();

  console.log(testingbot.dashboardUrl());
});

Throttling is reset automatically when the test's device session is released, so slow-network conditions never leak into the next test that reuses the device. Calling a command outside a test body raises an explanatory error rather than a null reference.

Recipe: name every session after its test

test.beforeEach(async ({ device }, testInfo) => {
  await testingbot.setName(testInfo.title);
  await testingbot.setTags([testInfo.project.name]);
});

Requesting the device fixture is what makes the session exist before the hook runs. With sessionPerTest: true each session gets exactly one name; in pooled mode the last test to run on a session wins.

Sessions, naming and reporting

• By default a device slot's session is reused across tests, so you pay device startup once. A session that hosted exactly one test is renamed after that test and carries its own verdict. A session that hosted several keeps its configured name (mobilewright unless you set one) and gets a run summary in its status message instead, such as 12/12 tests passed, or 1 of 3 tests failed. ... with the individual failure messages appended.
• Set sessionPerTest: true for one session, one video and one named verdict per test. That is the clearest dashboard, at the cost of device startup on every test.
• Every session is tagged mobilewright, so you can filter the dashboard down to these runs. Git metadata is reported on top of that when Mobilewright captures it: the commit hash, branch, author and subject land in the session's extra field, alongside your own extra rather than replacing it, and the branch is added as a second tag. Enable it with captureGitInfo: { commit: true } in mobilewright.config.ts.
• Failure messages are reported to the session's status message. build groups sessions on the dashboard and is auto-detected from GitHub Actions, GitLab, CircleCI, Buildkite, Bitrise, Travis, Azure DevOps, Jenkins and TeamCity, or read from a TESTINGBOT_BUILD variable.

Testing localhost and staging servers

If your app talks to a backend that is not reachable from the internet, use the TestingBot Tunnel. Either run the tunnel yourself and hand the driver its identifier:

driver: new TestingBotDriver({ tunnelIdentifier: 'my-tunnel' })

Or let the driver manage the tunnel around the run. This requires Java and npm install --save-dev testingbot-tunnel-launcher:

driver: new TestingBotDriver({ tunnel: true })                    // anonymous tunnel
driver: new TestingBotDriver({ tunnel: { identifier: 'ci-42' } }) // named tunnel

The driver starts the tunnel before any device is allocated and closes it after the run.

Running in CI/CD

Store your credentials as secrets and run Mobilewright like any other test command. The build name is auto-detected, so no extra configuration is needed.

jobs:
  mobile-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 20, cache: npm }
      - run: npm ci
      - run: npx mobilewright test
        env:
          TESTINGBOT_KEY: ${{ secrets.TESTINGBOT_KEY }}
          TESTINGBOT_SECRET: ${{ secrets.TESTINGBOT_SECRET }}
          # build name auto-detects as "owner/repo #run" - no config needed

The same pattern works on the other CI systems listed under CI/CD integrations.

How Mobilewright maps to TestingBot

Troubleshooting

Current limitations

• One TestingBot session may host several Mobilewright tests unless sessionPerTest: true. Multi-test sessions report the run level verdict.
• Modifier key chords such as pressKeys(['ctrl+a']) work on Android only, because XCUITest cannot hold modifier keys.
• pressButton on iOS supports HOME, VOLUME_UP and VOLUME_DOWN. listApps() reports the foreground app only.
• Screenshots are always PNG. applyDeviceSettings turns Android animations off on a best-effort basis and is a no-op on iOS.
• Real iOS devices need a test-signed .ipa. Simulator builds are rejected on real devices with a clear error.
• While attached to a webview, the session's single Appium context is the web layer.

Frequently asked questions

Next steps

• Driver on GitHub
  Source, issues and releases
• Supported devices
  Real iPhones, iPads and Android devices on the grid
• Real device testing
  How the real-hardware cloud works
• TestingBot Tunnel
  Test against internal or staging APIs
• Appium guide
  The protocol the driver speaks underneath
• Pricing
  Real-device plans and concurrency