Exit course
Chapter 9 of 11 Intermediate 7 min read 2 quick checks Updated

Run XCUITest on Real iPhones with TestingBot

On this page
  1. Install the CLI and pick devices
  2. Run with the CLI
  3. Run with the REST API
  4. In the dashboard
  5. Simulator, real device, or both?

A simulator runs on your Mac's CPU, with your Mac's network and none of a phone's sensors, memory limits or system dialogs. Before a release you want the same tests on real iPhones and on the iOS versions your users have. This chapter takes the two files from chapter 7, Tasks.ipa and TasksUITests.zip, and runs them on TestingBot's real devices: first with the CLI, then with the REST API, then in the dashboard.

On TestingBot, XCUITest runs on real devices only. A run executes your whole test bundle on one device, and the results come back per test, with a video of the device and the logs.

Install the CLI and pick devices#

npm install -g @testingbot/cli
testingbot login

testingbot login opens the browser and stores your key and secret in ~/.testingbot. On CI, set TB_KEY and TB_SECRET instead (chapter 9). To see which devices accept XCUITest runs right now:

testingbot devices --framework xcuitest --real
PLATFORM  DEVICE                 AVAILABLE  VERSIONS
--------------------------------------------------------------
iOS       iPhone 14              yes        17.6
iOS       iPhone 16              yes        26.5
iOS       iPhone 16 Pro          yes        26.4
iOS       iPhone 17 Pro Max      yes        26.6
iOS       iPhone Air             yes        26.2
...

That is an excerpt from November 2026; the list and the availability change, so run the command yourself. Pick devices whose iOS version is at or above your app's deployment target (iOS 16 for Tasks).

Run with the CLI#

One command uploads both files, starts the run, streams progress and downloads a JUnit report:

testingbot xcuitest Tasks.ipa TasksUITests.zip \
  --device "iPhone 16 Pro" --real-device \
  --name "Tasks UI tests" \
  --build certificate-xcuitest \
  --report junit --report-output-dir ./reports
Uploading XCUITest App
  Tasks.ipa: done (91.2 KB)
Uploading XCUITest Test App
  TasksUITests.zip: done (5.89 MB)
Running XCUITests
Waiting for test results...
  ✔ Run 3150 (iPhone 16 Pro): Test has finished running
  ✔ Run 3150 (iPhone 16 Pro): Test completed successfully
All tests completed successfully!
Fetching junit report(s)...
  Saved report: reports/xcuitest_report_3111_3150.xml

That run (session 594173d23352-..., iPhone 16 Pro, iOS 26.4) executed all ten tests of the course; from the start of the upload to the saved report took under two minutes. The same suite also passed on an iPhone 16 with iOS 26.5 (run 3156, session 7af91f31cbec-...). If your account's parallel slots are busy, the run waits in a queue first; the CLI keeps waiting and reports when it starts.

The options that matter most:

  • --device takes a device name or a regular expression ("iPhone.*" takes any free iPhone), and --platform-version pins the iOS version.
  • --real-device is required for XCUITest on TestingBot.
  • --build sets the build name, which groups runs in the dashboard. Use one build per CI run or release candidate. This is not the same as --name, which only labels the run.
  • --report junit (or html) with --report-output-dir saves a report when the run is done.
  • --async starts the run and exits without waiting, for pipelines that collect results later.
  • --language, --locale, --timezone, --orientation, --throttle-network and --geo-country-code change the device's conditions for the run. The XCUITest documentation lists them all.

Two lines of the JUnit report for run 3150:

<testsuite name="XcuiRun #3150" tests="10" failures="0" skipped="0" errors="0" time="62">
  <testcase name="testSignInAndAddTwoTasks" result="passed" classname="TasksUITests.ScreenObjectTests.testSignInAndAddTwoTasks" ... />

Each test method becomes a testcase, so CI systems can show which test failed on which device.

A team runs the suite on four iPhones from CI and passes --name "nightly". In the dashboard every night's runs are mixed together with the default build names. What should they add?

The build name groups runs; --name only labels each run. One build per CI run keeps each night's four devices together.

Run with the REST API#

The CLI is a wrapper around three API calls, which you can make from any language or CI system. With your key and secret in TB_KEY and TB_SECRET:

# 1. Upload the app. The answer contains the project id.
curl -u "$TB_KEY:$TB_SECRET" -X POST \
  "https://api.testingbot.com/v1/app-automate/xcuitest/app" \
  -F "file=@Tasks.ipa"
# {"id":3114}

# 2. Upload the test runner to that project.
curl -u "$TB_KEY:$TB_SECRET" -X POST \
  "https://api.testingbot.com/v1/app-automate/xcuitest/3114/tests" \
  -F "file=@TasksUITests.zip"

# 3. Start one run per entry in "capabilities".
curl -u "$TB_KEY:$TB_SECRET" -X POST \
  "https://api.testingbot.com/v1/app-automate/xcuitest/3114/run" \
  -H "Content-Type: application/json" \
  -d '{"capabilities":[{"platformName":"iOS","deviceName":"iPhone Air","version":"26.2",
       "realDevice":true,"name":"Tasks UI tests","build":"certificate-xcuitest"}]}'
# {"success":true,"id":3114,"runs":[{"id":3153,"capabilities":{...}}],"priority":"normal"}

Put several objects in capabilities to run on several devices at once; each becomes its own run with its own id. The build name goes into each capability object as "build": there is no separate build parameter, and a run without it lands in a default build called "XCUITest Project" plus the project id.

Poll a run until its status is DONE:

curl -u "$TB_KEY:$TB_SECRET" \
  "https://api.testingbot.com/v1/app-automate/xcuitest/3114/3153"
{"id":3153,"created_at":"2026-11-13T08:52:06.000Z","status":"DONE",
 "capabilities":{"platformName":"iOS","deviceName":"iPhone Air","version":"26.2","realDevice":true,
                 "name":"Tasks UI tests","build":"certificate-xcuitest"},
 "success":true,
 "test":{"sessionId":"dce673008cbe-d78bf75bfc65-e811be062f49-179455984164-98037203",
         "environment":{"name":"Safari","os":"iPhone Air - 26.2","version":"26.2"}}}

That is the real answer for the course's API run: all ten tests passed on an iPhone Air with iOS 26.2. success is true only when every test in the run passed, and test.sessionId is the session you open in the dashboard.

/report on the project returns the JUnit XML for all its runs. The test reports page documents both report formats.

In the dashboard#

Every run appears in the TestingBot dashboard as a session under its build. For each one you get:

  • The video of the device for the whole run. For a failed test, jump to the time of the failure and watch what the screen showed; it answers "was a sheet in the way?" faster than any log.
  • The per-test results, with each test method's name, outcome and duration.
  • Logs: the XCUITest output (the same lines xcodebuild prints locally) and the device log.

The Save Password sheet from chapter 4 is a good example of why the video matters: a system sheet that appears on its own schedule is easy to miss in a log and obvious in a video. Tests that handle it explicitly pass on both.

The REST API call to /run sends three capability objects but only the first has "build": "release-4.2". Where do the other two runs end up?

Each capability object becomes an independent run, and each run takes its build name from its own object. Repeat the build value in every entry.

Simulator, real device, or both?#

Run the full suite on simulators on every pull request, because it is fast and free of device queues. Run it on a few real devices when it matters: before merging to the main branch, nightly, before a release. Real devices catch what simulators cannot: real performance and memory pressure, device-specific system UI, and hardware such as the camera. The Mobile App Testing course discusses how to choose a device matrix; chapter 9 wires both levels into CI.