Exit course
Chapter 8 of 11 Intermediate 5 min read 2 quick checks Updated

Build for Testing: The App and the Test Runner

On this page
  1. build-for-testing and test-without-building
  2. Simulator builds and device builds
  3. Signing
  4. Package the two files

So far Xcode has built and run the tests in one step. To run the same tests somewhere else (another simulator, a CI machine, real iPhones in a device cloud) you split that step in two: build for testing once, then run without building wherever the tests should execute. This chapter shows what the build produces, how simulator and device builds differ, and how to package the two files a device cloud needs.

build-for-testing and test-without-building#

xcodebuild build-for-testing \
  -project Tasks.xcodeproj -scheme Tasks \
  -destination 'generic/platform=iOS Simulator' \
  -derivedDataPath build-sim

generic/platform=iOS Simulator builds for "any simulator" instead of one particular device, and -derivedDataPath puts the output in a folder you choose instead of somewhere under ~/Library/Developer/Xcode/DerivedData. The products:

build-sim/Build/Products/
  Debug-iphonesimulator/
    Tasks.app
    TasksUITests-Runner.app
  Tasks_iphonesimulator26.4-arm64-x86_64.xctestrun

Three things, and you need all of them:

  • Tasks.app, the app under test.
  • TasksUITests-Runner.app, the test runner. It is a small app generated by Xcode that contains your compiled tests as a plug-in (PlugIns/TasksUITests.xctest). This is the process that drives Tasks.app.
  • The .xctestrun file, a property list that describes which test bundles to run against which app, with which launch arguments and environment. It is what makes running without building possible.

Run the tests from that build, without compiling anything:

xcodebuild test-without-building \
  -xctestrun build-sim/Build/Products/Tasks_iphonesimulator26.4-arm64-x86_64.xctestrun \
  -destination 'platform=iOS Simulator,name=iPhone 17,OS=26.4.1' \
  -only-testing:TasksUITests/LaunchTests
Test Case '-[TasksUITests.LaunchTests testLoginScreenAppears]' passed (3.501 seconds).
** TEST EXECUTE SUCCEEDED **

The split pays off in CI: build once, then run the same products on several simulators in parallel jobs, or archive them and run them later. Chapter 9 uses it.

You changed one assertion in a UI test and ran xcodebuild test-without-building with the existing .xctestrun. The old assertion still fails. Why?

Your Swift tests are compiled into TasksUITests.xctest inside the runner app. "Without building" means exactly that, so any change to the app or the tests needs a new build-for-testing first.

Simulator builds and device builds#

Simulators on a Mac run simulator binaries; iPhones run device binaries. A build for one does not install on the other, so for real devices you build again with a device destination:

xcodebuild build-for-testing \
  -project Tasks.xcodeproj -scheme Tasks \
  -destination 'generic/platform=iOS' \
  -derivedDataPath build-device \
  CODE_SIGNING_ALLOWED=NO
** TEST BUILD SUCCEEDED **

The products are in build-device/Build/Products/Debug-iphoneos/ (note iphoneos, not iphonesimulator): the same Tasks.app and TasksUITests-Runner.app, compiled for the device.

Signing#

iOS only installs signed apps on a device. Which signature you need depends on where the tests will run:

  • Your own iPhone, from Xcode or xcodebuild test: sign with your development team (Signing & Capabilities in Xcode, or DEVELOPMENT_TEAM=... plus -allowProvisioningUpdates on the command line). The device must be registered in that team's provisioning profile.
  • A device cloud: the devices belong to the provider, not to your team, so a development profile from your account does not list them. TestingBot re-signs the uploaded app and test runner for its own devices. The course's build above used CODE_SIGNING_ALLOWED=NO, which produces unsigned bundles (codesign -dv reports "code object is not signed at all"), and that upload installed and ran on TestingBot's iPhones in chapter 8. If your pipeline already produces a development- or ad hoc-signed build, uploading that one is the conventional route the XCUITest documentation describes.

Turning signing off is also handy in CI: the build machine does not need your certificates at all.

Package the two files#

A device cloud wants the app as an .ipa and the runner as a .zip. An .ipa is a zip file with the app inside a folder called Payload:

cd build-device/Build/Products/Debug-iphoneos
mkdir Payload
cp -R Tasks.app Payload/
zip -qry ../../../../Tasks.ipa Payload
rm -rf Payload
zip -qry --symlinks ../../../../TasksUITests.zip TasksUITests-Runner.app

--symlinks (as in the TestingBot documentation) stores any symbolic links inside the bundle as links instead of copying what they point to. The result for this course: Tasks.ipa at 91 KB and TasksUITests.zip at 5.9 MB (most of that is the XCTest frameworks the runner carries). Check the top of each archive before uploading:

$ unzip -l Tasks.ipa
  Payload/
  Payload/Tasks.app/
  ...
$ unzip -l TasksUITests.zip
  TasksUITests-Runner.app/
  TasksUITests-Runner.app/PlugIns/
  ...

Common mistakes, each of which makes the upload fail to install:

  • Zipping Debug-iphonesimulator products for a real device.
  • Zipping the folder that contains the runner instead of the runner itself, so the archive starts with Debug-iphoneos/.
  • Uploading TasksUITests.xctest on its own. The cloud needs the whole -Runner.app that hosts it.
  • Building the Release configuration when the app's test hooks are inside #if DEBUG (chapter 6).

The Build an XCUITest suite page shows the same steps from the Xcode menus (Product > Build For > Testing, then Show Build Folder in Finder) if you prefer clicking.

Which pair of files does a real-device XCUITest run in the cloud need?

The cloud installs compiled bundles; it does not build source. They must be device builds, and the tests travel inside the runner app, not as a bare .xctest. An .xcresult holds results, not something to run.

Next: uploading those two files and running them on real iPhones.