Filter XCUITest Tests
By default, TestingBot runs every test in your XCUITest bundle. Pass a filter with the test run to run only part of the suite, for example:
- Re-run only the tests that cover a bugfix.
- Run a quick smoke subset instead of waiting for the full suite.
- Leave out tests that do not apply to a specific environment.
Two filters are available:
- onlyTesting: run only the test classes or test methods you list
- skipTesting: run everything except the test classes or test methods you list
Test identifiers
Both filters take a list of test identifiers, as a JSON array or as one comma-separated string. Each identifier is one of:
-
TestClass: every test in that class, for exampleLoginTests -
TestClass/testMethod: a single test, for exampleLoginTests/testValidPassword
Use the class name without the module name, the way it is written in your source. If you copy identifiers from an xcodebuild -only-testing command in the TestTarget/TestClass/testMethod form, that works too: the test target part is dropped, because a test run uses a single test bundle.
An identifier that is not in one of these forms makes the API answer with a 400 error, so a typo does not silently run your whole suite.
onlyTesting Filter
Use onlyTesting to run only the listed test classes or test methods.
testingbot xcuitest app.ipa app-test.zip \
--device "iPhone.*" \
--real-device \
--only-testing "LoginTests,CartTests/testCheckout"
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/xcuitest/:id/run" \
-d '{ "options": { "onlyTesting": ["LoginTests", "CartTests/testCheckout"] }, "capabilities":[{"deviceName":"iPhone.*", "platformName":"iOS", "realDevice": true}]}' \
-H "Content-Type: application/json"
This runs every test in LoginTests plus the single testCheckout test of CartTests.
skipTesting Filter
Use skipTesting to run the whole suite except the listed test classes or test methods. You can combine it with onlyTesting: for example, run one class but leave out one slow test in it.
testingbot xcuitest app.ipa app-test.zip \
--device "iPhone.*" \
--real-device \
--only-testing "LoginTests" \
--skip-testing "LoginTests/testSlowSignup"
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/xcuitest/:id/run" \
-d '{ "options": { "onlyTesting": ["LoginTests"], "skipTesting": ["LoginTests/testSlowSignup"] }, "capabilities":[{"deviceName":"iPhone.*", "platformName":"iOS", "realDevice": true}]}' \
-H "Content-Type: application/json"
Split a suite over parallel runs
To finish a large suite sooner, split it into groups of test classes and start one test run per group, each with its own onlyTesting list. The runs use separate devices and run in parallel, up to the number of parallel real-device tests your account allows.
testingbot xcuitest app.ipa app-test.zip --device "iPhone.*" --real-device \
--only-testing "LoginTests,ProfileTests" --async
testingbot xcuitest app.ipa app-test.zip --device "iPhone.*" --real-device \
--only-testing "CartTests,CheckoutTests" --async
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/xcuitest/:id/run" \
-d '{ "options": { "onlyTesting": ["LoginTests", "ProfileTests"] }, "capabilities":[{"deviceName":"iPhone.*", "platformName":"iOS", "realDevice": true}]}' \
-H "Content-Type: application/json"
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/xcuitest/:id/run" \
-d '{ "options": { "onlyTesting": ["CartTests", "CheckoutTests"] }, "capabilities":[{"deviceName":"iPhone.*", "platformName":"iOS", "realDevice": true}]}' \
-H "Content-Type: application/json"