URL: https://testingbot.com/
# Cross Browser Testing on 6100+ Real Browsers & Devices
Run automated, manual, visual and AI-assisted tests on 6100+ real browsers and physical iOS and Android devices, with parallel execution and CI/CD integrations.
Trusted by some of the world's most innovative companies
[Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo)
No credit card required
1+bn
Tests Executed
180k
Active Users
6100+
Browsers & Devices
About TestingBot
## All-in-one cross-browser, mobile and AI testing platform
TestingBot runs your Selenium, Appium, Playwright, Cypress, Puppeteer, Espresso, XCUITest and Maestro tests on 6100+ real browsers and physical iOS and Android devices.
Automated, manual, visual and AI-assisted testing with parallel execution and CI/CD integrations.
### Web Testing
Test on 6100+ real browsers across Windows, macOS and Linux.
#### [Live / Manual](https://testingbot.com/features/manual-browser-testing)
Interactive browser testing
#### [Automated](https://testingbot.com/features)
Selenium, Playwright, Puppeteer & Cypress
#### [AI Testing](https://testingbot.com/features/ai-testing)
AI-driven testing & MCP integration
#### [Visual Testing](https://testingbot.com/features/screenshot-testing)
Screenshot comparison of websites
### Mobile App Testing
Test on real physical iOS and Android devices.
#### [Live / Manual](https://testingbot.com/features/manual-mobile-testing)
Interactive device testing
#### [Automated](https://testingbot.com/real-device-testing)
Appium, Espresso, XCUITest & Maestro
#### [AI Testing](https://testingbot.com/features/ai-testing)
AI-driven testing on real devices
#### [Visual Testing](https://testingbot.com/features/screenshot-testing)
App screenshot comparison
## Cross-Browser Testing Made Simple
Get one-click access to real browsers running on macOS, Windows and Linux platforms. Switch platforms, locations and screen resolutions instantly.
Test across all versions of Internet Explorer (IE8 to IE11), Edge, Chrome, Safari, Firefox and Opera.
 Chrome
 Safari
 Firefox
 Edge
[Start Testing](https://testingbot.com/features/manual-browser-testing)
## Why teams choose TestingBot
### Improve test coverage
Cross-browser, real device and visual testing on 6100+ browser and device combinations.
### Boost productivity with AI
AI-powered test creation and execution. Write tests in natural language, let AI do the rest.
### Accelerate testing cycles
Run tests in parallel across multiple browsers and devices. Ship faster with confidence.
## Works with your favorite testing frameworks
Native support for all major web and mobile testing frameworks. Get started in minutes.
### Web Testing
[Selenium](https://testingbot.com/features/automation/selenium) [Playwright](https://testingbot.com/features/automation/playwright) [Cypress](https://testingbot.com/features/automation/cypress) [Puppeteer](https://testingbot.com/features/automation/puppeteer) [WebdriverIO](https://testingbot.com/support/web-automate/selenium/nodejs/webdriverio) [RobotFramework](https://testingbot.com/support/web-automate/selenium/robotframework)
### Mobile Testing
[Appium](https://testingbot.com/features/automation/appium) [Espresso](https://testingbot.com/features/automation/espresso) [XCUITest](https://testingbot.com/features/automation/xcuitest) [Maestro](https://testingbot.com/features/automation/maestro) [Flutter](https://testingbot.com/support/app-automate/appium/flutter) [React Native](https://testingbot.com/features/automation/appium)
### Languages & SDKs
[Java](https://testingbot.com/support/web-automate/selenium/java) [Python](https://testingbot.com/support/web-automate/selenium/python) [NodeJS](https://testingbot.com/support/web-automate/selenium/nodejs) [C# / .NET](https://testingbot.com/support/web-automate/selenium/csharp) [Ruby](https://testingbot.com/support/web-automate/selenium/ruby) [PHP](https://testingbot.com/support/web-automate/selenium/php)
[View all documentation](https://testingbot.com/support)
Real Device Cloud
## Test on real iOS & Android devices
Access hundreds of real physical devices for manual and automated testing. iPhones, iPads, Samsung Galaxy, Pixel, Smart TVs and more.
[Appium](https://testingbot.com/features/automation/appium) [Espresso](https://testingbot.com/features/automation/espresso) [XCUITest](https://testingbot.com/features/automation/xcuitest) [Maestro](https://testingbot.com/features/automation/maestro)
[Explore device cloud](https://testingbot.com/real-device-testing)
AI-Powered Testing
## AI Test Agent
Create and run tests without writing code. Simply describe what you want to test in plain language.
TestingBot AI understands your application and generates reliable test steps automatically.
[Try AI Testing](https://testingbot.com/features/ai-testing)
Accessibility Testing
## WCAG Compliance Testing
Identify and fix accessibility issues across your websites. Ensure WCAG compliance and deliver an inclusive experience for all users.
Meet legal requirements like the European Accessibility Act (EAA), reach a broader audience and improve SEO with accessibility best practices.
WCAG 2.1 Compliance
Scheduled Testing
Detailed Reports
Severity Classification
[Explore Accessibility Testing](https://testingbot.com/features/accessibility-testing)
## Trusted by teams worldwide
Join thousands of companies shipping better software with TestingBot
1B+
Tests Executed
180,000+
Active Users
6,100+
Browsers & Devices
106
Countries
GDPR
Compliant
SAML SSO
Single Sign-On
Star Level 1
Certified
99.9%
Uptime SLA
## Seamlessly integrate into your workflow
Connect TestingBot with your CI/CD pipeline, issue trackers and communication tools.
[Jenkins](https://testingbot.com/support/integrations/ci-cd/jenkins) [GitHub](https://testingbot.com/support/integrations/ci-cd/github-actions) [GitLab](https://testingbot.com/support/integrations/ci-cd/gitlab) [CircleCI](https://testingbot.com/support/integrations/ci-cd/circleci) [Travis CI](https://testingbot.com/support/integrations/ci-cd/travis-ci) [Slack](https://testingbot.com/support/integrations/slack) [Jira](https://testingbot.com/support/integrations/jira) [Azure](https://testingbot.com/support/integrations/ci-cd/azure) [Bitbucket](https://testingbot.com/support/integrations/ci-cd/bitbucket) [TeamCity](https://testingbot.com/support/integrations/ci-cd/teamcity) [Bamboo](https://testingbot.com/support/integrations/ci-cd/bamboo) [App Center](https://testingbot.com/support/integrations/appcenter)
[View all integrations](https://testingbot.com/integrations)
FAQ
## Frequently Asked Questions
What is cross-browser testing?
Cross-browser testing verifies that a website or web application works correctly across different browsers (Chrome, Safari, Firefox, Edge, Internet Explorer), browser versions, operating systems and screen sizes. It catches rendering inconsistencies, JavaScript compatibility issues and CSS bugs before users encounter them. TestingBot runs your tests on 6100+ real browser and OS combinations in parallel.
How do I test a website on multiple browsers?
Use a cross-browser testing platform like TestingBot. Sign up, point your Selenium, Playwright, Cypress or Puppeteer test scripts at the TestingBot grid by changing the WebDriver URL, and your tests run in parallel across thousands of real browsers and devices. You can also start live, interactive sessions on any browser without writing code.
Which test automation frameworks does TestingBot support?
TestingBot natively supports Selenium, Appium, Playwright, Cypress, Puppeteer, Espresso, XCUITest and Maestro. SDKs are available for Java, JavaScript, TypeScript, Python, Ruby, C#, PHP and Go. AI assistants like Claude Code and Cursor can also access the TestingBot REST API via a [Model Context Protocol (MCP) server](https://testingbot.com/support/ai/mcp) to inspect test runs and debug failures from your editor.
Does TestingBot support real iOS and Android devices?
Yes. TestingBot operates a cloud of physical iOS and Android devices, including iPhones (from iPhone 6s to iPhone 17 Pro), iPads, Samsung Galaxy phones, Pixel devices and tablets. You can run automated Appium, Espresso, XCUITest or Maestro tests on real hardware, or take manual control via the [live testing interface](https://testingbot.com/features/manual-mobile-testing).
How is TestingBot different from BrowserStack, Sauce Labs and LambdaTest?
TestingBot is more affordable than BrowserStack and Sauce Labs (typically 30 to 40 percent cheaper), includes real device testing in every paid plan, hosts data in the European Union for GDPR compliance, and ships features faster, Apple Silicon support arrived more than ten months before Sauce Labs. See the dedicated comparisons: [BrowserStack alternative](https://testingbot.com/browserstack-alternative), [Sauce Labs alternative](https://testingbot.com/saucelabs-alternative), [LambdaTest alternative](https://testingbot.com/lambdatest-alternative).
Does TestingBot support AI-driven testing?
Yes. TestingBot offers [codeless AI testing](https://testingbot.com/support/ai/codeless), describe in plain language what you want to test, and an AI agent creates and runs the test on the browsers you choose, reporting back any failures. TestingBot also exposes its REST API via a [Model Context Protocol (MCP) server](https://testingbot.com/support/ai/mcp) so AI assistants like Claude Code and Cursor can inspect test runs, fetch logs and debug failures from your editor. See the full [AI testing overview](https://testingbot.com/support/ai) for integrations with Magnitude, Stagehand and Notte.
Where is TestingBot data hosted?
TestingBot is built and hosted in Europe. All test data, video recordings and logs are stored on European infrastructure (EU and Switzerland), making TestingBot a strong choice for GDPR-compliant testing programs and for teams with European data-residency requirements.
Is there a free trial?
Yes. TestingBot offers a free plan and a free trial of paid features. The free plan needs no credit card: it runs automated tests, live manual sessions, screenshot tests and visual comparisons, including Safari on macOS, on a recent version of each browser with a limited number of live macOS sessions. The 7-day full-access trial asks for a card, charges nothing during the trial, and unlocks every browser version, every macOS release and real iOS and Android devices.
No credit card required
## Ready to get started?
Join thousands of teams shipping better software with TestingBot
[Start Free Trial](https://testingbot.com/users/sign_up) [View pricing](https://testingbot.com/pricing)
Free 14-day trial
5-minute setup
24/7 support
---
URL: https://testingbot.com/features
# All-in-One Cross Browser & Mobile Testing Platform
TestingBot is a cloud testing platform that runs your Selenium, Appium, Playwright, Cypress, Puppeteer, Espresso, XCUITest and Maestro tests on 6100+ real desktop browsers and physical iOS and Android devices. Test in parallel, debug with video and screenshot recording and integrate with any CI/CD system.
[Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo)
- **Browsers & devices:** 6100+
- **Uptime SLA:** 99.9%
## Trusted by some of the world's most innovative companies
## All-in-one cross browser testing platform
The TestingBot cloud brings you the power to test your website and mobile app on [+6100 browsers & devices](https://testingbot.com/support/web-automate/browsers).
Test on the latest Chrome, Firefox, Edge and Safari, plus legacy versions like Internet Explorer 11, all running on real Windows, macOS and Linux machines.
Let us handle your testing infrastructure while you focus on development.
Run tests effortlessly on our maintained cloud of desktop and mobile platforms.
[Start free](https://testingbot.com/users/sign_up)
Built for scale
## 6100+ real desktop and mobile browser combinations
Test on the widest range of desktop and mobile devices in our real device cloud. We handle setup, procurement and maintenance, so you can focus entirely on testing.
### Windows, macOS & Linux
One-click access to every macOS, Linux and Windows version. From Yosemite to macOS Tahoe, and Windows 7 to Windows 11. We've got you covered.
### Every browser, all versions
Edge, Safari, Firefox, Chrome, Opera and IE, constantly updated and instantly available.
### Security
Single-Use VMs and real devices, reset after each use.
### Real Android and iOS devices
Test across real iPhone, iPads, Google, OnePlus, Xiaomi and Samsung phones and tabs.
### No setup and maintenance
Instantly access 6100+ devices and browsers. Experience zero setup or maintenance hassles, and focus only on testing.
### Performance
Our infrastructure is built for performance to eliminate test flakiness and latency.
Automation frameworks
## Test Automation Frameworks
TestingBot supports these popular Test Automation Frameworks:
- [](https://testingbot.com/features/automation/selenium)
### [Selenium](https://testingbot.com/features/automation/selenium)
Popular Browser Automation Framework, using WebDriver, which supports all major browsers.
Best for: enterprise-grade browser automation across every major language.
[Try with Selenium](https://testingbot.com/features/automation/selenium)
- [](https://testingbot.com/features/automation/appium)
### [Appium](https://testingbot.com/features/automation/appium)
Mobile Automation Framework, using WebDriver, runs on physical devices and simulators/emulators.
Best for: native mobile app automation on real iOS & Android devices.
[Try with Appium](https://testingbot.com/features/automation/appium)
- [](https://testingbot.com/features/automation/cypress)
### [Cypress](https://testingbot.com/features/automation/cypress)
Javascript based E2E testing on Chrome and Firefox.
Best for: fast JavaScript-only end-to-end testing.
[Try with Cypress](https://testingbot.com/features/automation/cypress)
- [](https://testingbot.com/features/automation/puppeteer)
### [Puppeteer](https://testingbot.com/features/automation/puppeteer)
Chrome DevTools based Automation Framework by Google. Supports Chrome and Edge browsers.
Best for: Chromium DevTools-based automation and scraping.
[Try with Puppeteer](https://testingbot.com/features/automation/puppeteer)
- [](https://testingbot.com/features/automation/playwright)
### [Playwright](https://testingbot.com/features/automation/playwright)
Similar to Puppeteer, built by Microsoft, with support for Chrome and Edge.
Best for: modern multi-browser testing in TypeScript, Python and .NET.
[Try with Playwright](https://testingbot.com/features/automation/playwright)
- [](https://testingbot.com/features/automation/espresso)
### [Espresso](https://testingbot.com/features/automation/espresso)
Android Automation Framework, runs on physical devices and emulators.
Best for: Android-only UI testing inside the app process.
[Try with Espresso](https://testingbot.com/features/automation/espresso)
- [](https://testingbot.com/features/automation/xcuitest)
### [XCUITest](https://testingbot.com/features/automation/xcuitest)
iOS Automation Framework, runs on physical devices and simulators.
Best for: iOS-only UI testing with Apple's native framework.
[Try with XCUITest](https://testingbot.com/features/automation/xcuitest)
- [](https://testingbot.com/features/automation/maestro)
### [Maestro](https://testingbot.com/features/automation/maestro)
Maestro is a mobile UI testing framework, which supports testing on both iOS and Android.
Best for: declarative mobile flows without writing code.
[Try with Maestro](https://testingbot.com/features/automation/maestro)
Comparing frameworks? See [Playwright vs Selenium](https://testingbot.com/playwright-vs-selenium) for a head-to-head breakdown.
Whichever you use, [running tests in parallel](https://testingbot.com/features/parallel-testing) is what cuts the wall-clock time.
Buy vs build
## TestingBot vs a DIY in-house grid
Building and maintaining your own browser and device lab is expensive and slow. Here is how a hosted cloud compares to running the infrastructure yourself.
| Capability | TestingBot | DIY in-house grid |
| --- | --- | --- |
| Browser & device coverage | 6100+ ready to use | You buy and maintain |
| Setup time | Minutes | Weeks to months |
| Maintenance burden | None, managed for you | Constant patching, OS upgrades and capacity planning |
| Real iOS devices | iPhone, iPad, real hardware | Hard (Apple licensing & MDM) |
| Parallel sessions | Scale to hundreds in parallel | Capped by your hardware |
| EU data residency | Hosted in the EU | Build it yourself |
| Video & screenshot recording | Recorded automatically on every test | Build it yourself |
| CI/CD integrations | Jenkins, GitHub Actions, CircleCI, GitLab, Azure DevOps | DIY scripts and plugins |
| Live debugging | Built-in | Custom VNC tooling |
| Support | 24/7 included | Internal team only |
Live mobile testing
## Take control of a device
Interact with the remote mobile device, just as you would with a device in hand. Tap, scroll, zoom, swipe and more.
Open a real device anywhere in the world and drive it from your browser. Tap, scroll, swipe, install your APK or IPA, capture network logs and reproduce a customer issue in real time.
[Try Live Mobile Testing](https://testingbot.com/features/manual-mobile-testing)
Developer experience
## Test in your favorite language
Use the SDK of your choice. TestingBot integrates with Jest, Mocha, PyTest, NUnit, RSpec and other popular test frameworks, plus every major CI/CD system.
hub.testingbot.com Copy
JavaScript Python Java Ruby C# PHP
```javascript
// Run a Selenium test on the TestingBot grid
const wd = require('selenium-webdriver');
const driver = await new wd.Builder()
.usingServer('https://hub.testingbot.com/wd/hub')
.withCapabilities({
'tb:options': { name: 'Checkout flow' },
browserName: 'chrome',
browserVersion: 'latest',
platformName: 'Windows 11'
})
.build();
await driver.get('https://example.com');
await driver.quit();
```
```python
# Run a Selenium test on the TestingBot grid
from selenium import webdriver
options = webdriver.ChromeOptions()
options.set_capability('tb:options', {'name': 'Checkout flow'})
options.set_capability('browserVersion', 'latest')
options.set_capability('platformName', 'Windows 11')
driver = webdriver.Remote(
command_executor='https://hub.testingbot.com/wd/hub',
options=options
)
driver.get('https://example.com')
driver.quit()
```
```java
// Run a Selenium test on the TestingBot grid
DesiredCapabilities caps = new DesiredCapabilities();
caps.setCapability("browserName", "chrome");
caps.setCapability("browserVersion", "latest");
caps.setCapability("platformName", "Windows 11");
WebDriver driver = new RemoteWebDriver(
new URL("https://hub.testingbot.com/wd/hub"),
caps
);
driver.get("https://example.com");
driver.quit();
```
```ruby
# Run a Selenium test on the TestingBot grid
require 'selenium-webdriver'
caps = Selenium::WebDriver::Remote::Capabilities.chrome(
browserVersion: 'latest',
platformName: 'Windows 11',
'tb:options' => { name: 'Checkout flow' }
)
driver = Selenium::WebDriver.for(:remote,
url: 'https://hub.testingbot.com/wd/hub',
capabilities: caps
)
driver.get 'https://example.com'
driver.quit
```
```csharp
// Run a Selenium test on the TestingBot grid
var options = new ChromeOptions();
options.AddAdditionalOption("browserVersion", "latest");
options.AddAdditionalOption("platformName", "Windows 11");
IWebDriver driver = new RemoteWebDriver(
new Uri("https://hub.testingbot.com/wd/hub"),
options
);
driver.Navigate().GoToUrl("https://example.com");
driver.Quit();
```
```php
// Run a Selenium test on the TestingBot grid
$capabilities = DesiredCapabilities::chrome();
$capabilities->setCapability('browserVersion', 'latest');
$capabilities->setCapability('platformName', 'Windows 11');
$driver = RemoteWebDriver::create(
'https://hub.testingbot.com/wd/hub',
$capabilities
);
$driver->get('https://example.com');
$driver->quit();
```
Connected · [View documentation](https://testingbot.com/support/web-automate/selenium/nodejs) [View documentation](https://testingbot.com/support/web-automate/selenium/python) [View documentation](https://testingbot.com/support/web-automate/selenium/java) [View documentation](https://testingbot.com/support/web-automate/selenium/ruby) [View documentation](https://testingbot.com/support/web-automate/selenium/csharp) [View documentation](https://testingbot.com/support/web-automate/selenium/php)
## Test on real mobile and desktop devices
TestingBot only runs real browsers on real Windows, macOS and Linux machines, never simulators. Tests on TestingBot match what your customers actually see, so your results are always accurate.
Test on the latest Chrome, Firefox, Edge and Safari, plus legacy versions like Internet Explorer 11, all running on real Windows, macOS and Linux machines.
Test across all versions of Internet Explorer (IE8 to IE11), Edge, Chrome, Safari, Firefox and Opera.
[Start free](https://testingbot.com/users/sign_up)
Secure tunnel
## Test Staged Websites
Use the secure TestingBot Tunnel to easily test your staging and development environments across all browsers, ensuring seamless access behind firewalls or private networks. Test data and websites from behind your firewall on the TestingBot cloud.
[Read more](https://testingbot.com/support/tunnel)
FAQ
## Frequently asked questions about TestingBot features
What testing frameworks does TestingBot support?
TestingBot natively supports **Selenium, Appium, Playwright, Cypress, Puppeteer, Espresso, XCUITest and Maestro** , so you can keep your existing test framework and point it at the TestingBot cloud by changing a single endpoint URL. Official SDKs ship for Java, JavaScript, TypeScript, Python, Ruby, C#, PHP and Go, and each returns the same session metadata, video recordings and logs regardless of language. Selenium, Playwright, Cypress and Puppeteer handle desktop browser automation, while Appium, Espresso, XCUITest and Maestro drive real iOS and Android devices. Beyond writing code, TestingBot offers codeless AI test creation, where you describe a scenario in plain language and an agent builds and runs it for you. AI assistants like Claude Code and Cursor can also drive TestingBot through a [Model Context Protocol (MCP) server](https://testingbot.com/support/ai/mcp), inspecting runs, fetching logs and debugging failures without leaving your editor.
Can I run automated tests in parallel?
Yes. Run automated Selenium, Appium, Playwright, Cypress, Puppeteer, Espresso, XCUITest and Maestro tests in parallel, from a handful of concurrent sessions on the free plan up to hundreds of parallel sessions on Enterprise. Parallel execution typically reduces total test runtime by 10× to 50× compared to a single-machine grid, because your suite is split across many real browsers and devices that all run at the same time. You control concurrency from your test configuration or CI pipeline by setting the number of parallel sessions, and TestingBot queues and distributes the tests for you. Each session is isolated on its own single-tenant virtual machine or freshly reset device, so parallel runs never share state or interfere with one another. Most teams use this to keep a growing regression suite inside the same CI time budget, turning an overnight run into one that finishes in minutes.
Does TestingBot support real iOS and Android devices?
Yes. TestingBot operates a cloud of physical iOS and Android devices, including iPhones (from iPhone 6s to the latest iPhone), iPads, Samsung Galaxy phones, Pixel devices and tablets, all real hardware rather than emulators or simulators. You can run automated Appium, Espresso, XCUITest or Maestro tests on these devices, or take manual control through the [live testing interface](https://testingbot.com/features/manual-mobile-testing) to tap, swipe, type and inspect elements in real time. Every device is wiped and reset to a clean state after each session, so no test data, credentials or installed apps carry over between runs. New iOS and Android releases are added quickly, real iOS 26 device support arrived the same week Apple released the OS. Because the devices are hosted in Europe, your mobile test data, video recordings and logs stay on European infrastructure (EU and Switzerland), which helps teams with GDPR or data-residency requirements.
How do I test internal or staging applications with TestingBot?
Use the [TestingBot Tunnel](https://testingbot.com/support/tunnel) to expose a staging or internal application to the TestingBot cloud over a secure connection. The tunnel is encrypted end-to-end, requires no firewall changes, and lets every browser and device in the cloud reach hostnames behind your VPN.
What CI/CD integrations does TestingBot offer?
TestingBot ships first-party plugins and integrations for Jenkins, GitHub Actions, GitLab CI, CircleCI, Azure DevOps, Bamboo, TeamCity and Travis CI. The platform also exposes a REST API and webhook system, so any CI/CD tool can launch tests, fetch results and react to failures.
Can I do manual or live browser testing on TestingBot?
Yes. [Live browser testing](https://testingbot.com/features/manual-browser-testing) and [live mobile testing](https://testingbot.com/features/manual-mobile-testing) let you remote-control any real browser, device, or simulator from inside your own browser: type, click, scroll, pinch and inspect. There is no SDK to install, and sessions launch in seconds.
Is TestingBot GDPR-compliant and secure?
TestingBot is built and hosted in Europe, GDPR-compliant by design and CSA STAR Level 1 registered. The infrastructure is single-tenant per test session: virtual machines and devices are wiped after every test.
### Sign up for a Free Trial
Start testing your apps with TestingBot.
[Start a free trial](https://testingbot.com/users/sign_up)
Working on an open source project? [Get these features free for open source](https://testingbot.com/open-source).
---
URL: https://testingbot.com/real-device-testing
Real Device Cloud · iOS & Android
# Real device testing online, on iOS and Android
Real device testing means running your automated and manual tests on physical phones and tablets rather than on emulators. TestingBot provides a cloud of real iOS and Android devices, hosted in a secure EU datacenter, ready to run your tests in seconds, alongside hosted emulators and simulators for the faster checks you want on every commit.
- Real iOS & Android devices
- European-hosted, GDPR-compliant
- [Appium](https://testingbot.com/features/automation/appium), [XCUITest](https://testingbot.com/features/automation/xcuitest), [Espresso](https://testingbot.com/features/automation/espresso), [Maestro](https://testingbot.com/features/automation/maestro), [Mobilewright](https://testingbot.com/support/app-automate/mobilewright)
- Pristine, never jailbroken
[Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo)
Trusted by some of the world's most innovative companies
Built for scale
## Reliable Device Farm
Get instant access to a fast, reliable device infrastructure.
### 24/7 device availability
Physical iOS and Android devices, available at any time of the day.
### Zero setup & Maintenance
Focus on testing instead of worrying about setup and maintenance of a mobile device farm.
### Security
Devices located in a datacenter in Europe. Each test runs on a secure and pristine device.
### Real Android and iOS devices
Test across real iPhone, iPads, Google, OnePlus, Xiaomi and Samsung phones & tabs.
### Variety of devices
Test on any device, flagship or budget, with support for the newest and legacy iOS and Android versions.
### Performance
Our infrastructure is built for performance to eliminate test flakiness and latency.
The actual fleet
## Every real device you can test on
This is the full list, rendered from our live inventory rather than from a marketing number. Every entry is a physical phone or tablet racked in our European datacenter, wiped and re-imaged between sessions.
42
Physical devices
22
iPhone & iPad
20
Android phones & tablets
iPhone 13 Pro
iOS 27.0
Pixel 8
Android 17.0
Pixel 11
Android 17.0
iPhone 17 Pro
iOS 26.5
iPhone 17 Pro Max
iOS 26.6
iPhone 16 Pro
iOS 26.4
iPad (9th generation)
iOS 26.6
Galaxy S26
Android 16.0
Huawei P40
Android 12.0
iPhone Air
iOS 26.2
Pixel 10
Android 16.0
iPhone 17
iOS 26.0
Pixel 9
Android 16.0
Galaxy S25
Android 15.0
iPhone 16
iOS 26.5
Redmi Note 13
Android 14.0
iPhone SE 2022
iOS 17.5
Galaxy A55
Android 14.0
Galaxy S24
Android 14.0
Pixel 8
Android 14.0
iPhone 15
iOS 18.0
iPad (8th generation)
iOS 14.7
iPhone 11
iOS 14.8
iPhone 14
iOS 17.6
Galaxy S23
Android 13.0
iPhone 13
iOS 18.5
iPhone XR
iOS 14.8
iPad (8th generation)
iOS 15.0
Galaxy A12
Android 11.0
Mi 10
Android 12.0
Galaxy S21
Android 11.0
iPhone 12
iOS 14.5
Galaxy S20
Android 10.0
iPhone 11
iOS 14.0
iPhone 8
iOS 16.7
Galaxy Tab S11
Android 16.0
Galaxy S10
Android 10.0
Pixel
Android 7.1
iPhone XR
iOS 18.7
iPad (6th generation)
iOS 13.5
iPhone 6s
iOS 15.8
Galaxy S8
Android 9.0
Need a model that is not listed? A [private device cloud](https://testingbot.com/enterprise/private-device-cloud) lets us buy and rack specific hardware for your account alone. For quick checks on hardware nobody keeps in a rack, we also run hosted [emulators and simulators](https://testingbot.com/features/manual-mobile-testing).
Automation frameworks
## Test Automation Frameworks
Works instantly with the most popular mobile automation frameworks:
- [](https://testingbot.com/features/automation/appium)
### [Appium](https://testingbot.com/features/automation/appium)
Mobile Automation Framework, using WebDriver, runs on physical devices and simulators/emulators.
[Try with Appium](https://testingbot.com/features/automation/appium)
- [](https://testingbot.com/features/automation/espresso)
### [Espresso](https://testingbot.com/features/automation/espresso)
Android Automation Framework, runs on physical devices and emulators.
[Try with Espresso](https://testingbot.com/features/automation/espresso)
- [](https://testingbot.com/features/automation/xcuitest)
### [XCUITest](https://testingbot.com/features/automation/xcuitest)
iOS Automation Framework, runs on physical devices and simulators.
[Try with XCUITest](https://testingbot.com/features/automation/xcuitest)
- [](https://testingbot.com/features/automation/maestro)
### [Maestro](https://testingbot.com/features/automation/maestro)
Maestro is a mobile UI testing framework, which supports testing on both iOS and Android.
[Try with Maestro](https://testingbot.com/features/automation/maestro)
- [](https://testingbot.com/support/app-automate/mobilewright)
### [Mobilewright](https://testingbot.com/support/app-automate/mobilewright)
Mobilewright runs Playwright-style mobile tests in TypeScript on both iOS and Android.
[Try with Mobilewright](https://testingbot.com/support/app-automate/mobilewright)
Speed up your suite
## Cut suite time from hours to minutes
Drastically shorten your total app test duration by running tests simultaneously. TestingBot provides a variety of devices, ready to run your tests in parallel.
[Start free](https://testingbot.com/users/sign_up)
Debug faster
## Video, screenshots & logs
Every mobile test comes with access to generated logs, videos and screenshots.
[Start free](https://testingbot.com/users/sign_up)
Upload & test in seconds
## TestingBot Storage
Upload your native mobile app, as an .apk or .ipa, to [TestingBot Storage](https://testingbot.com/support/api#upload) and start testing immediately.
[Start free](https://testingbot.com/users/sign_up)
An honest comparison
## Real devices vs emulators: when each one is the right call
Real device testing is not automatically the correct answer. Emulators and simulators are faster to start, cheaper to run in bulk and perfectly adequate for a large share of everyday checks. We sell both, so here is the split we would actually recommend.
### An emulator or simulator is enough for
- 1 Layout, navigation and functional regression on every commit, where you care that the flow works rather than how the hardware behaves.
- 2 Broad OS-version sweeps, because spinning up twenty emulator images costs a fraction of occupying twenty physical handsets.
- 3 Early development feedback, where a session that starts in seconds matters more than hardware fidelity.
- 4 Screen-size and resolution checks, which an emulator reproduces faithfully.
### You need real hardware for
- 1 Performance and responsiveness, because an emulator borrows your host machine's CPU and memory and will happily hide a device that is too slow.
- 2 Camera, GPS, biometrics, Bluetooth and other sensors, which emulators either stub out or approximate.
- 3 Real network behaviour on a real radio, including how the app copes with a genuinely poor mobile connection.
- 4 Vendor-specific behaviour, where the manufacturer's own build of Android is what breaks rather than Android itself.
Most teams end up running both: [emulators and simulators](https://testingbot.com/features/manual-mobile-testing) on every commit, and real hardware before a release, on the handsets their own analytics say matter most.
What you need to buy
## Which plans include real devices
Real devices are not a premium tier here. Every TestingBot subscription includes them, and so do the pay-as-you-go minute packs.
### Manual plan
Live, hands-on sessions on real iPhones, iPads and Android handsets, driven from your browser. Includes the hosted emulators and simulators too.
### Automated plan
Appium, XCUITest, Espresso and Maestro runs on real devices, plus the full desktop browser grid. Codeless AI, visual testing and accessibility scans are included rather than sold separately.
### Pro plan
Everything in Manual and Automated on a single subscription, for teams doing both against the same device fleet.
### Pay-as-you-go credits
One-time minute packs with no subscription. Minutes cover live manual sessions and automated runs on real devices, desktop browsers, emulators and simulators.
Current tiers and parallel-session counts are on the [pricing page](https://testingbot.com/pricing). Signing up is free and needs no card. The 7-day full-access trial, which unlocks every real device, asks for a card and charges nothing during the trial.
How we compare
## Real Device Testing: Quick Comparison
How TestingBot's real device cloud compares to other major providers.
| Capability | TestingBot | BrowserStack | SauceLabs |
| --- | --- | --- | --- |
| Real iOS & Android devices | | | |
| European-hosted devices (data residency) | Yes | US primary | US primary |
| GDPR compliant, CSA STAR Level 1 self-assessment | | | |
| Appium / XCUITest / Espresso / Maestro | | | No Maestro |
| Free for open source | Yes | | |
| Dedicated private device cloud | | | |
| Secure tunnel for private/staging apps | | | |
| Transparent flat-rate pricing | Yes | Custom quote | Custom quote |
See more on the [SauceLabs alternative](https://testingbot.com/saucelabs-alternative), [BrowserStack alternative](https://testingbot.com/browserstack-alternative) and [LambdaTest alternative](https://testingbot.com/lambdatest-alternative) pages.
Real-world conditions
## Test native device features
Access and modify device features to test your app in various real-world conditions.
### Network Simulation
Test for scenarios like connectivity, fluctuating network & many more.
### Localization testing
Test your apps across different timezones, languages and geolocations.
### Dark Mode
Test your app's UI with a dark background to ensure a smooth user experience.
### App Orientation
Test your app in the landscape or portrait mode. Switch between orientations to test your app's response.
### Audio
Record and hear in realtime the audio generated on the device.
### Settings
Control various settings on the device.
Private device cloud
## Your very own Mobile Devices
Choose your own iOS and Android devices. We will purchase these devices, install and configure them in our secure datacenter.
These devices will be only available to you, 24x7.
Private devices can be customised to your needs. Have us install specific profiles and apps, or change specific settings of the device. You decide.
[Learn more](https://testingbot.com/enterprise/private-device-cloud)
Secure Tunnel
## Test Staged Websites
Use the secure TestingBot Tunnel to easily test your staging and development environments across all browsers, ensuring seamless access behind firewalls or private networks.
Test data and websites from behind your firewall on the TestingBot cloud.
[Read more](https://testingbot.com/support/tunnel)
FAQ
## Frequently Asked Questions
Common questions about real device testing on TestingBot.
What is real device testing?
Real device testing is the practice of running automated or manual tests on physical smartphones and tablets, as opposed to emulators or simulators. It surfaces hardware-specific issues like CPU/RAM constraints, battery behaviour, real network conditions and device-specific rendering quirks that emulators or simulators cannot reproduce.
Should I use a real device or an emulator?
Use emulators for fast feedback during early development and unit-style flows. Use real devices for release validation, performance testing, biometric flows, camera/sensor logic, push notifications and final acceptance testing. Most teams run both: emulators in CI on every commit, real devices on the release branch.
Where are TestingBot's real devices located?
All TestingBot devices are hosted in a secure datacenter in Belgium, Europe. Every device is wiped and re-imaged after each test session. None are jailbroken or rooted.
Which test frameworks does the real device cloud support?
Appium (iOS and Android), XCUITest (iOS), Espresso (Android), Maestro (iOS and Android), Mobilewright (iOS and Android) and any WebDriver-compatible framework. You can also drive devices manually from your browser, or use TestingBot's codeless AI test creator to generate tests from plain English.
Can I test apps that are not yet on the App Store or Google Play?
Yes. Upload an .ipa or .apk build to TestingBot Storage and install it on any real device. You can also test apps already published to the App Store or Google Play, or connect to staging environments behind a corporate firewall using TestingBot Tunnel.
What's the difference between the public and private device cloud?
Public-cloud devices are shared across customers and are wiped between sessions. Private-cloud devices are dedicated phones we purchase, configure and host exclusively for your account, with optional persistence between tests, custom profiles and pre-installed apps.
How does TestingBot compare to BrowserStack and SauceLabs for real device testing?
TestingBot offers European-hosted real devices (BrowserStack and SauceLabs are primarily US-hosted), GDPR and CSA STAR compliance, transparent flat-rate pricing, free for open source, and identical Appium/XCUITest/Espresso APIs. Customers usually migrate for the EU data residency, faster device allocation and direct support from the engineering team.
Looking for hands-on details? See the [live mobile app testing](https://testingbot.com/features/manual-mobile-testing) page or browse our [supported devices](https://testingbot.com/support/app-automate/devices).
Start in seconds
### Sign up for a Free Trial
Start testing your apps with TestingBot.
[Start a free trial](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo)
Resources you may like
[](https://testingbot.com/resources/articles/appium-flutter-app-testing)
### [Appium Flutter Driver: Testing Flutter Apps](https://testingbot.com/resources/articles/appium-flutter-app-testing)
How to test a Flutter app with Appium: what the framework compiles to, how the Appium Flutter Driver works, and how to build an app for testing.
[Read more: Appium Flutter Driver: Testing Flutter Apps](https://testingbot.com/resources/articles/appium-flutter-app-testing)
[](https://testingbot.com/resources/articles/localized-testing-mobile-app)
### [How to do localisation testing for mobile apps](https://testingbot.com/resources/articles/localized-testing-mobile-app)
Find out how to perform localisation testing with mobile apps. Change your language or locale with Appium, XCUITest and Espresso.
[Read more: How to do localisation testing for mobile apps](https://testingbot.com/resources/articles/localized-testing-mobile-app)
[](https://testingbot.com/resources/articles/dark-mode-testing)
### [Dark Mode Testing with Appium](https://testingbot.com/resources/articles/dark-mode-testing)
How to switch a native app between light and dark mode from an Appium test on iOS and Android, and the layout bugs dark mode tends to expose.
[Read more: Dark Mode Testing with Appium](https://testingbot.com/resources/articles/dark-mode-testing)
---
URL: https://testingbot.com/playwright-vs-selenium
Framework comparison · 2026 Last updated 2026-08-21
# Playwright vs Selenium
Playwright and Selenium are the two dominant browser automation frameworks. This guide compares them head-to-head on speed, language support, browser coverage, debugging, and which to pick, with code samples and a verdict you can act on in five minutes.
[Run both on TestingBot](https://testingbot.com/users/sign_up) [Jump to comparison](https://testingbot.com#comparison)
- **Browsers & devices:** 6100+
- **Cloud parallelism:** 100×
- **Uptime SLA:** 99.9%
## Trusted by some of the world's most innovative companies
## On this page
1. [Which one should you pick?](https://testingbot.com#which-should-you-pick)
2. [What are Playwright and Selenium?](https://testingbot.com#what-are-playwright-and-selenium)
3. [Playwright vs Selenium: side-by-side comparison](https://testingbot.com#comparison)
4. [Filling a form, asserting the result](https://testingbot.com#code-comparison)
5. [Where Cypress fits](https://testingbot.com#where-cypress-fits)
6. [When to choose which](https://testingbot.com#when-to-choose-which)
7. [Stop choosing: run both on the same grid](https://testingbot.com#run-both)
8. [Frequently Asked Questions](https://testingbot.com#faq)
30-second answer
## Which one should you pick?
 Pick Playwright if…
You're starting a new web project, your team writes JavaScript or TypeScript, and you want fast tests, auto-waiting and built-in tracing out of the box.
 Pick Selenium if…
Your team writes Java, C# or Ruby, you need legacy browser support (Internet Explorer, older Safari), or you have an existing Selenium suite and a strong Page Object library.
Pick both if…
You have separate frontend and backend test suites, you're migrating gradually, or you want Playwright for new code and Selenium for the long-tail of legacy browsers. TestingBot runs both on the same grid.
Background
## What are Playwright and Selenium?
Two frameworks, two philosophies. The same goal: drive a real browser to automate end-to-end testing.
### Playwright
Released 2020 · Microsoft · Apache 2.0
Playwright is an end-to-end testing framework built by Microsoft that controls Chromium, Firefox and WebKit through a single API. It was designed by the team that originally built Puppeteer at Google, with cross-browser support and modern web app needs in mind.
It uses the browser's DevTools protocol (or its own bridge for Firefox/WebKit) instead of WebDriver, which lets it auto-wait for elements, intercept network traffic and capture rich traces by default.
- TypeScript / JavaScript / Python / Java / C# bindings
- Auto-wait, network interception, fixtures, traces
- Chromium, Firefox, WebKit · no IE / legacy Safari
### Selenium
Released 2004 · Open Source · Apache 2.0
Selenium is the original browser automation framework and the foundation of the W3C WebDriver standard. Selenium WebDriver drives any browser through a vendor-supplied driver (chromedriver, geckodriver, safaridriver, edgedriver, IEDriverServer) using the same wire protocol.
Twenty years of ecosystem maturity means deep IDE integration, mature Page Object patterns, and bindings for every major language. Selenium 4 added Chrome DevTools support, BiDi events and bidirectional protocols.
- Java / Python / C# / Ruby / JavaScript / Kotlin
- W3C WebDriver standard, works with every browser
- Chrome, Firefox, Safari, Edge, IE 11: full coverage
Head to head
## Playwright vs Selenium: side-by-side comparison
Across the dimensions that matter for picking, migrating or running both in CI.
| Dimension | Playwright | Selenium | Cypress |
| --- | --- | --- | --- |
| First release | 2020 | 2004 | 2017 |
| Maintained by | Microsoft | Open-source community + W3C | Cypress.io |
| Languages | TS, JS, Python, Java, C# | Java, Python, C#, Ruby, JS, Kotlin | JavaScript / TypeScript only |
| Browsers | Chromium, Firefox, WebKit | Chrome, Firefox, Safari, Edge, IE 11 | Chrome, Edge, Firefox, WebKit (experimental) |
| Mobile testing | Mobile emulation only | Real iOS + Android via Appium | Viewport emulation only |
| Wire protocol | CDP + custom (Firefox/WebKit) | W3C WebDriver standard | Executes inside the browser |
| Auto-waiting | Built in everywhere | Manual waits or WebDriverWait | Built in everywhere |
| Network interception | First-class API | BiDi (Selenium 4) or proxy | First-class (cy.intercept) |
| Multiple tabs / origins | Native, same context | Window switching API | No multi-tab; cy.origin for cross-origin |
| Test runner | Built-in (@playwright/test) | Bring your own (JUnit, pytest, Mocha) | Built-in |
| Parallel execution | Built-in worker pool | Selenium Grid or test-runner level | Cypress Cloud or CI-level sharding |
| Trace / replay | Built-in Trace Viewer | Screenshots + manual logs | Time-travel debugger + video |
| Debugging | Inspector + UI mode | IDE + driver logs | Interactive test runner |
| Speed (typical suite) | About 30-40% faster on Chromium | Mature, predictable | Fast, but one browser per run |
| Ecosystem maturity | Growing fast since 2020 | 20 years of integrations | Large plugin ecosystem |
| Page Object pattern | Supported, less common | Industry standard | Supported, discouraged by the docs |
| Runs on TestingBot | ✓ | ✓ | ✓ |
Speed numbers from public benchmarks; vary by application. Both frameworks run on TestingBot's cloud with identical APIs to local execution.
The same test in both
## Filling a form, asserting the result
A login flow with auto-wait (Playwright) versus an explicit wait (Selenium). Both pass on the same TestingBot grid.
Playwright + TypeScript login.spec.ts
```typescript
// playwright.config.ts → set TestingBot endpoint
import { test, expect } from '@playwright/test';
test('login redirects to dashboard', async ({ page }) => {
await page.goto('https://app.example.com/login');
// auto-wait, no explicit wait needed
await page.getByLabel('Username').fill('jane@example.com');
await page.getByLabel('Password').fill('••••••••');
await page.getByRole('button', { name: 'Sign in' }).click();
await expect(page).toHaveURL(/\/dashboard/);
await expect(page.getByRole('heading', { name: /welcome, jane/i })).toBeVisible();
});
```
Selenium + Python test\_login.py
```python
# Driver points at TestingBot remote URL
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
def test_login_redirects_to_dashboard(driver):
driver.get('https://app.example.com/login')
driver.find_element(By.NAME, 'username').send_keys('jane@example.com')
driver.find_element(By.NAME, 'password').send_keys('••••••••')
driver.find_element(By.CSS_SELECTOR, 'button[type=submit]').click()
# explicit wait
WebDriverWait(driver, 10).until(EC.url_contains('/dashboard'))
welcome = driver.find_element(By.TAG_NAME, 'h1')
assert 'Welcome, Jane' in welcome.text
```
Both tests are pointed at the same [Playwright](https://testingbot.com/features/automation/playwright) / [Selenium](https://testingbot.com/features/automation/selenium) endpoint on TestingBot's cloud. Same browsers, same parallelism, same dashboard.
The third option
## Where Cypress fits
Most teams weighing Playwright against Selenium look at Cypress too. Here is the short version.
Cypress runs your test code inside the browser alongside the application, rather than driving it from the outside. That is what gives it the interactive test runner and time-travel debugging people like, and it is also the source of its limits: JavaScript and TypeScript only, no native mobile app testing, no multiple browser tabs, and cross-origin work needs cy.origin.
Against Playwright, the trade is developer experience versus reach. Playwright covers more languages and more browser engines and has no trouble with tabs or origins. Against Selenium, the trade is ergonomics versus coverage: Selenium remains the only one of the three that drives real iOS and Android devices, through Appium.
If Cypress is genuinely on your shortlist, the dedicated [Cypress vs Playwright](https://testingbot.com/cypress-vs-playwright) and [Cypress vs Selenium](https://testingbot.com/cypress-vs-selenium) comparisons go through it properly. All three run on TestingBot.
Decision matrix
## When to choose which
### Choose Playwright when
- You're starting a green-field test suite, no migration cost to weigh.
- Your stack is JavaScript / TypeScript-first and you want the test runner that ships with the framework.
- You need network mocking, multi-tab flows, file uploads or geolocation overrides without bolt-on tools.
- You want trace-based debugging; record, save, scrub through a failed run frame-by-frame.
- Your target browsers are Chromium, Firefox and WebKit. No Internet Explorer 11 or older Safari requirement.
[Run Playwright on TestingBot](https://testingbot.com/features/automation/playwright)
### Choose Selenium when
- Your team writes Java, C# or Ruby. Playwright's bindings exist but the JS-first ergonomics get diluted.
- You need to test on Internet Explorer 11, older Safari, or unusual browser builds where only WebDriver works.
- You have a mature Page Object library, custom locators, or BDD framework (Cucumber, SpecFlow) you don't want to throw away.
- You're standardising across web AND real mobile devices. Selenium's API is the closest thing to a unified WebDriver story (with Appium for mobile).
- Stability and ecosystem maturity outweigh fewer features. Selenium has been battle-tested for 20 years.
[Run Selenium on TestingBot](https://testingbot.com/features/automation/selenium)
TestingBot supports both
## Stop choosing: run both on the same grid
Point Playwright at the TestingBot endpoint, point Selenium at the same one, and your tests share the same 6100+ browsers and devices, the same dashboard, the same parallel slots and the same EU data residency.
- Same auth, same project, same billing
- Side-by-side test history for both frameworks
- Free for open source, both frameworks
[Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo)
testingbot.com / endpoint
// Playwright
use: { wsEndpoint: 'wss://cloud.testingbot.com/playwright' }
# Selenium
webdriver.Remote(
command\_executor='https://hub.testingbot.com/wd/hub'
)
FAQ
## Frequently Asked Questions
The questions teams ask before picking these frameworks, or migrating between them.
Is Playwright faster than Selenium?
Generally yes, on Chromium-based browsers. Typical suites run 30–40% faster on Playwright thanks to the DevTools protocol bypassing WebDriver's HTTP round-trips. The gap closes for Firefox and Safari (where Playwright uses its own bridges) and disappears once your tests are network-bound. Selenium 4's BiDi support narrows the gap further. For most teams the speed difference is real but not the deciding factor.
Should I migrate from Selenium to Playwright?
Only if the cost is justified. If your existing Selenium suite is stable, in a non-JavaScript language, and uses Page Objects extensively, migrating is rarely worth the effort. If your suite is flaky, JS-based and lacks tracing, Playwright's auto-wait and Trace Viewer alone usually pay for the migration. Pragmatic teams keep both: Selenium for what's already working, Playwright for new test suites.
Does Playwright support all browsers Selenium does?
No. Playwright supports Chromium (and Chrome/Edge by extension), Firefox and WebKit (which is close to Safari but not identical). Selenium drives the actual browser builds, including Internet Explorer 11, older Safari versions and rare browsers, using vendor-shipped drivers. If you need to test on IE 11 or a specific legacy Safari, Selenium is your only option.
Can Playwright test mobile apps?
Not natively. Playwright offers mobile browser emulation (you can simulate a mobile viewport and user agent in Chromium), but it cannot drive native iOS or Android apps. For real mobile app testing you need [Appium](https://testingbot.com/features/automation/appium), [XCUITest](https://testingbot.com/features/automation/xcuitest), [Espresso](https://testingbot.com/features/automation/espresso) or [Maestro](https://testingbot.com/features/automation/maestro). Selenium can drive mobile via Appium, which is built on the WebDriver protocol. TestingBot supports all of these on real iOS and Android devices.
Which is better for cross-browser testing?
For modern Chromium / Firefox / WebKit, Playwright's single API gives you faster, more reliable cross-browser runs out of the box. For comprehensive coverage including legacy browsers (IE 11, older Safari, Edge Legacy), Selenium is still the only option that drives the real browser binaries. Either way, running across many real browsers in parallel benefits hugely from using TestingBot's cloud. Both frameworks integrate with TestingBot's cloud with a one-line config change.
Do I need to learn JavaScript to use Playwright?
No, but Playwright's ergonomics are JS/TS-first. The Python, Java and C# bindings are first-class but lag the JS API on new features by weeks. If your team already writes Python or Java, Playwright is still a strong choice, but Selenium's binding parity makes it the more natural fit for non-JS stacks.
Can I run Playwright and Selenium on TestingBot?
Yes, both run on the same TestingBot cloud. Playwright connects via the wsEndpoint config to `wss://cloud.testingbot.com/playwright`, Selenium connects to `https://hub.testingbot.com/wd/hub` via `webdriver.Remote`. Tests show up in the same dashboard, share parallel slots, and benefit from the same EU data residency, video recording, live debugging and CI/CD integrations. Both are free for open source projects.
Is Playwright going to replace Selenium?
Not wholesale. Playwright is taking a large share of new JavaScript and TypeScript projects, but Selenium is a W3C standard with 20 years of installed base, bindings for six languages, and the only route to real iOS and Android devices via Appium. The realistic outcome is coexistence: new suites lean Playwright, large existing suites and non-JS stacks stay on Selenium. Plenty of teams run both.
Is Selenium outdated now?
No. Selenium 4 moved to the W3C WebDriver standard and added relative locators, a Chrome DevTools Protocol bridge and BiDi support. It is slower to write and needs explicit waits more often than Playwright, but it is actively maintained and remains the broadest option for language and browser coverage, including Safari, Internet Explorer 11 and real mobile devices.
Can I learn Playwright without Selenium?
Yes. Playwright does not build on Selenium and shares none of its API, so there is nothing you need to unlearn or learn first. Prior Selenium experience helps mainly with general concepts such as locators, waits and the Page Object pattern, not with Playwright syntax. If you are starting fresh, learning Playwright directly is the shorter path.
Does Playwright require coding?
Yes. Playwright is a code-first framework: tests are written in TypeScript, JavaScript, Python, Java or C#. The Codegen tool records your browser actions and generates a working test script, which lowers the barrier considerably, but the output is still code you edit and maintain. There is no no-code mode.
Going deeper? See the dedicated [Playwright](https://testingbot.com/features/automation/playwright) and [Selenium](https://testingbot.com/features/automation/selenium) pages.
## Related comparisons
[Cypress vs Selenium](https://testingbot.com/cypress-vs-selenium) [Cypress vs Playwright](https://testingbot.com/cypress-vs-playwright)
### Sign up for a Free Trial
Run Playwright, Selenium, or both, on TestingBot's cloud.
[Start a free trial](https://testingbot.com/users/sign_up)
Resources you may like
[](https://testingbot.com/resources/articles/playwright-visual-regression-testing)
### [Visual Testing with Playwright](https://testingbot.com/resources/articles/playwright-visual-regression-testing)
Playwright provides automated browser testing. It offers a built-in feature to perform visual regression testing for your website.
[Read more: Visual Testing with Playwright](https://testingbot.com/resources/articles/playwright-visual-regression-testing)
[](https://testingbot.com/resources/articles/generative-ai-selenium)
### [Selenium and Generative AI](https://testingbot.com/resources/articles/generative-ai-selenium)
How to use generative AI to produce realistic test data for Selenium runs, and why varied data uncovers bugs that fixed fixtures never will.
[Read more: Selenium and Generative AI](https://testingbot.com/resources/articles/generative-ai-selenium)
[](https://testingbot.com/resources/articles/playwright-recorder)
### [Playwright Codegen: Record Tests in VS Code](https://testingbot.com/resources/articles/playwright-recorder)
How to record Playwright scripts from Visual Studio Code rather than writing them by hand, then run the recorded tests on a browser grid.
[Read more: Playwright Codegen: Record Tests in VS Code](https://testingbot.com/resources/articles/playwright-recorder)
---
URL: https://testingbot.com/cypress-vs-selenium
Framework comparison · 2026
# Cypress vs Selenium
Cypress and Selenium take opposite approaches to browser testing: one runs inside the browser, the other drives it from outside. This guide compares them head-to-head on architecture, languages, browser coverage, parallelisation and debugging, with code samples and a verdict you can act on in five minutes.
[Run both on TestingBot](https://testingbot.com/users/sign_up) [Jump to comparison](https://testingbot.com#comparison)
- **Browsers & devices:** 6100+
- **Cloud parallelism:** 100×
- **Uptime SLA:** 99.9%
## Trusted by some of the world's most innovative companies
30-second answer
## Which one should you pick?
 Pick Cypress if…
Your team writes JavaScript or TypeScript, you test a modern web app on Chrome, Edge or Firefox, and you want the best developer experience with built-in retries, time-travel debugging and a test runner included.
 Pick Selenium if…
Your team writes Java, C# or Ruby, you need legacy browser support (Internet Explorer, Safari), real mobile devices via Appium, or multi-tab and cross-origin flows that Cypress cannot handle.
Pick both if…
You want Cypress for fast component and frontend tests and Selenium for broad cross-browser and mobile coverage. TestingBot runs both on the same grid, in parallel, in one dashboard.
Background
## What are Cypress and Selenium?
Two frameworks, two architectures. The same goal: drive a browser to automate end-to-end testing.
### Cypress
Released 2017 · Cypress.io · MIT
Cypress is a JavaScript end-to-end testing framework that runs your tests inside the browser, in the same run loop as your application. That architecture gives it direct access to the DOM, automatic waiting and time-travel snapshots out of the box.
Because it lives in the browser, Cypress has an excellent developer experience: a built-in test runner, real-time reloads and a visual debugger. The same architecture also limits it to JavaScript and TypeScript, a fixed set of browsers, and single-tab flows.
- JavaScript and TypeScript only
- Auto-retries, time-travel debugging, cy.intercept
- Chrome, Edge, Firefox, Electron · no IE / Safari
### Selenium
Released 2004 · Open Source · Apache 2.0
Selenium is the original browser automation framework and the foundation of the W3C WebDriver standard. Selenium WebDriver drives any browser from the outside through a vendor-supplied driver (chromedriver, geckodriver, safaridriver, edgedriver) using the same wire protocol.
Twenty years of ecosystem maturity means bindings for every major language, deep IDE integration, mature Page Object patterns, and real mobile device testing through Appium, which is built on the same protocol.
- Java / Python / C# / Ruby / JavaScript / Kotlin
- W3C WebDriver standard, works with every browser
- Chrome, Firefox, Safari, Edge, IE 11 · real mobile via Appium
Head to head
## Cypress vs Selenium: side-by-side comparison
Across the dimensions that matter for picking, migrating or running both in CI.
Cypress versus Selenium feature comparison| Dimension | Cypress | Selenium |
| --- | --- | --- |
| First release | 2017 | 2004 |
| Maintained by | Cypress.io | Open-source community + W3C |
| Languages | JavaScript / TypeScript only | Java, Python, C#, Ruby, JS, Kotlin |
| Architecture | Runs inside the browser | Drives the browser via WebDriver |
| Browsers | Chrome, Edge, Firefox, Electron | Chrome, Firefox, Safari, Edge, IE 11 |
| Mobile testing | Viewport emulation only | Real iOS + Android via Appium |
| Auto-waiting | Built-in retries | Manual waits or WebDriverWait |
| Network stubbing | First-class (cy.intercept) | BiDi (Selenium 4) or proxy |
| Multiple tabs / windows | Not supported | Window switching API |
| Cross-origin | cy.origin() (constrained) | Native |
| Test runner | Built in | Bring your own (JUnit, pytest, Mocha…) |
| Debugging | Time-travel snapshots + runner | IDE + driver logs |
| Parallel execution | Cypress Cloud (paid) or a grid | Selenium Grid |
| Developer experience | Real-time reload, in-browser | Functional, IDE-based |
| Ecosystem maturity | Growing fast since 2017 | 20 years of integrations |
| Free for open source on TestingBot | ✓ | ✓ |
Feature notes reflect Cypress 13.x and Selenium 4.x as of 2026. Both frameworks run on TestingBot's cloud, Cypress through the testingbot-cypress-cli and Selenium through the WebDriver hub.
The same test in both
## Logging in, asserting the result
A login flow with built-in retries (Cypress) versus an explicit wait (Selenium). Both run on the same TestingBot grid.
Cypress + TypeScript login.cy.ts
```typescript
// run via: testingbot-cypress run
describe('login', () => {
it('redirects to the dashboard', () => {
cy.visit('https://app.example.com/login');
// auto-retries until actionable
cy.get('#username').type('jane@example.com');
cy.get('#password').type('••••••••');
cy.contains('button', 'Sign in').click();
cy.url().should('include', '/dashboard');
cy.contains('h1', /welcome, jane/i).should('be.visible');
});
});
```
Selenium + Python test\_login.py
```python
# Driver points at TestingBot remote URL
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
def test_login_redirects_to_dashboard(driver):
driver.get('https://app.example.com/login')
driver.find_element(By.NAME, 'username').send_keys('jane@example.com')
driver.find_element(By.NAME, 'password').send_keys('••••••••')
driver.find_element(By.CSS_SELECTOR, 'button[type=submit]').click()
# explicit wait
WebDriverWait(driver, 10).until(EC.url_contains('/dashboard'))
welcome = driver.find_element(By.TAG_NAME, 'h1')
assert 'Welcome, Jane' in welcome.text
```
The Cypress spec runs through the [testingbot-cypress-cli](https://testingbot.com/support/web-automate/cypress); the Selenium test connects to the [WebDriver hub](https://testingbot.com/support/web-automate/selenium). Both appear in the same TestingBot dashboard.
Decision matrix
## When to choose which
### Choose Cypress when
- Your stack is JavaScript or TypeScript-first and you want the test runner, assertions and retries to ship with the framework.
- You are testing a modern web app on Chrome, Edge or Firefox, with no Internet Explorer or Safari requirement.
- You value developer experience: time-travel debugging, real-time reloads and readable failure snapshots.
- You want first-class network stubbing with cy.intercept, without bolting on a proxy.
- Your flows stay within a single tab and mostly a single origin (cy.origin helps, within limits).
[Run Cypress on TestingBot](https://testingbot.com/features/automation/cypress)
### Choose Selenium when
- Your team writes Java, C#, Ruby or Python and you want full binding parity, not a JavaScript-first API.
- You need to test on Internet Explorer 11, Safari, or browser builds Cypress does not support.
- You test real iOS and Android devices, where Appium reuses the WebDriver protocol Selenium is built on.
- Your tests open multiple tabs or windows, or cross several origins, which Cypress handles only partially.
- You have a mature Page Object library or BDD framework (Cucumber, SpecFlow) you do not want to rewrite.
[Run Selenium on TestingBot](https://testingbot.com/features/automation/selenium)
TestingBot supports both
## Stop choosing, run both on the same grid
Run Cypress through the testingbot-cypress-cli and point Selenium at the WebDriver hub. Your tests share the same 6100+ browsers and devices, the same dashboard, the same parallel slots and the same EU data residency, with no Cypress Cloud subscription required.
- Cypress parallelisation without a Cypress Cloud plan
- Side-by-side test history for both frameworks
- Free for open source, both frameworks
[Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo)
testingbot.com / run both
// Cypress
$ npm i -D testingbot-cypress-cli
$ testingbot-cypress run # reads testingbot.json
# Selenium
webdriver.Remote(
command\_executor='https://hub.testingbot.com/wd/hub'
)
FAQ
## Frequently Asked Questions
The questions teams ask before picking, or migrating between, these frameworks.
Is Cypress better than Selenium?
Neither is universally better; they suit different teams. Cypress wins on developer experience for JavaScript and TypeScript teams testing modern web apps: built-in retries, time-travel debugging and a test runner included. Selenium wins on breadth: more languages, every browser including Internet Explorer and Safari, real mobile via Appium, and multi-tab and cross-origin flows. Pick Cypress for ergonomics within its constraints; pick Selenium for coverage and flexibility.
Should I migrate from Selenium to Cypress?
Only if your needs fit within Cypress's constraints. If your suite is JavaScript-based, targets Chrome, Edge or Firefox, and stays within a single tab, Cypress's retries and debugging often justify the move. If you rely on Java, C# or Ruby, test Internet Explorer or Safari, drive real mobile devices, or need multi-tab and cross-origin flows, migrating will cost you capabilities. Many teams keep both: Cypress for frontend tests, Selenium for broad cross-browser and mobile coverage.
Does Cypress support all the browsers Selenium does?
No. Cypress runs on Chromium-based browsers (Chrome, Edge, Electron) and Firefox, with experimental WebKit support. It cannot drive Internet Explorer or real Safari. Selenium drives the actual browser binaries through vendor drivers, including Internet Explorer 11 and Safari. If you need IE or Safari coverage, Selenium is the only option of the two.
Can Cypress test mobile apps?
Not natively. Cypress offers viewport resizing to emulate a mobile screen, but it cannot drive native iOS or Android apps. For real mobile testing you need [Appium](https://testingbot.com/features/automation/appium), [XCUITest](https://testingbot.com/features/automation/xcuitest), [Espresso](https://testingbot.com/features/automation/espresso) or [Maestro](https://testingbot.com/features/automation/maestro). Appium reuses the WebDriver protocol Selenium is built on. TestingBot runs all of these on real iOS and Android devices.
Do I need Cypress Cloud to run Cypress in parallel?
No. Cypress's own parallelisation and recording run through Cypress Cloud, a paid subscription. TestingBot is an alternative: the testingbot-cypress-cli zips your specs, uploads them and runs them in parallel across Chrome, Edge and Firefox on the cloud grid, with live logs and video, and no Cypress Cloud plan. You set the parallel count in testingbot.json.
Can Cypress and Selenium share the same CI pipeline?
Yes. Both run in CI/CD and report results back to the same place on TestingBot. A common setup runs Cypress for fast frontend specs and Selenium for broad cross-browser and mobile coverage in the same pipeline, both pointed at the TestingBot grid with the same credentials and build name.
Can I run Cypress and Selenium on TestingBot?
Yes, both run on the same TestingBot cloud. Cypress runs through the `testingbot-cypress-cli` npm package, configured with a `testingbot.json` file and started with `testingbot-cypress run`. Selenium connects to `https://hub.testingbot.com/wd/hub` via `webdriver.Remote`. Tests from both show up in the same dashboard, share parallel slots, and benefit from the same EU data residency, video recording and CI/CD integrations. Both are free for open source projects.
Going deeper? See the dedicated [Cypress](https://testingbot.com/features/automation/cypress) and [Selenium](https://testingbot.com/features/automation/selenium) pages.
## Related comparisons
[Cypress vs Playwright](https://testingbot.com/cypress-vs-playwright) [Playwright vs Selenium](https://testingbot.com/playwright-vs-selenium)
### Sign up for a Free Trial
Run Cypress, Selenium, or both, on TestingBot's cloud. No Cypress Cloud subscription required.
[Start a free trial](https://testingbot.com/users/sign_up)
Resources you may like
[](https://testingbot.com/resources/articles/how-to-debug-cypress-tests)
### [Tutorial on debugging Cypress tests](https://testingbot.com/resources/articles/how-to-debug-cypress-tests)
How to debug a failing Cypress test using the debugger, console logs, the stack trace, and the debug and pause commands.
[Read more: Tutorial on debugging Cypress tests](https://testingbot.com/resources/articles/how-to-debug-cypress-tests)
[](https://testingbot.com/resources/articles/generative-ai-selenium)
### [Selenium and Generative AI](https://testingbot.com/resources/articles/generative-ai-selenium)
How to use generative AI to produce realistic test data for Selenium runs, and why varied data uncovers bugs that fixed fixtures never will.
[Read more: Selenium and Generative AI](https://testingbot.com/resources/articles/generative-ai-selenium)
[](https://testingbot.com/resources/articles/cypress-cucumber-testing)
### [Cypress and Cucumber Browser Testing](https://testingbot.com/resources/articles/cypress-cucumber-testing)
How to combine Cypress with Cucumber so end-to-end tests read as plain language, including a worked example and the TypeScript setup.
[Read more: Cypress and Cucumber Browser Testing](https://testingbot.com/resources/articles/cypress-cucumber-testing)
---
URL: https://testingbot.com/cypress-vs-playwright
Framework comparison · 2026
# Cypress vs Playwright
Cypress and Playwright are the two leading modern JavaScript end-to-end testing frameworks. This guide compares them head-to-head on architecture, browser coverage, languages, parallel execution and debugging, with code samples and a verdict you can act on in five minutes.
[Run both on TestingBot](https://testingbot.com/users/sign_up) [Jump to comparison](https://testingbot.com#comparison)
- **Browsers & devices:** 6100+
- **Cloud parallelism:** 100×
- **Uptime SLA:** 99.9%
## Trusted by some of the world's most innovative companies
30-second answer
## Which one should you pick?
 Pick Cypress if…
You want the most polished developer experience: an in-browser test runner, time-travel debugging, automatic retries and first-class component testing, and your targets are Chrome, Edge and Firefox.
 Pick Playwright if…
You need WebKit (Safari engine) coverage, more than one language (TypeScript, Python, Java, C#), multi-tab or multi-origin flows, or free built-in parallel execution without a paid dashboard.
Pick both if…
You have an existing Cypress suite you like and want Playwright for new cross-browser or multi-tab scenarios. TestingBot runs both on the same grid, in parallel, in one dashboard.
Background
## What are Cypress and Playwright?
Two modern JavaScript testing frameworks with different architectures. The same goal: drive a browser to automate end-to-end testing.
### Cypress
Released 2017 · Cypress.io · MIT
Cypress is a JavaScript end-to-end testing framework that runs your tests inside the browser, in the same run loop as your application. That architecture gives it direct DOM access, automatic retries and time-travel snapshots out of the box.
Cypress is known for its developer experience: a built-in test runner, real-time reloads, a visual debugger and strong component testing. The in-browser model also limits it to JavaScript and TypeScript, a fixed set of Chromium-based browsers plus Firefox, and single-tab flows.
- JavaScript and TypeScript only
- Auto-retries, time-travel debugging, component testing
- Chrome, Edge, Firefox, Electron · no WebKit/Safari
### Playwright
Released 2020 · Microsoft · Apache 2.0
Playwright is an end-to-end testing framework built by Microsoft that controls Chromium, Firefox and WebKit through a single API. It drives the browser out-of-process over the DevTools protocol (and its own bridges for Firefox and WebKit).
Playwright was designed for scale: bindings in five languages, native multi-tab and multi-origin support through browser contexts, built-in parallel workers, network interception and a rich Trace Viewer, all included.
- TypeScript, JavaScript, Python, Java, C#
- Auto-wait, browser contexts, Trace Viewer, free parallel
- Chromium, Firefox, WebKit · no IE / legacy Safari
Head to head
## Cypress vs Playwright: side-by-side comparison
Across the dimensions that matter for picking, migrating or running both in CI.
Cypress versus Playwright feature comparison| Dimension | Cypress | Playwright |
| --- | --- | --- |
| First release | 2017 | 2020 |
| Maintained by | Cypress.io | Microsoft |
| Languages | JavaScript / TypeScript only | TS, JS, Python, Java, C# |
| Architecture | Runs inside the browser | Drives browser out-of-process |
| Browsers | Chrome, Edge, Firefox, Electron | Chromium, Firefox, WebKit |
| WebKit / Safari engine | Experimental | Supported |
| Auto-waiting | Built-in retries | Built-in auto-wait |
| Multiple tabs / windows | Not supported | Native (browser contexts) |
| Cross-origin | cy.origin() (constrained) | Native |
| Network interception | First-class (cy.intercept) | First-class (route) |
| Test runner | Built in | Built in (@playwright/test) |
| Component testing | First-class | Experimental |
| Parallel execution | Cypress Cloud (paid) or a grid | Built-in workers (free) |
| Debugging | Time-travel runner | Trace Viewer + UI mode |
| Mobile testing | Viewport emulation only | Viewport emulation only |
| Ecosystem maturity | Since 2017 | Since 2020, growing fast |
| Free for open source on TestingBot | ✓ | ✓ |
Feature notes reflect Cypress 13.x and Playwright 1.x as of 2026. Both frameworks run on TestingBot's cloud, Cypress through the testingbot-cypress-cli and Playwright through the wsEndpoint connection.
The same test in both
## Logging in, asserting the result
A login flow with built-in retries (Cypress) and auto-wait (Playwright). Both run on the same TestingBot grid.
Cypress + TypeScript login.cy.ts
```typescript
// run via: testingbot-cypress run
describe('login', () => {
it('redirects to the dashboard', () => {
cy.visit('https://app.example.com/login');
// auto-retries until actionable
cy.get('#username').type('jane@example.com');
cy.get('#password').type('••••••••');
cy.contains('button', 'Sign in').click();
cy.url().should('include', '/dashboard');
cy.contains('h1', /welcome, jane/i).should('be.visible');
});
});
```
Playwright + TypeScript login.spec.ts
```typescript
// playwright.config.ts → set TestingBot wsEndpoint
import { test, expect } from '@playwright/test';
test('login redirects to dashboard', async ({ page }) => {
await page.goto('https://app.example.com/login');
// auto-wait, no explicit wait needed
await page.getByLabel('Username').fill('jane@example.com');
await page.getByLabel('Password').fill('••••••••');
await page.getByRole('button', { name: 'Sign in' }).click();
await expect(page).toHaveURL(/\/dashboard/);
});
```
The Cypress spec runs through the [testingbot-cypress-cli](https://testingbot.com/support/web-automate/cypress); the Playwright test connects to the [wsEndpoint](https://testingbot.com/support/web-automate/playwright). Both appear in the same TestingBot dashboard.
Decision matrix
## When to choose which
### Choose Cypress when
- You want the best in-class developer experience: in-browser runner, time-travel snapshots and readable failures.
- You do component testing alongside end-to-end tests and want one tool for both.
- Your targets are Chrome, Edge and Firefox, with no WebKit or Safari-engine requirement.
- Your team is fully JavaScript or TypeScript and values a gentle learning curve.
- Your flows stay within a single tab and mostly a single origin (cy.origin helps, within limits).
[Run Cypress on TestingBot](https://testingbot.com/features/automation/cypress)
### Choose Playwright when
- You need WebKit coverage to approximate Safari, which Cypress cannot drive.
- Your team writes Python, Java or C# as well as JavaScript and wants one framework across all of them.
- Your tests open multiple tabs, windows or origins, which Playwright handles natively through browser contexts.
- You want free, built-in parallel execution without a paid dashboard subscription.
- You want the fastest CI runs and trace-based debugging with the Playwright Trace Viewer.
[Run Playwright on TestingBot](https://testingbot.com/features/automation/playwright)
TestingBot supports both
## Stop choosing, run both on the same grid
Run Cypress through the testingbot-cypress-cli and connect Playwright through its wsEndpoint. Your tests share the same 6100+ browsers and devices, the same dashboard, the same parallel slots and the same EU data residency, with no Cypress Cloud subscription required.
- Cypress parallelisation without a Cypress Cloud plan
- Side-by-side test history for both frameworks
- Free for open source, both frameworks
[Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo)
testingbot.com / run both
// Cypress
$ npm i -D testingbot-cypress-cli
$ testingbot-cypress run # reads testingbot.json
// Playwright
use: { wsEndpoint: 'wss://cloud.testingbot.com/playwright' }
FAQ
## Frequently Asked Questions
The questions teams ask before picking, or migrating between, these frameworks.
Is Playwright better than Cypress?
Neither is universally better; they fit different teams. Playwright is more flexible: it drives Chromium, Firefox and WebKit, supports five languages, handles multiple tabs and origins natively, and parallelises for free. Cypress is loved for its developer experience: an in-browser runner, time-travel debugging and first-class component testing. Pick Playwright for breadth and scale; pick Cypress for ergonomics within its constraints.
Should I migrate from Cypress to Playwright?
Only if you are hitting Cypress's limits. If you need WebKit (Safari) coverage, more than one language, multi-tab or multi-origin flows, or free parallel execution, Playwright removes those constraints. If your Cypress suite is stable, JavaScript-only and within a single tab, the migration is rarely worth the effort. Many teams keep their Cypress suite and add Playwright for new cross-browser work.
Does Cypress or Playwright support more browsers?
Playwright supports more rendering engines. It drives Chromium (Chrome, Edge), Firefox and WebKit, which approximates Safari. Cypress runs on Chromium-based browsers (Chrome, Edge, Electron) and Firefox, with experimental WebKit support. Neither drives the real Safari binary or Internet Explorer; for those you need [Selenium](https://testingbot.com/features/automation/selenium). TestingBot runs both frameworks across its browser grid.
Which is faster, Cypress or Playwright?
Playwright is usually faster for full suites, mainly because it parallelises across workers for free out of the box and drives the browser out-of-process. Cypress runs each spec in a single browser instance and needs Cypress Cloud or a cloud grid to parallelise. On a single spec the difference is small; across a large suite, parallelisation is the deciding factor. Running either on TestingBot's grid parallelises across browsers regardless.
Can Cypress or Playwright test mobile apps?
Neither tests native mobile apps. Both offer mobile viewport emulation in Chromium, but cannot drive native iOS or Android apps. For real mobile testing you need [Appium](https://testingbot.com/features/automation/appium), [XCUITest](https://testingbot.com/features/automation/xcuitest), [Espresso](https://testingbot.com/features/automation/espresso) or [Maestro](https://testingbot.com/features/automation/maestro). TestingBot runs all of these on real iOS and Android devices.
Do I need Cypress Cloud, or can I parallelise for free?
Playwright parallelises for free with its built-in worker pool. Cypress's own parallelisation and recording run through Cypress Cloud, a paid subscription. TestingBot is an alternative for Cypress: the testingbot-cypress-cli runs your specs in parallel across Chrome, Edge and Firefox on the cloud grid, with live logs and video, and no Cypress Cloud plan. You set the parallel count in testingbot.json.
Can I run Cypress and Playwright on TestingBot?
Yes, both run on the same TestingBot cloud. Cypress runs through the `testingbot-cypress-cli` npm package, configured with a `testingbot.json` file and started with `testingbot-cypress run`. Playwright connects via its `wsEndpoint` to `wss://cloud.testingbot.com/playwright`. Tests from both show up in the same dashboard, share parallel slots, and benefit from the same EU data residency, video recording and CI/CD integrations. Both are free for open source projects.
Going deeper? See the dedicated [Cypress](https://testingbot.com/features/automation/cypress) and [Playwright](https://testingbot.com/features/automation/playwright) pages.
## Related comparisons
[Cypress vs Selenium](https://testingbot.com/cypress-vs-selenium) [Playwright vs Selenium](https://testingbot.com/playwright-vs-selenium)
### Sign up for a Free Trial
Run Cypress, Playwright, or both, on TestingBot's cloud. No Cypress Cloud subscription required.
[Start a free trial](https://testingbot.com/users/sign_up)
Resources you may like
[](https://testingbot.com/resources/articles/how-to-debug-cypress-tests)
### [Tutorial on debugging Cypress tests](https://testingbot.com/resources/articles/how-to-debug-cypress-tests)
How to debug a failing Cypress test using the debugger, console logs, the stack trace, and the debug and pause commands.
[Read more: Tutorial on debugging Cypress tests](https://testingbot.com/resources/articles/how-to-debug-cypress-tests)
[](https://testingbot.com/resources/articles/playwright-visual-regression-testing)
### [Visual Testing with Playwright](https://testingbot.com/resources/articles/playwright-visual-regression-testing)
Playwright provides automated browser testing. It offers a built-in feature to perform visual regression testing for your website.
[Read more: Visual Testing with Playwright](https://testingbot.com/resources/articles/playwright-visual-regression-testing)
[](https://testingbot.com/resources/articles/cypress-cucumber-testing)
### [Cypress and Cucumber Browser Testing](https://testingbot.com/resources/articles/cypress-cucumber-testing)
How to combine Cypress with Cucumber so end-to-end tests read as plain language, including a worked example and the TypeScript setup.
[Read more: Cypress and Cucumber Browser Testing](https://testingbot.com/resources/articles/cypress-cucumber-testing)
---
URL: https://testingbot.com/selenium-vs-puppeteer
Framework comparison · 2026
# Selenium vs Puppeteer
Selenium and Puppeteer solve different problems: one is the cross-browser W3C WebDriver standard, the other is a fast Chrome-focused automation library. This guide compares them head-to-head on protocol, languages, browser coverage, speed and use cases, with code samples and a verdict you can act on in five minutes.
[Run both on TestingBot](https://testingbot.com/users/sign_up) [Jump to comparison](https://testingbot.com#comparison)
- **Browsers & devices:** 6100+
- **Cloud parallelism:** 100×
- **Uptime SLA:** 99.9%
## Trusted by some of the world's most innovative companies
30-second answer
## Which one should you pick?
 Pick Selenium if…
You need real cross-browser testing across Chrome, Firefox, Safari, Edge and Internet Explorer, more than one language, or real mobile devices through Appium, on a W3C-standard protocol.
 Pick Puppeteer if…
You automate Chrome or Chromium from Node.js for fast end-to-end tests, scraping, PDF generation or performance tracing, and you do not need Safari, Internet Explorer or other languages.
Pick both if…
You use Puppeteer for fast Chrome-only checks and Selenium for broad cross-browser coverage. TestingBot runs both on the same grid, in parallel, in one dashboard.
Background
## What are Selenium and Puppeteer?
Two browser automation tools with different goals. One standardises cross-browser control; the other drives Chrome at speed.
### Selenium
Released 2004 · Open Source · Apache 2.0
Selenium is the original browser automation framework and the foundation of the W3C WebDriver standard. Selenium WebDriver drives any browser from the outside through a vendor-supplied driver (chromedriver, geckodriver, safaridriver, edgedriver) using the same wire protocol.
Twenty years of ecosystem maturity means bindings for every major language, deep IDE integration, mature Page Object patterns, and real mobile device testing through Appium, which is built on the same protocol.
- Java / Python / C# / Ruby / JavaScript / Kotlin
- W3C WebDriver standard, works with every browser
- Chrome, Firefox, Safari, Edge, IE 11 · real mobile via Appium
### Puppeteer
Released 2017 · Google · Apache 2.0
Puppeteer is a Node.js library from the Google Chrome team that controls Chrome, Chromium and Edge through the Chrome DevTools Protocol. It powers fast end-to-end testing, web scraping, PDF generation and performance tracing.
Because it talks the DevTools protocol directly instead of WebDriver, Puppeteer is fast and has first-class access to the network, the console and tracing. The same design ties it to Chromium-based browsers and the JavaScript ecosystem.
- JavaScript and TypeScript (Node.js)
- Chrome DevTools Protocol, network and tracing access
- Chrome, Chromium, Edge · Firefox experimental, no Safari/IE
Head to head
## Selenium vs Puppeteer: side-by-side comparison
Across the dimensions that matter for picking, migrating or running both in CI.
Selenium versus Puppeteer feature comparison| Dimension | Selenium | Puppeteer |
| --- | --- | --- |
| First release | 2004 | 2017 |
| Maintained by | Open-source community + W3C | Google (Chrome team) |
| Protocol | W3C WebDriver | Chrome DevTools Protocol |
| Languages | Java, Python, C#, Ruby, JS, Kotlin | JavaScript / TypeScript |
| Browsers | Chrome, Firefox, Safari, Edge, IE 11 | Chrome, Chromium, Edge |
| Firefox / Safari | Both supported | Firefox experimental, no Safari |
| Mobile testing | Real iOS + Android via Appium | None |
| Speed | Mature, predictable | Faster on Chrome (direct CDP) |
| Network interception | BiDi (Selenium 4) or proxy | First-class (CDP) |
| Scraping / PDF / tracing | Possible | First-class |
| Test runner | Bring your own (JUnit, pytest) | Bring your own (Jest, Mocha) |
| Standard | W3C WebDriver standard | Chrome-specific protocol |
| Parallel execution | Selenium Grid | Worker-based (Jest, Mocha) |
| Ecosystem maturity | 20 years of integrations | Strong since 2017 |
| Free for open source on TestingBot | ✓ | ✓ |
Feature notes reflect Selenium 4.x and Puppeteer 23.x as of 2026. Both run on TestingBot's cloud, Selenium through the WebDriver hub and Puppeteer through the browserWSEndpoint connection.
The same test in both
## Logging in, asserting the result
A login flow with an explicit wait (Selenium) and a DevTools-protocol session (Puppeteer). Both run on the same TestingBot grid.
Selenium + Python test\_login.py
```python
# Driver points at TestingBot remote URL
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
def test_login_redirects_to_dashboard(driver):
driver.get('https://app.example.com/login')
driver.find_element(By.NAME, 'username').send_keys('jane@example.com')
driver.find_element(By.NAME, 'password').send_keys('••••••••')
driver.find_element(By.CSS_SELECTOR, 'button[type=submit]').click()
# explicit wait
WebDriverWait(driver, 10).until(EC.url_contains('/dashboard'))
welcome = driver.find_element(By.TAG_NAME, 'h1')
assert 'Welcome, Jane' in welcome.text
```
Puppeteer + Node.js login.test.js
```javascript
// connect to TestingBot over the DevTools protocol
import puppeteer from 'puppeteer-core';
const caps = { key: 'KEY', secret: 'SECRET', browserName: 'chrome', browserVersion: 'latest' };
const browser = await puppeteer.connect({
browserWSEndpoint: `wss://cloud.testingbot.com/puppeteer?capabilities=${encodeURIComponent(JSON.stringify(caps))}`,
});
const page = await browser.newPage();
await page.goto('https://app.example.com/login');
await page.type('#username', 'jane@example.com');
await page.type('#password', '••••••••');
await page.click('button[type=submit]');
await page.waitForSelector('h1');
```
The Selenium test connects to the [WebDriver hub](https://testingbot.com/support/web-automate/selenium); the Puppeteer script connects via [browserWSEndpoint](https://testingbot.com/support/web-automate/puppeteer). Both appear in the same TestingBot dashboard.
Decision matrix
## When to choose which
### Choose Selenium when
- You need true cross-browser coverage, including Safari, Internet Explorer 11 and Edge, not just Chromium.
- Your team writes Java, C#, Ruby or Python and wants full client-library parity.
- You test real iOS and Android devices, where Appium reuses the same WebDriver protocol.
- You want a vendor-neutral, W3C-standard protocol rather than a Chrome-specific one.
- You have a mature Page Object library or BDD framework (Cucumber, SpecFlow) you do not want to rewrite.
[Run Selenium on TestingBot](https://testingbot.com/features/automation/selenium)
### Choose Puppeteer when
- You only need to automate Chrome, Chromium or Edge and want the fastest possible runs.
- Your team is Node.js-first and you want a JavaScript or TypeScript API.
- You do web scraping, PDF generation, screenshots or performance tracing alongside testing.
- You want direct DevTools-protocol access to the network, console and CDP events.
- You do not need Safari, Internet Explorer, real mobile devices or non-JavaScript languages.
[Run Puppeteer on TestingBot](https://testingbot.com/features/automation/puppeteer)
TestingBot supports both
## Stop choosing, run both on the same grid
Point Selenium at the WebDriver hub and connect Puppeteer through its browserWSEndpoint. Your tests share the same 6100+ browsers and devices, the same dashboard, the same parallel slots and the same EU data residency.
- Same auth, same project, same billing
- Side-by-side test history for both tools
- Free for open source, both tools
[Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo)
testingbot.com / run both
# Selenium
webdriver.Remote(
command\_executor='https://hub.testingbot.com/wd/hub'
)
// Puppeteer
puppeteer.connect({
browserWSEndpoint:
'wss://cloud.testingbot.com/puppeteer'
})
FAQ
## Frequently Asked Questions
The questions teams ask before picking, or combining, these tools.
Is Puppeteer faster than Selenium?
Usually yes, on Chrome. Puppeteer talks the Chrome DevTools Protocol directly, avoiding the HTTP round-trips of the WebDriver protocol, so Chrome-only suites often run faster. But the gain only applies to Chromium-based browsers, and it narrows once tests are network-bound. Selenium 4's BiDi support also closes the gap. Speed is real but rarely the deciding factor against Selenium's cross-browser reach.
Can Puppeteer replace Selenium?
Only for Chrome-focused work. Puppeteer is excellent for automating Chrome, Chromium and Edge from Node.js, but it cannot drive Safari or Internet Explorer, has no real mobile support, and is JavaScript-only. If you need cross-browser testing, multiple languages or real devices, Selenium remains the broader tool. Many teams use Puppeteer for fast Chrome checks and Selenium for full cross-browser coverage.
Does Puppeteer support browsers other than Chrome?
Puppeteer drives Chrome, Chromium and Chromium-based Edge, with experimental Firefox support. It cannot drive Safari or Internet Explorer. Selenium drives the real binaries of every major browser through vendor drivers, including Safari and Internet Explorer 11. If you need anything beyond Chromium, Selenium is the option of the two. TestingBot runs both across its browser grid.
Can Puppeteer test mobile apps?
No. Puppeteer offers mobile viewport emulation in Chromium, but it cannot drive native iOS or Android apps. For real mobile testing you need [Appium](https://testingbot.com/features/automation/appium), [XCUITest](https://testingbot.com/features/automation/xcuitest), [Espresso](https://testingbot.com/features/automation/espresso) or [Maestro](https://testingbot.com/features/automation/maestro). Appium reuses the WebDriver protocol Selenium is built on. TestingBot runs all of these on real iOS and Android devices.
Is Puppeteer or Selenium better for web scraping?
Puppeteer is the more common choice for scraping, PDF generation and headless Chrome automation, thanks to its direct DevTools access and Node.js ecosystem. Selenium can scrape too, but its strength is cross-browser test automation across many languages. Pick the tool that matches the job: Puppeteer for Chrome automation tasks, Selenium for standards-based cross-browser testing.
Can I run Puppeteer in parallel on TestingBot?
Yes. Puppeteer's worker-based parallelism (via Jest, Mocha or a custom runner) works against the TestingBot grid: each worker opens its own remote browser over the WebSocket endpoint. Concurrency is capped by your plan. Selenium parallelises the same way through the WebDriver hub.
Can I run Selenium and Puppeteer on TestingBot?
Yes, both run on the same TestingBot cloud. Selenium connects to `https://hub.testingbot.com/wd/hub` via `webdriver.Remote`. Puppeteer connects with `puppeteer.connect` using a `browserWSEndpoint` of `wss://cloud.testingbot.com/puppeteer`. Tests from both show up in the same dashboard, share parallel slots, and benefit from the same EU data residency, video recording and CI/CD integrations. Both are free for open source projects.
Going deeper? See the dedicated [Selenium](https://testingbot.com/features/automation/selenium) and [Puppeteer](https://testingbot.com/features/automation/puppeteer) pages.
## Related comparisons
[Playwright vs Selenium](https://testingbot.com/playwright-vs-selenium) [Cypress vs Playwright](https://testingbot.com/cypress-vs-playwright)
### Sign up for a Free Trial
Run Selenium, Puppeteer, or both, on TestingBot's cloud.
[Start a free trial](https://testingbot.com/users/sign_up)
Resources you may like
[](https://testingbot.com/resources/articles/generative-ai-selenium)
### [Selenium and Generative AI](https://testingbot.com/resources/articles/generative-ai-selenium)
How to use generative AI to produce realistic test data for Selenium runs, and why varied data uncovers bugs that fixed fixtures never will.
[Read more: Selenium and Generative AI](https://testingbot.com/resources/articles/generative-ai-selenium)
[](https://testingbot.com/resources/articles/automated-testing-with-puppeteer)
### [Automated Testing with Puppeteer](https://testingbot.com/resources/articles/automated-testing-with-puppeteer)
Puppeteer combined with a test framework provides a great way to run automated browser tests. Follow this guide for more information.
[Read more: Automated Testing with Puppeteer](https://testingbot.com/resources/articles/automated-testing-with-puppeteer)
[](https://testingbot.com/resources/articles/playwright-visual-regression-testing)
### [Visual Testing with Playwright](https://testingbot.com/resources/articles/playwright-visual-regression-testing)
Playwright provides automated browser testing. It offers a built-in feature to perform visual regression testing for your website.
[Read more: Visual Testing with Playwright](https://testingbot.com/resources/articles/playwright-visual-regression-testing)
---
URL: https://testingbot.com/appium-vs-espresso
Framework comparison · 2026
# Appium vs Espresso
Appium and Espresso are the two leading ways to automate Android UI tests. Appium is a cross-platform, black-box framework on the W3C WebDriver protocol; Espresso is Google's fast, white-box framework for Android only. This guide compares them head-to-head, with code samples and a verdict you can act on in five minutes.
[Run both on TestingBot](https://testingbot.com/users/sign_up) [Jump to comparison](https://testingbot.com#comparison)
- **Browsers & devices:** 6100+
- **Cloud parallelism:** 100×
- **Uptime SLA:** 99.9%
## Trusted by some of the world's most innovative companies
30-second answer
## Which one should you pick?
 Pick Appium if…
You want one test suite that runs on both Android and iOS, your team works in Java, Python, JavaScript, Ruby or C#, or you test apps you cannot rebuild with instrumentation.
 Pick Espresso if…
You test Android only, you own the app source, and you want the fastest, least flaky UI tests with automatic synchronisation, written in Kotlin or Java alongside your app.
Pick both if…
You use Espresso for fast Android-only component and UI tests and Appium for cross-platform end-to-end coverage. TestingBot runs both on the same real-device grid, in parallel.
Background
## What are Appium and Espresso?
Two mobile UI automation frameworks with opposite designs. One is cross-platform and black-box; the other is Android-native and white-box.
### Appium
Released 2012 · OpenJS Foundation · Apache 2.0
Appium is an open-source, cross-platform test framework for native, hybrid and mobile-web apps. It implements the W3C WebDriver protocol, so the same client library drives both iOS and Android from Java, Python, Node.js, Ruby, C# and more.
Appium drives the app from the outside as a black box, so it needs no app source code. On Android it uses the UiAutomator2 or Espresso driver under the hood; on iOS it uses XCUITest. That layer is what makes one test reusable across platforms.
- Java / Python / Node.js / Ruby / C# / PHP
- Cross-platform, native, hybrid and mobile web
- Black-box, no app source required
### Espresso
Released 2013 · Google · Apache 2.0
Espresso is Google's UI testing framework for Android, part of the Android Jetpack test libraries. Tests run in-process on the device, in the same process as the app under test, which gives them direct access to the app and automatic synchronisation with the UI thread.
Because Espresso runs inside the app process, it is fast and reliable, with very little flakiness. The trade-offs: it is Android only, it is written in Kotlin or Java, and it needs an instrumented build of the app.
- Kotlin and Java
- In-process, automatic UI synchronisation
- Android only · needs an instrumented build
Head to head
## Appium vs Espresso: side-by-side comparison
Across the dimensions that matter for Android UI testing and cross-platform coverage.
Appium versus Espresso feature comparison| Dimension | Appium | Espresso |
| --- | --- | --- |
| First release | 2012 | 2013 |
| Maintained by | Open source (OpenJS Foundation) | Google |
| Platforms | iOS + Android | Android only |
| Languages | Java, Python, JS, Ruby, C#, PHP | Java, Kotlin |
| Protocol | W3C WebDriver | Android instrumentation |
| Test execution | Out-of-process (black-box) | In-process (white-box) |
| Speed | Slower (WebDriver round-trips) | Very fast (in-process) |
| App source needed | No (black-box) | Yes (instrumented build) |
| UI synchronization | Manual waits | Automatic, built in |
| Cross-platform reuse | Same test on iOS + Android | Android only |
| Web / hybrid apps | Yes | Limited |
| Flakiness | Good | Excellent (tight coupling) |
| On TestingBot | Upload app, WebDriver hub | Upload app + test, CLI |
| Free for open source on TestingBot | ✓ | ✓ |
Both run on TestingBot's real Android devices and emulators. Appium connects to the WebDriver hub with an uploaded app; Espresso runs through the TestingBot CLI.
The same login test
## Logging in on Android
A login flow driven black-box by Appium and white-box by Espresso. Both run on the same real Android devices on TestingBot.
Appium + Python test\_login.py
```python
# real Android device on TestingBot
from appium import webdriver
from appium.options.android import UiAutomator2Options
from appium.webdriver.common.appiumby import AppiumBy
opts = UiAutomator2Options().load_capabilities({
'platformName': 'Android',
'appium:app': 'tb://APP_ID',
'tb:options': { 'realDevice': True },
})
d = webdriver.Remote('https://hub.testingbot.com/wd/hub', options=opts)
d.find_element(AppiumBy.ACCESSIBILITY_ID, 'username').send_keys('jane')
d.find_element(AppiumBy.ACCESSIBILITY_ID, 'sign_in').click()
assert d.find_element(AppiumBy.ID, 'welcome').is_displayed()
```
Espresso + Kotlin LoginTest.kt
```kotlin
// runs via the TestingBot CLI
@Test
fun loginRedirectsToDashboard() {
onView(withId(R.id.username))
.perform(typeText("jane@example.com"))
onView(withId(R.id.password))
.perform(typeText("secret"))
onView(withId(R.id.sign_in))
.perform(click())
onView(withId(R.id.welcome))
.check(matches(isDisplayed()))
}
```
The Appium test connects to the [WebDriver hub](https://testingbot.com/support/app-automate/appium) with an uploaded app; the Espresso suite runs through the [TestingBot CLI](https://testingbot.com/support/app-automate/espresso). Both run on real Android devices.
Decision matrix
## When to choose which
### Choose Appium when
- You need one test suite that runs on both Android and iOS without a rewrite.
- Your team writes Java, Python, JavaScript, Ruby or C# rather than Kotlin.
- You test an app you cannot rebuild with instrumentation, or a third-party or hybrid app.
- You already use Selenium or WebDriver and want the same protocol for mobile.
- You test native, hybrid and mobile-web flows with one tool.
[Run Appium on TestingBot](https://testingbot.com/features/automation/appium)
### Choose Espresso when
- You test Android only and own the app source, so an instrumented build is fine.
- You want the fastest, least flaky UI tests with automatic synchronisation.
- Your team writes Kotlin or Java and wants tests next to the app code.
- You test internal app state, custom views or components, not just the public UI.
- You run UI tests as part of the Android build and CI from day one.
[Run Espresso on TestingBot](https://testingbot.com/features/automation/espresso)
TestingBot supports both
## Stop choosing, run both on the same grid
Upload your app and drive it black-box with Appium over the WebDriver hub, or run your Espresso suite through the TestingBot CLI. Both run on the same real Android devices, with the same dashboard, parallel slots, video recording and EU data residency.
- Real Android devices and emulators
- Side-by-side test history for both frameworks
- Free for open source, both frameworks
[Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo)
testingbot.com / run both
# Appium
appium:app = 'tb://APP\_ID'
hub: https://hub.testingbot.com/wd/hub
# Espresso (@testingbot/cli)
$ testingbot espresso app.apk app-test.apk
--device "Galaxy S24" --real-device
FAQ
## Frequently Asked Questions
The questions teams ask before picking, or combining, these frameworks.
Is Espresso faster than Appium?
Yes, on Android. Espresso runs in-process, in the same process as the app, so it avoids the network round-trips of Appium's WebDriver protocol and synchronises automatically with the UI thread. That makes Espresso faster and less flaky. The trade-off is that Espresso is Android only and needs an instrumented build, while Appium runs the same test across Android and iOS as a black box.
Should I use Appium or Espresso for Android?
It depends on your goals. For Android-only UI tests where you own the source, Espresso gives the fastest, most reliable runs. For cross-platform coverage, tests in a non-Kotlin language, or apps you cannot instrument, Appium is the better fit. Many Android teams run Espresso for component and UI tests and Appium for cross-platform end-to-end flows.
Can Espresso test iOS apps?
No. Espresso is an Android-only framework from Google. To test iOS you need [XCUITest](https://testingbot.com/features/automation/xcuitest) (Apple's native framework) or [Appium](https://testingbot.com/features/automation/appium), which drives iOS through XCUITest under the hood. Appium is the only one of these that runs the same test on both platforms. TestingBot runs all of them on real devices.
Can I reuse the same tests across Android and iOS?
With Appium, yes. Because Appium implements the W3C WebDriver protocol and abstracts the native drivers (UiAutomator2 or Espresso on Android, XCUITest on iOS), a single test can target both platforms with minor capability changes. Espresso cannot do this; it is Android only. If cross-platform reuse matters, Appium is the framework to choose.
Does Espresso need the app source code?
Yes. Espresso compiles into an instrumented test APK that runs alongside your app, so you need the app source or at least a debuggable, instrumented build. Appium drives the installed app as a black box and needs no source, which is why it can test third-party or already-published apps that Espresso cannot.
Can I run Appium and Espresso on real devices in the cloud?
Yes. TestingBot provides real Android phones and tablets, plus emulators, in an EU datacenter. Appium tests upload your .apk to TestingBot Storage and run against the hub with realDevice set; Espresso tests upload the app and test APKs through the TestingBot CLI. Both record video and logs you can review in the dashboard.
Can I run Appium and Espresso on TestingBot?
Yes, both run on the same TestingBot grid. Appium uploads your app to TestingBot Storage, then connects to `https://hub.testingbot.com/wd/hub` with an `appium:app` capability of `tb://APP_ID`. Espresso runs through the TestingBot CLI: `testingbot espresso app.apk app-test.apk --device "Galaxy S24" --real-device`. Both share the same dashboard, parallel slots and EU data residency, and are free for open source.
Going deeper? See the dedicated [Appium](https://testingbot.com/features/automation/appium) and [Espresso](https://testingbot.com/features/automation/espresso) pages.
## Related comparisons
[Appium vs XCUITest](https://testingbot.com/appium-vs-xcuitest) [Playwright vs Selenium](https://testingbot.com/playwright-vs-selenium)
### Sign up for a Free Trial
Run Appium and Espresso on real Android devices in TestingBot's cloud.
[Start a free trial](https://testingbot.com/users/sign_up)
Resources you may like
[](https://testingbot.com/resources/articles/touch-actions-appium)
### [Using Touch Actions with Appium](https://testingbot.com/resources/articles/touch-actions-appium)
How to simulate taps, swipes and pinches on real devices with Appium Touch Actions, with examples, best practices and common pitfalls.
[Read more: Using Touch Actions with Appium](https://testingbot.com/resources/articles/touch-actions-appium)
[](https://testingbot.com/resources/articles/android-espresso-testing)
### [Android Espresso Tutorial](https://testingbot.com/resources/articles/android-espresso-testing)
Why Espresso suits Android UI testing, how to add it to a project and write a first test, and how it compares with the alternatives.
[Read more: Android Espresso Tutorial](https://testingbot.com/resources/articles/android-espresso-testing)
[](https://testingbot.com/resources/articles/appium-2-migration)
### [Appium 2 and Appium 3 Migration Guide](https://testingbot.com/resources/articles/appium-2-migration)
What changed between Appium 1, 2 and 3: drivers, capabilities, the removed touch endpoints, and how to upgrade an existing suite.
[Read more: Appium 2 and Appium 3 Migration Guide](https://testingbot.com/resources/articles/appium-2-migration)
---
URL: https://testingbot.com/appium-vs-xcuitest
Framework comparison · 2026
# Appium vs XCUITest
Appium and XCUITest are the two leading ways to automate iOS UI tests. Appium is a cross-platform, black-box framework on the W3C WebDriver protocol; XCUITest is Apple's fast, native framework for iOS only. This guide compares them head-to-head, with code samples and a verdict you can act on in five minutes.
[Run both on TestingBot](https://testingbot.com/users/sign_up) [Jump to comparison](https://testingbot.com#comparison)
- **Browsers & devices:** 6100+
- **Cloud parallelism:** 100×
- **Uptime SLA:** 99.9%
## Trusted by some of the world's most innovative companies
30-second answer
## Which one should you pick?
 Pick Appium if…
You want one test suite that runs on both iOS and Android, your team works in Java, Python, JavaScript, Ruby or C#, or you test apps you cannot rebuild with a test target.
 Pick XCUITest if…
You test iOS only, you own the app in Xcode, and you want the fastest, most reliable native UI tests written in Swift or Objective-C alongside your app.
Pick both if…
You use XCUITest for fast iOS-only native tests and Appium for cross-platform end-to-end coverage. TestingBot runs both on the same real-device grid, in parallel.
Background
## What are Appium and XCUITest?
Two iOS UI automation frameworks with opposite designs. One is cross-platform and black-box; the other is Apple-native and tightly coupled to Xcode.
### Appium
Released 2012 · OpenJS Foundation · Apache 2.0
Appium is an open-source, cross-platform test framework for native, hybrid and mobile-web apps. It implements the W3C WebDriver protocol, so the same client library drives both iOS and Android from Java, Python, Node.js, Ruby, C# and more.
Appium drives the app from the outside as a black box, so it needs no app source code. On iOS it uses Apple's XCUITest under the hood (through WebDriverAgent); on Android it uses UiAutomator2. That layer is what makes one test reusable across platforms.
- Java / Python / Node.js / Ruby / C# / PHP
- Cross-platform, native, hybrid and mobile web
- Black-box, no app source required
### XCUITest
Released 2015 · Apple · part of Xcode
XCUITest is Apple's UI testing framework, part of the larger XCTest framework that ships with Xcode. Tests are written in Swift or Objective-C and run on-device through XCTest, with first-class access to the iOS accessibility tree.
Because XCUITest is native and maintained by Apple, it is fast, reliable and always current with the latest iOS. The trade-offs: it is iOS only, it is written in Swift or Objective-C, and it needs an Xcode test target.
- Swift and Objective-C
- Native XCTest, accessibility-tree access
- iOS only · needs an Xcode test target
Head to head
## Appium vs XCUITest: side-by-side comparison
Across the dimensions that matter for iOS UI testing and cross-platform coverage.
Appium versus XCUITest feature comparison| Dimension | Appium | XCUITest |
| --- | --- | --- |
| First release | 2012 | 2015 |
| Maintained by | Open source (OpenJS Foundation) | Apple |
| Platforms | iOS + Android | iOS only |
| Languages | Java, Python, JS, Ruby, C#, PHP | Swift, Objective-C |
| Protocol | W3C WebDriver | XCTest / XCUITest (native) |
| iOS engine | XCUITest via WebDriverAgent | XCUITest (direct) |
| Speed | Slower (extra layers) | Faster (native) |
| App source needed | No (black-box) | Yes (Xcode test target) |
| Tooling | Any IDE / CI | Xcode |
| Cross-platform reuse | Same test on iOS + Android | iOS only |
| Web / hybrid apps | Yes | Limited |
| Flakiness | Good | Excellent (native sync) |
| On TestingBot | Upload app, WebDriver hub | Upload app + test zip, CLI |
| Free for open source on TestingBot | ✓ | ✓ |
Both run on TestingBot's real iPhones and iPads. Appium connects to the WebDriver hub with an uploaded app; XCUITest runs through the TestingBot CLI.
The same login test
## Logging in on iOS
A login flow driven black-box by Appium and natively by XCUITest. Both run on the same real iPhones on TestingBot.
Appium + Python test\_login.py
```python
# real iPhone on TestingBot
from appium import webdriver
from appium.options.ios import XCUITestOptions
from appium.webdriver.common.appiumby import AppiumBy
opts = XCUITestOptions().load_capabilities({
'platformName': 'iOS',
'appium:app': 'tb://APP_ID',
'tb:options': { 'realDevice': True },
})
d = webdriver.Remote('https://hub.testingbot.com/wd/hub', options=opts)
d.find_element(AppiumBy.ACCESSIBILITY_ID, 'username').send_keys('jane')
d.find_element(AppiumBy.ACCESSIBILITY_ID, 'sign_in').click()
assert d.find_element(AppiumBy.ACCESSIBILITY_ID, 'welcome').is_displayed()
```
XCUITest + Swift LoginTests.swift
```swift
// runs via the TestingBot CLI
func testLoginRedirectsToDashboard() {
let app = XCUIApplication()
app.launch()
app.textFields["username"].typeText("jane@example.com")
app.secureTextFields["password"].typeText("secret")
app.buttons["sign_in"].tap()
XCTAssertTrue(app.staticTexts["welcome"].exists)
}
```
The Appium test connects to the [WebDriver hub](https://testingbot.com/support/app-automate/appium) with an uploaded app; the XCUITest suite runs through the [TestingBot CLI](https://testingbot.com/support/app-automate/xcuitest). Both run on real iPhones and iPads.
Decision matrix
## When to choose which
### Choose Appium when
- You need one test suite that runs on both iOS and Android without a rewrite.
- Your team writes Java, Python, JavaScript, Ruby or C# rather than Swift.
- You test an app you cannot rebuild with a test target, or a third-party or hybrid app.
- You already use Selenium or WebDriver and want the same protocol for mobile.
- You test native, hybrid and mobile-web flows with one tool.
[Run Appium on TestingBot](https://testingbot.com/features/automation/appium)
### Choose XCUITest when
- You test iOS only and own the app in Xcode, so a test target is fine.
- You want the fastest, most reliable native iOS tests that track the latest iOS.
- Your team writes Swift or Objective-C and wants tests next to the app code.
- You need deep access to the iOS accessibility tree and native components.
- You run UI tests as part of the Xcode build and CI from day one.
[Run XCUITest on TestingBot](https://testingbot.com/features/automation/xcuitest)
TestingBot supports both
## Stop choosing, run both on the same grid
Upload your app and drive it black-box with Appium over the WebDriver hub, or run your XCUITest suite through the TestingBot CLI. Both run on the same real iPhones and iPads, with the same dashboard, parallel slots, video recording and EU data residency.
- Real iPhones and iPads
- Side-by-side test history for both frameworks
- Free for open source, both frameworks
[Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo)
testingbot.com / run both
# Appium
appium:app = 'tb://APP\_ID'
hub: https://hub.testingbot.com/wd/hub
# XCUITest (@testingbot/cli)
$ testingbot xcuitest app.ipa app-test.zip
--device "iPhone 16" --real-device
FAQ
## Frequently Asked Questions
The questions teams ask before picking, or combining, these frameworks.
Is XCUITest faster than Appium?
Yes, on iOS. XCUITest is Apple's native framework and runs on-device through XCTest, so it avoids the extra WebDriver and WebDriverAgent layers Appium adds. That makes XCUITest faster and very reliable. The trade-off is that XCUITest is iOS only and written in Swift or Objective-C, while Appium runs the same test across iOS and Android in the language of your choice.
Should I use Appium or XCUITest for iOS?
It depends on your goals. For iOS-only tests where you own the Xcode project, XCUITest gives the fastest, most reliable native runs. For cross-platform coverage, tests in a non-Swift language, or apps you cannot rebuild, Appium is the better fit. Note that Appium drives iOS through XCUITest under the hood, so you are choosing the abstraction level, not a different engine.
Does Appium use XCUITest internally?
Yes. On iOS, Appium drives the app through Apple's XCUITest framework using a component called WebDriverAgent. So an Appium iOS test ultimately runs on top of XCUITest, with Appium adding the cross-platform WebDriver API on top. That is why Appium can offer the same test interface for iOS and Android while still using each platform's native automation underneath.
Can XCUITest test Android apps?
No. XCUITest is an iOS-only framework from Apple. To test Android you need [Espresso](https://testingbot.com/features/automation/espresso) (Google's native framework) or [Appium](https://testingbot.com/features/automation/appium), which drives Android through UiAutomator2. Appium is the only one of these that runs the same test on both platforms. TestingBot runs all of them on real devices.
Can I reuse the same tests across iOS and Android?
With Appium, yes. Because Appium implements the W3C WebDriver protocol and abstracts the native drivers (XCUITest on iOS, UiAutomator2 on Android), a single test can target both platforms with minor capability changes. XCUITest cannot do this; it is iOS only. If cross-platform reuse matters, Appium is the framework to choose.
Do I need a Mac and Xcode for XCUITest?
You need a Mac with Xcode to write and build XCUITest tests locally. To run them at scale you do not need your own Mac fleet: build the app .ipa and the XCUITest .zip once, then upload both to TestingBot and run on real iPhones and iPads in the cloud. Appium tests do not require Xcode on your side at all.
Can I run Appium and XCUITest on TestingBot?
Yes, both run on the same TestingBot grid. Appium uploads your app to TestingBot Storage, then connects to `https://hub.testingbot.com/wd/hub` with an `appium:app` capability of `tb://APP_ID`. XCUITest runs through the TestingBot CLI: `testingbot xcuitest app.ipa app-test.zip --device "iPhone 16" --real-device`. Both share the same dashboard, parallel slots and EU data residency, and are free for open source.
Going deeper? See the dedicated [Appium](https://testingbot.com/features/automation/appium) and [XCUITest](https://testingbot.com/features/automation/xcuitest) pages.
## Related comparisons
[Appium vs Espresso](https://testingbot.com/appium-vs-espresso) [Playwright vs Selenium](https://testingbot.com/playwright-vs-selenium)
### Sign up for a Free Trial
Run Appium and XCUITest on real iPhones and iPads in TestingBot's cloud.
[Start a free trial](https://testingbot.com/users/sign_up)
Resources you may like
[](https://testingbot.com/resources/articles/touch-actions-appium)
### [Using Touch Actions with Appium](https://testingbot.com/resources/articles/touch-actions-appium)
How to simulate taps, swipes and pinches on real devices with Appium Touch Actions, with examples, best practices and common pitfalls.
[Read more: Using Touch Actions with Appium](https://testingbot.com/resources/articles/touch-actions-appium)
[](https://testingbot.com/resources/articles/automate-native-ios-apps-xcuitest)
### [Automate native iOS Apps with XCUITest](https://testingbot.com/resources/articles/automate-native-ios-apps-xcuitest)
How to automate Apple's own iOS apps with XCUITest, with code for driving Settings, Messages and Photos, and how to extend it to other apps.
[Read more: Automate native iOS Apps with XCUITest](https://testingbot.com/resources/articles/automate-native-ios-apps-xcuitest)
[](https://testingbot.com/resources/articles/appium-2-migration)
### [Appium 2 and Appium 3 Migration Guide](https://testingbot.com/resources/articles/appium-2-migration)
What changed between Appium 1, 2 and 3: drivers, capabilities, the removed touch endpoints, and how to upgrade an existing suite.
[Read more: Appium 2 and Appium 3 Migration Guide](https://testingbot.com/resources/articles/appium-2-migration)
---
URL: https://testingbot.com/pricing
# TestingBot Pricing
TestingBot offers four cross-browser and mobile testing plans (Live, Automated, Pro, Enterprise) plus a flexible Pay-As-You-Go option. All plans include access to 6100+ real browsers and devices, video recordings, screenshots, local testing, and 24/7 support. Free plan with 100 testing minutes, no credit card required. Full-access 7-day trial with a card.
All amounts on this page are in USD. Prices exclude VAT. EU customers pay 21% VAT unless they enter a valid VAT number from another EU country at checkout. Customers outside the EU pay no VAT.
Subscription prices are shown
both as the monthly equivalent of an annual plan and as the month-to-month rate.
A parallel test (also called a concurrent session) is one test running at a
time, so a 4-parallel plan runs 4 tests simultaneously.
## Live
Manual testing of websites and apps. Seats are the unit here, not
minutes: live testing is unlimited on every tier.
| Users | Annual plan (per month) | Month to month |
| ----- | ----------------------- | -------------- |
| 1 | $20 | $30 |
| 2 | $40 | $60 |
| 4 | $80 | $120 |
| 10 | $200 | $300 |
| 25 | $500 | $750 |
Includes unlimited live testing, unlimited visual testing, real iOS and Android
devices, and 24x7 support by email, chat and Slack.
## Automated
Automated web and app testing. Priced per parallel test, with a
monthly allowance of automated testing minutes.
| Parallel tests | Automated minutes / month | Annual plan (per month) | Month to month |
| -------------- | ------------------------- | ----------------------- | -------------- |
| 1 | 1,000 | $50 | $70 |
| 2 | 2,000 | $120 | $140 |
| 4 | 4,000 | $240 | $280 |
| 8 | 6,000 | $350 | $560 |
| 16 | 10,000 | $700 | $1120 |
| 24 | 20,000 | $1200 | $1680 |
Live and visual testing are unlimited on every tier; only automated testing
draws down the minute allowance. Unused minutes do not roll over.
## Automated Pro
Unlimited web and app testing. Same parallelism as
Automated, with no cap on automated testing minutes.
| Parallel tests | Automated minutes / month | Annual plan (per month) | Month to month |
| -------------- | ------------------------- | ----------------------- | -------------- |
| 1 | Unlimited | $90 | $120 |
| 2 | Unlimited | $180 | $240 |
| 4 | Unlimited | $360 | $480 |
| 8 | Unlimited | $720 | $960 |
| 16 | Unlimited | $1440 | $1920 |
| 24 | Unlimited | $2160 | $2880 |
## Enterprise
Enterprise-grade testing for large teams. Starts at
10+ Parallel Tests, priced per quote.
For Premium features & Enterprise-grade security: unlimited testing, Single Sign-On
(SAML), priority support, and a private device cloud.
See https://testingbot.com/enterprise or contact sales at
https://testingbot.com/contact/new.
## Pay As You Go
Credit bundles for teams that do not want a monthly commitment. Bought once,
never expire, and can be topped up automatically when credits run low.
| Bundle | Testing minutes | Price (one-time) | Parallel sessions | Cost per minute |
| ------ | --------------- | ---------------- | ----------------- | --------------- |
| Starter | 1,000 | $60 | 1 | $0.06 |
| Growth | 2,000 | $120 | 2 | $0.06 |
| Boost | 5,000 | $250 | 5 | $0.05 |
| Professional | 15,000 | $600 | 10 | $0.04 |
Pay As You Go covers automated and live manual testing on all desktop browsers,
real iOS and Android devices, and iOS and Android simulators and emulators.
Video recordings, local testing and 24/7 support are included.
## All plans include
- Over 6100+ real desktop browsers and physical mobile devices
- Pristine VMs with real browsers, and a range of display resolutions
- Video recordings and screenshots of every test
- Local testing (tunnel) for sites behind a firewall
- Team management, and an open REST API for CI/CD
- Android and iOS native app testing
- Tech support and documentation
## Frequently asked questions
### Do I need a credit card to register?
No, you do not need a credit card. Our 14-day free plan lets you try the service without paying anything. When you are ready to upgrade you can pay by credit card or wire transfer.
### Can I cancel my subscription at any time?
Yes. Subscriptions can be cancelled at any time. If you cancel mid-month you will be charged for that month.
### What is a parallel test?
Parallel tests are how many tests you can run at the same time. More parallel tests means a shorter total test run. Use our [Parallel Test Calculator](https://testingbot.com/support/parallel-calculator) to estimate the right number for your suite.
### Do you have a free open-source plan?
Yes, open-source projects can apply for a free TestingBot plan. [Sign up for our open-source plan](https://testingbot.com/open-source).
### What happens if I exceed my monthly testing time?
Once you reach your monthly testing time you can no longer run tests until your next billing month, or you can upgrade to a larger plan.
### What forms of payment do you accept?
We accept credit cards and wire transfers. EU customers also have access to additional payment options. [Contact us](https://testingbot.com/contact/new) for wire transfer.
### Can I get a private cloud or on-prem deployment?
Our Enterprise plan supports private cloud. We do not yet support on-premise deployments. [See enterprise features](https://testingbot.com/enterprise) or contact us for a custom quote.
### Do you offer high-volume packages?
Yes, for very large test suites we offer custom high-volume packages with discounted per-minute rates. [Contact us](https://testingbot.com/contact/new) with your expected monthly volume.
### How is device time calculated?
Device usage is counted from the time your app or browser is started on the device, until you stop the test. We do not charge for device startup, cleaning or preparation. For example: if you you run 1 test for 20 minutes, or 5 tests for 4 minutes, both times 20 minutes will be deducted from your plan.
### Do unused device minutes roll over to the next billing month?
Unused minutes do not roll over to your next billing month.
At the start of your next billing month, the minutes for your account are set to their default value.
### What happens when I upgrade or downgrade?
If you decide to upgrade your account, the upgrade will take effect immediately. The unused portion of your current plan is automatically credited towards the new plan, so you only pay the difference. Downgrades take effect on the next billing cycle.
## Links
- Pricing page: https://testingbot.com/pricing
- Enterprise features: https://testingbot.com/enterprise
- Free plan for open-source projects: https://testingbot.com/open-source
- Parallel test calculator: https://testingbot.com/support/parallel-calculator
- Contact sales: https://testingbot.com/contact/new
---
URL: https://testingbot.com/newsletter
# The TestingBot newsletter
New browsers and devices as they land in the cloud, product changes worth knowing about, and the occasional testing tip. Written by the people who build it.
- A few emails a year, never a drip campaign
- One click to stop, from every email
- Your address is never shared or sold
## Recently
- Oct 21 Linear Integration
- Oct 13 New TestingBot Tunnel 5.0
- Oct 6 Mobilewright tests on the TestingBot device cloud
[Let me read some first](https://testingbot.com/product-updates) · [RSS](https://testingbot.com/product-updates.atom)
## Subscribe
We will send one email to check the address is yours. Nothing else happens until you click the link in it.
Email address
Subscribe
By subscribing you agree that we may email you about TestingBot. We record the date, source and IP of your confirmation, as we are required to. See our [privacy policy](https://testingbot.com/privacy).
---
URL: https://testingbot.com/enterprise
Enterprise plan
# Test at enterprise scale, secured for the way you ship.
TestingBot powers cross-browser, mobile and AI-driven testing for global engineering teams. Single-use firewalled VMs, SSO, auto-provisioning and a private device cloud, on European-hosted infrastructure with a 99.9% uptime SLA.
[Contact Sales](https://testingbot.com/contact/new) [Book a demo](https://testingbot.com/demo)
- **Browsers & devices:** 6100+
- **Uptime SLA:** 99.9%
- **Years in business:** 14
Enterprise features
1. Priority SLA
2. Security
3. Data privacy
4. Private devices
Priority support · 24/7 SLA
Response \< 1 hour
Uptime SLA 99.9%
Channels Slack · Chat · Email
SK
Customer Success Manager
INC-2917 · acknowledged in 2 min
custom SLA · dedicated CSM · custom legal terms
Your network 10.0.0.0/16
 TestingBot grid
TLS 1.3 IP whitelisted Tunnel up
Data residency · EU
session-2917.mp4
12.4 MB
console.log
238 KB
screenshots/\*.png
4.1 MB
encrypted at rest · per-test retention
Private device cloud
Reserved iPhone 17 iOS 26
Reserved iPad Pro iPadOS 18
Reserved Galaxy S26 Android 16
dedicated to your team · never shared
Custom SLA · 24/7 priority support · dedicated CSM Encrypted tunnel · IP whitelisting · inbound + outbound European-hosted · GDPR · per-test retention windows Reserved exclusively for your team, never shared
Trusted by engineering teams worldwide
Built for security & scale
## Everything large engineering teams need from a testing cloud.
Enterprise-grade controls layered on top of every TestingBot session: security, identity, governance and uninterrupted scale.
- ### Single-Use VMs
Every test runs on a fresh, isolated, firewalled VM that is wiped immediately after the session. Customize the OS or pre-installed software per run.
- ### Network Security
IP whitelisting (inbound and outbound) and the encrypted TestingBot Tunnel keep test traffic between your private network and our grid fully secured.
- ### Data Control & Privacy
Disable logging or recording per test. Pick exact data retention windows per asset type. All test artefacts stay on European-hosted infrastructure.
- ### SSO & 2FA
SAML 2.0 SSO with Okta, Azure AD, OneLogin and any custom SAML provider. Enforce 2FA across your entire organisation.
- ### Auto-provisioning
Onboard and offboard team members through your Identity Provider. Roles, teams and product access stay in sync automatically.
- ### Sub-teams & Quotas
Control access with role-based permissions (separate IAM and RBAC roles plus custom roles).
Compliance & assurance
## Built and operated to standards procurement teams already trust.
EU-only data residency, a published CSA STAR self-assessment and a public Trust Center keep your security review on the fast track.
[Trust Center](https://trust.testingbot.com)
- 
CSA STAR Level 1
Cloud Security Alliance
- 
GDPR
EU data protection
- 
CCPA
California privacy
- ISO
ISO 27001
Aligned controls
- CSA
CSA STAR Level 1
Published self-assessment (CAIQ)
- European-hosted
Dedicated devices
## Private Device Cloud
With Private Device Cloud, we take care of the setup, configuration and maintenance of dedicated devices exclusively for your company.
- Unlike our Public Device Cloud, these devices are reserved only for your team, ensuring full control and privacy.
- Select your preferred devices, and we will make them available in our data center for seamless testing.
- Run both automated and manual tests on your private iOS and Android devices with full control and security.
[Learn more](https://testingbot.com/enterprise/private-device-cloud)
Side by side
## How TestingBot for Enterprise compares.
Everything in Standard, plus the security, governance and support large teams need.
| Feature | Standard Plans | Enterprise Plans Recommended |
| --- | --- | --- |
| Security & Compliance |
| IP Whitelisting (inbound & outbound) | – | |
| Single-Use VMs + Firewalled | | |
| 2FA enforcement | – | |
| Administration |
| SAML 2.0 Single sign-on (SSO) | – | |
| Role-based access control & custom roles | – | |
| Analytics per team & team-member | – | |
| Support |
| Dedicated Support person & Customer Success Manager | – | |
| Custom SLA | – | |
| 24/7 priority support via phone and e-mail | – | |
| Custom Legal Terms | – | |
| Product |
| Customize OS / Pre-installed Software | – | |
| Early access to beta features | – | |
| Unlimited Users | – | |
### Ready to roll out TestingBot across your organisation?
Talk to our enterprise team about SSO, sub-teams, dedicated devices and a deployment plan that fits your security review.
[Contact Sales](https://testingbot.com/contact/new) [Book a demo](https://testingbot.com/demo)
---
URL: https://testingbot.com/enterprise/private-device-cloud
Dedicated devices
# Private Real Device Cloud, dedicated to your team.
Pick the exact iOS and Android devices you want and we provision them in our datacenter. Reserved 24/7 for your team, customised to your testing needs and reachable on real carrier networks.
[Contact Sales](https://testingbot.com/contact/new) [Book a demo](https://testingbot.com/demo)
- **Devices on offer:** Any iOS / Android
- **Availability:** 24/7
Trusted by engineering teams worldwide
What dedicated devices unlock
## The exact device matrix your customers use: at your fingertips, around the clock.
We buy, configure and host the iOS and Android devices you choose. Reserved exclusively for your team and customised to fit how you actually test.
- ### Pick the exact devices
Choose the iPhone, iPad, Pixel, Galaxy or any other device that matches your customer base. We source it, image it and install it for you.
- ### Reserved for your team
Your devices are never shared with other customers. No waiting in queues, guaranteed availability.
- ### Real carrier networks
Equip devices with a physical SIM or eSIM to test scenarios Wi-Fi cannot reach: SMS flows, phone calls, mobile-data behaviour and roaming.
- ### Custom device profiles
We pre-install your apps, profiles, certificates and beta builds. Keep persistent state across runs when you need it.
- ### Available 24 / 7
Your dedicated devices are reachable around the clock from any region for both automated runs and manual live sessions.
- ### Secure & European-hosted
Devices live in our locked, European-hosted datacenter. Network access stays inside your tunnel; logs and recordings stay on EU infrastructure.
How we host your devices
## Real devices, in our secure EU datacenter.
We purchase the iOS and Android devices you specify, install and configure them in our European-hosted datacenter, and connect them to our cloud so your team can run automated and manual tests around the clock.
- Your model list, sourced and provisioned by us, including legacy or hard-to-find devices.
- Custom OS settings, certificates, profiles, beta apps and persistent state, set up once and re-applied automatically.
- Devices stay reachable 24/7.
Carrier connectivity
## Test on real carrier networks with SIM and eSIM.
Equip your dedicated devices with physical SIMs or eSIMs and exercise the full set of mobile-network behaviours your customers actually depend on.
- Send and receive real SMS, including 2FA codes, magic links and OTP flows.
- Place and receive real phone calls; verify in-app calling, voicemail and call-state UI.
- Validate mobile-data, low-signal and roaming behaviours that Wi-Fi can't reproduce.
Frequently asked
## Private Device Cloud: questions teams ask us.
Everything procurement, security and engineering tend to want to know before rolling out dedicated devices.
What is a Private Device Cloud?
A Private Device Cloud is a set of real iOS and Android devices that TestingBot purchases, hosts and maintains in our EU datacenter exclusively for your team. The devices are reachable 24/7 over the same Selenium and Appium endpoints you already use, but no other customer can ever run a test on them.
Can we choose the exact device models we want?
Yes. You pick the exact iPhone, iPad, Pixel, Galaxy or other models you need, including specific OS versions, regional variants and legacy devices that have left our public cloud. If we don't already stock the model, we will source it for you.
Do dedicated devices support real carrier networks?
Yes. We can equip your devices with physical SIMs or eSIMs from your preferred carriers, so you can test SMS, phone calls, mobile data, low-signal and roaming behaviour just like an end user would experience them.
Can we customise OS, profiles or pre-installed apps?
We pre-install your apps and beta builds, deploy custom configuration profiles, certificates and entitlements, and adjust device-level settings to match your needs. By default we wipe state between tests, but on private devices we can keep persistent state when that's useful.
Where are the devices hosted and how are they secured?
All Private Device Cloud devices are hosted in our EU-based datacenter. Devices are physically locked, network access is restricted, and logs, screenshots and video recordings stay on EU infrastructure. We are GDPR compliant and CSA STAR Level 1 registered.
How is the Private Device Cloud priced?
Pricing is custom and depends on the number of devices, models, carrier connectivity and the support tier you need. Talk to our enterprise team, we will put together a quote that fits your scope and timeline.
### Tell us which devices you need.
Share the device matrix, the testing scenarios and the timeline. Our enterprise team will draft a Private Device Cloud setup that fits.
[Contact Sales](https://testingbot.com/contact/new) [Book a demo](https://testingbot.com/demo)
---
URL: https://testingbot.com/partners
# Partner with TestingBot
Join our partner ecosystem to expand your business opportunities and enhance your product capabilities with TestingBot's next generation testing platform.
[Technology Partners](https://testingbot.com/partners/technology-partners) [Reseller Partners](https://testingbot.com/partners/reseller-partners)
- 1+bn
Tests Executed
- 180k
Active Users
- 5,500+
Browsers & Devices
        
        
## Partner Programs
Choose the partnership that best fits your business needs and goals.
### Technology Partners
TestingBot platform integrates with tools that developers use daily. Partner with us to enhance your product capabilities, reduce interruptions for developers and expand customer reach.
- API integration support
- Co-marketing opportunities
- Technical support and documentation
[Learn More](https://testingbot.com/partners/technology-partners)
### Reseller Partners
Expand your business by partnering with TestingBot. Join our Partner Program and earn lucrative commissions, receive dedicated support and gain recognition for your sales success.
- Competitive commission rates
- Dedicated partner support
- Sales and marketing materials
[Learn More](https://testingbot.com/partners/reseller-partners)
### Ready to Partner with Us?
Join TestingBot's growing partner ecosystem and unlock new opportunities for your business.
[See existing integrations](https://testingbot.com/integrations)
---
URL: https://testingbot.com/partners/technology-partners
# Technology Partners
TestingBot platform integrates with tools that developers use daily. Partner with us to enhance your product capabilities, reduce interruptions for developers and expand customer reach.
[Apply now](https://testingbot.com#apply) [All Partner Programs](https://testingbot.com/partners)
        
        
## Partner Benefits
Join our technology partner program and unlock powerful integrations with TestingBot's comprehensive testing platform.
### API Integration Support
Get comprehensive technical documentation, SDKs and dedicated engineering support for seamless integration.
### Co-Marketing Opportunities
Joint marketing initiatives, case studies, webinars and placement in our partner directory to expand your reach.
### Technical Documentation
Access to comprehensive technical resources, integration guides and best practices documentation.
### Partner Portal Access
Dedicated partner portal with resources, training materials and integration tracking tools.
### Priority Support
Dedicated technical support channel with priority response times for integration issues.
### Partner Recognition
Featured placement in our partner ecosystem, case studies and recognition at industry events.
## Integration Opportunities
Partner with TestingBot to create powerful integrations that enhance developer workflows and provide seamless testing experiences.
- **Integrate into your product:** Embed our testing capabilities into your app. Offer automated testing to your customers through TestingBot's whitelabel automated testing offerings, or embed realtime live testing in your product.
- **Developer Advocate:** Help developers use TestingBot's platform. We supply you with documentation, tools and free extended trial credits.
- **Integrate your product with TestingBot:** Create an integration between TestingBot and your product.
## Apply to Become a Technology Partner
Ready to integrate with TestingBot? Fill out the form below and our partnership team will be in touch.
Company Name \*
Email Address \*
Tell us about your integration \*
Submit Application
---
URL: https://testingbot.com/partners/reseller-partners
# Reseller Partners
Expand your business by partnering with TestingBot. Resellers in the Partner Program enjoy lucrative commissions, dedicated support and recognition for their sales excellence.
[Apply now](https://testingbot.com#apply) [All Partner Programs](https://testingbot.com/partners)
        
        
## Reseller Benefits
Join our reseller program and unlock revenue opportunities with comprehensive support and competitive commission structures.
### Competitive Commissions
Earn attractive commission rates on all TestingBot subscriptions you refer, with performance-based bonuses for top performers.
### Dedicated Support
Get priority access to our partner success team for sales support, technical questions, and customer assistance.
### Sales & Marketing Materials
Access to professional sales collateral, presentations, case studies, and co-branded marketing materials.
### Training & Certification
Comprehensive product training, sales certification programs, and ongoing education to maximize your success.
### Partner Portal
Access to a comprehensive partner portal with commission tracking, customer management, and resource library.
### Partner Recognition
Top performers receive recognition awards, conference invitations, and exclusive networking opportunities.
## Who We're Looking For
We partner with organizations that share our commitment to quality and customer success. There is no partnership fee.
### Ideal Partner Profile
- Software consultancies and development agencies
- QA and testing service providers
- DevOps and infrastructure consultants
- Technology solution providers
## Apply to Become a Reseller Partner
Ready to join our reseller program? Fill out the form below and our partnerships team will review your application.
Company Name \*
Email Address \*
Tell us about your business \*
Submit Application
---
URL: https://testingbot.com/open-source
Free for Open-Source
# TestingBot loves open source
You move new ideas forward and make software better for everyone. Get free, unlimited cross browser and mobile testing for your open source project.
[Apply for free access](https://testingbot.com/users/sign_up) [See what's included](https://testingbot.com#whats-included)
1+bn
Tests Executed
180k
Active Users
6100+
Browsers & Devices
Trusted by some of the world's most innovative companies
What you get
## What the TestingBot Open Source plan includes
TestingBot's Open Source program offers free, unlimited cross browser and mobile testing to qualifying open source projects, including 2 concurrent tests on 6100+ real browsers and real iOS and Android devices, with support for Selenium, Cypress, Playwright and Appium.
### Unlimited testing
Run as many automated tests as you need. No monthly cap, ever.
### 2 parallel sessions
Two concurrent tests so your CI pipeline keeps moving.
### 6100+ browsers & devices
Every major browser plus physical iOS and Android devices.
### AI, codeless, video & CI
Codeless and AI test creation, video, screenshots, logs and badges, included.
- Free forever for active open source projects
- Unlimited automated test runs
- 2 concurrent (parallel) tests
- 6100+ browser and OS combinations
- Real iOS and Android devices
- Selenium, Cypress, Playwright, Puppeteer, Appium, Espresso, XCUITest
- Video, screenshots, console and network logs
- CI/CD support, status badges and a browser matrix
## Which frameworks and devices are supported
Keep your existing test suite. Point it at the TestingBot grid and run it for free.
### Web frameworks
Run cross browser tests with the tools you already use.
- [Selenium](https://testingbot.com/support/web-automate/selenium)
- [Cypress](https://testingbot.com/support/web-automate/cypress)
- [Playwright](https://testingbot.com/support/web-automate/playwright)
- [Puppeteer](https://testingbot.com/support/web-automate)
### Mobile and app testing
Test on [real iOS and Android devices](https://testingbot.com/support/app-automate).
- [Appium](https://testingbot.com/support/app-automate)
- [Espresso](https://testingbot.com/support/app-automate)
- [XCUITest](https://testingbot.com/support/app-automate)
- [Maestro](https://testingbot.com/support/app-automate)
### Codeless and AI testing
No script to write? Create and run tests in plain language.
- [AI test creation](https://testingbot.com/support/ai)
- [Codeless testing](https://testingbot.com/features/ai-testing)
## Who qualifies for free open source testing
The Open Source plan is for public, non-commercial projects that the community can use and contribute to.
Not sure if your project qualifies? Apply anyway, we review every request.
- Publicly hosted repository on GitHub, GitLab or Bitbucket
- An OSI-approved license (MIT, Apache 2.0, GPL, BSD and similar)
- Actively maintained, with recent commits
- A clear README describing the project
- Not a commercial or closed-source product
## How to apply for the Open Source plan
Three steps and you are testing for free, usually within a couple of business days.
1
### Create a free account
Sign up for the free plan, no credit card required.
2
### Submit your project
Tell us your repository URL and a short description.
3
### Get approved
We verify and upgrade your account, usually within a couple of business days.
First, create your free account. Once you are signed in, you can submit your project right here.
[Create free account](https://testingbot.com/users/sign_up)
Already have an account? [Sign in](https://testingbot.com/users/sign_in).
## Everything you need to test with confidence
### Cross browser grid
Make sure your project works everywhere, across 6100+ browser and OS combinations.
[View supported browsers](https://testingbot.com/support/web-automate/browsers)
### CI/CD integration
Plug TestingBot into your pipeline with GitHub Actions, Jenkins, [Travis CI](https://testingbot.com/support/integrations/ci-cd/travis-ci) or [CircleCI](https://testingbot.com/support/integrations/ci-cd/circleci).
[See integrations](https://testingbot.com/support/integrations)
### Debugging artifacts
Every test ships with a video, screenshots, and console and network logs so you can debug failures fast.
[Explore features](https://testingbot.com/features)
### Status badges & browser matrix
Show your test results on your README with [status badges](https://testingbot.com/support/other/status-badges) and a [browser matrix](https://testingbot.com/support/other/browser-matrix).
[Add a badge](https://testingbot.com/support/other/status-badges)
## Badges and logos for your project
In return for the free plan, we ask approved projects to mention TestingBot in their README, website or docs. Pick whichever fits your project, then copy the snippet.
### README badge
Place it next to your other badges, at the top of your README.
Markdown
```markdown
[](https://testingbot.com/open-source?utm_source=oss-project&utm_medium=badge)
```
HTML
```html
```
reStructuredText and AsciiDoc
reStructuredText
```rst
.. image:: https://testingbot.com/badges/tested-with-testingbot.svg
:target: https://testingbot.com/open-source?utm_source=oss-project&utm_medium=badge
:alt: Tested with TestingBot
```
AsciiDoc
```asciidoc
image:https://testingbot.com/badges/tested-with-testingbot.svg[Tested with TestingBot,link="https://testingbot.com/open-source?utm_source=oss-project&utm_medium=badge"]
```
### Logo for your sponsors section
The snippet shows the white version when the page is in dark mode, including on GitHub.
HTML (Markdown-safe)
```html
```
Download: [SVG](https://testingbot.com/badges/testingbot-logo.svg), [PNG](https://testingbot.com/badges/testingbot-logo.png) (light backgrounds) · [SVG](https://testingbot.com/badges/testingbot-logo-dark.svg), [PNG](https://testingbot.com/badges/testingbot-logo-dark.png) (dark backgrounds)
### A line in your README or docs
For a credits, acknowledgements or "thanks" page.
Markdown
```markdown
Testing provided by [TestingBot](https://testingbot.com/open-source?utm_source=oss-project&utm_medium=mention).
```
HTML
```html
Testing provided by TestingBot.
```
## How TestingBot compares for open source
- **Price for open source:** Free
- **Concurrent tests:** 2
- **Browser and device combinations:** 6100+
- **Real mobile devices:** Yes, iOS and Android
- **Frameworks:** Selenium, Cypress, Playwright, Puppeteer, Appium, Espresso, XCUITest
- **Hosting and compliance:** European-hosted, GDPR-compliant
- **Free alternative to:** BrowserStack and Sauce Labs
Want the full breakdown? [Compare plans and pricing](https://testingbot.com/pricing).
## Frequently asked questions
### Is TestingBot really free for open source projects?
Yes. Qualifying open source projects get a free TestingBot Open Source plan with unlimited automated testing and 2 concurrent tests, at no cost. The plan stays free as long as your project remains an active, public open source project.
### Which testing frameworks does the open source plan support?
The plan supports Selenium, Cypress, Playwright and Puppeteer for web, and Appium, Espresso, XCUITest and Maestro for mobile apps, plus codeless and AI test creation. You use the same TestingBot cloud grid as paid plans.
### Which browsers and devices can I test on for free?
Open source projects can run tests on 6100+ browser and operating system combinations and on real iOS and Android devices in the TestingBot cloud, including older and current browser versions.
### How do I qualify for the open source program?
Your project must be publicly hosted (for example on GitHub, GitLab or Bitbucket), use an OSI-approved open source license, be non-commercial and be actively maintained. Sign up for a free trial, then submit your repository URL and a short description on this page.
### How many concurrent tests do I get on the open source plan?
The open source plan includes 2 concurrent tests, which lets you run tests in parallel and integrate with CI systems such as GitHub Actions, Travis CI and CircleCI.
### Can I use TestingBot with GitHub Actions and other CI systems?
Yes. The open source plan works with any CI/CD system. Point your existing Selenium, Cypress, Playwright or Appium tests at the TestingBot grid, and add status badges and a browser matrix to your README.
### Is TestingBot a free alternative to BrowserStack and Sauce Labs for open source?
Yes. TestingBot offers open source maintainers a free alternative to BrowserStack and Sauce Labs for cross browser and real device testing, with EU-hosted, GDPR-compliant infrastructure.
### What do I get with each test run?
Every test includes a video recording, screenshots, network and console logs and metadata so you can debug failures, plus shareable public links and embeddable status badges for your project page.
### Is there a badge or logo I can add to my project?
Yes. Add the "Tested with TestingBot" badge to your README, or the TestingBot logo to your website or the sponsors section of your docs. Copy-paste snippets for Markdown, HTML, reStructuredText and AsciiDoc are in the badges section on this page. In return for the free plan we ask approved projects to mention TestingBot with a badge, logo or link.
## Ready to test your open source project for free?
Apply today and ship with confidence on every browser and device.
[Apply for free access](https://testingbot.com/users/sign_up)
---
URL: https://testingbot.com/about
# About Us
Founded in 2011, TestingBot aims to provide an easy-to-use platform for cross-browser and mobile testing. By leveraging the power of cloud computing, we can scale effortlessly to meet your testing needs.
We offer flexible plans tailored for both startups and large enterprises, enabling you to test your website across hundreds of real browser and mobile device combinations. Together with **excellent customer service** , we're confident you will enjoy testing your websites with TestingBot.
## Trusted by some of the world's most innovative companies
## Helping developers and teams test. Reliably and at scale.
TestingBot provides a powerful online testing grid with a wide range of real browsers and physical mobile devices.
Our entire infrastructure is hosted in multiple European data centers, located in Belgium, Germany, Finland and Switzerland, ensuring optimal security, performance and reliability.
At TestingBot, we take pride in delivering a stable, fast and secure testing environment for our customers.
+6 million
Tests each month
99.9%
Uptime
12,000
New users annually
## What drives us
The principles that shape how we build, host and support the TestingBot platform.
### Security
Security is a first-class feature at TestingBot. We guarantee single-use machines and wiped devices for each test.
[Visit our Trust Center](https://trust.testingbot.com)
### European infrastructure
Our platform runs from our own infrastructure across Europe. No third-party cloud providers, no data leaving the EU. Your tests and test data stay safe.
### Customer-first support
Real engineers answer every support ticket, fast. We've been building TestingBot since 2011 and we still treat every customer like our first.
## Our journey
TestingBot was founded in 2011 to solve a personal need: the founder wanted an easy way to run automated tests using Selenium on a website. Automated web testing was a tedious task, and the only solution was to set up a Selenium Grid on your own server.
We wanted to change that and started TestingBot to provide a simple solution for automated web testing. Over the years, we've continuously expanded our platform with new features. Explore the timeline below for a complete overview of our journey.
1. 2011
🚀 TestingBot is released to the public as one of the first cloud-based Selenium grids.
2. 2012
Realtime manual testing was added for Desktop browsers.
3. 2013
🖥️ macOS based testing was released: the first company providing test access to remote Safari browsers.
4. 2014
iOS Simulators & Android Emulators are added for mobile testing.
5. 2015
TestingBot Tunnel is released: test websites running on your own computer or private network.
6. 2016
📱 We make our real device farm available to the public. Test your website on physical iOS and Android devices.
7. 2017
🗺️ This year we added geolocation testing, headless testing and various integrations with Slack, TeamCity etc.
8. 2018
We launch codeless automation. Import recorded tests from Selenium IDE and run the tests at specific intervals.
9. 2019
Live App Testing is launched: test your mobile app manually on (physical) mobile devices.
10. 2020
📈 Introducing Test Analytics: keep track of your test analytics. Release Cypress automation support, run your Cypress tests on TestingBot.
11. 2021
We've added Puppeteer Playwright testing to our features. Connect your Puppeteer scripts to the remote TestingBot browser grid.
12. 2022
Android Espresso and XCUITest support. Run tests against native apps on TestingBot's device grid.
13. 2023
Connect your Playwright scripts to the remote TestingBot browser grid.
14. 2024
🔎 Visual Regression Testing is launched: compare snapshots of your website or mobile app against baselines.
15. 2025
✨ Introducing AI-based testing: describe in natural language what you would like to test. TestingBot will monitor your website for failures.
16. 2026
🎭 Maestro mobile testing support: run Maestro flows against iOS simulators, Android emulators and physical devices on TestingBot's device cloud.
## Our team
### [Jochen](https://testingbot.com/about/jochen-d)
Founder 
### [Chandrika](https://testingbot.com/about/chandrika-deb)
Technical Writer
#### Laura
Support
#### Sander
Marketing
## Want to join our team?
Interested in working on exciting technologies?
[We're hiring](https://testingbot.com/careers)
---
URL: https://testingbot.com/about/jochen-d
# Jochen Delabie
Founder & CEO
Moerbeke-Waas, Belgium
Founder of TestingBot. Building cloud testing infrastructure since 2011.
Jochen Delabie founded TestingBot in 2011 to make cross-browser and mobile testing
approachable for development teams. Over the past 15+ years he has worked on test
automation, infrastructure and developer tooling around Selenium, Appium and
Playwright. He writes regularly on the TestingBot blog about browser automation,
mobile testing and AI-driven testing workflows.
## Areas of expertise
Cross browser testing Selenium WebDriver Appium Playwright AI-driven testing Mobile app testing Test automation Continuous integration
## Recent blog posts by Jochen
[ ](https://testingbot.com/blog/ai-test-insights)
Jun 4, 2026
### [Announcing AI Test Insights](https://testingbot.com/blog/ai-test-insights)
Use AI Insights to quickly discover why an automated test failed.
[Read more: Announcing AI Test Insights](https://testingbot.com/blog/ai-test-insights)
[ ](https://testingbot.com/blog/maestro-physical-device-testing)
Jan 19, 2026
### [Maestro Physical Device Testing](https://testingbot.com/blog/maestro-physical-device-testing)
Maestro testing in parallel on physical Android and iOS devices.
[Read more: Maestro Physical Device Testing](https://testingbot.com/blog/maestro-physical-device-testing)
[ ](https://testingbot.com/blog/tvos-automated-testing)
Oct 15, 2025
### [tvOS Physical Device Testing](https://testingbot.com/blog/tvos-automated-testing)
Run automated tvOS tests on physical AppleTV devices.
[Read more: tvOS Physical Device Testing](https://testingbot.com/blog/tvos-automated-testing)
[ ](https://testingbot.com/blog/macos-tahoe-beta-testing)
Jun 10, 2025
### [MacOS Tahoe Cloud Testing](https://testingbot.com/blog/macos-tahoe-beta-testing)
Test your websites and apps on Apple's upcoming macOS Tahoe (macOS 26).
[Read more: MacOS Tahoe Cloud Testing](https://testingbot.com/blog/macos-tahoe-beta-testing)
[ ](https://testingbot.com/blog/chromeos-testing)
Jun 10, 2025
### [ChromeOS Cloud Testing](https://testingbot.com/blog/chromeos-testing)
ChromeOS Manual and Automated testing in the cloud.
[Read more: ChromeOS Cloud Testing](https://testingbot.com/blog/chromeos-testing)
[ ](https://testingbot.com/blog/accessibility-web-testing)
May 26, 2025
### [Accessibility testing with TestingBot](https://testingbot.com/blog/accessibility-web-testing)
TestingBot offers automated accessibility testing for your websites.
[Read more: Accessibility testing with TestingBot](https://testingbot.com/blog/accessibility-web-testing)
[ ](https://testingbot.com/blog/shopify-store-monitoring)
Feb 27, 2025
### [Monitor your Shopify Store](https://testingbot.com/blog/shopify-store-monitoring)
Protect your Shopify Store from downtime: automated monitoring of Shopify Stores with real browsers.
[Read more: Monitor your Shopify Store](https://testingbot.com/blog/shopify-store-monitoring)
[ ](https://testingbot.com/blog/testingbot-ai-agent)
Feb 18, 2025
### [Generative AI test agent](https://testingbot.com/blog/testingbot-ai-agent)
TestingBot has launched an AI Test Agent that will help you to create automated tests
[Read more: Generative AI test agent](https://testingbot.com/blog/testingbot-ai-agent)
[ ](https://testingbot.com/blog/testing-holiday-season)
Nov 19, 2024
### [Automated Testing with TestingBot during the Holiday Season](https://testingbot.com/blog/testing-holiday-season)
Augment your testing this holiday season with TestingBot.
[Read more: Automated Testing with TestingBot during the Holiday Season](https://testingbot.com/blog/testing-holiday-season)
[ ](https://testingbot.com/blog/selenium-20-years)
Oct 29, 2024
### [Selenium 20th birthday](https://testingbot.com/blog/selenium-20-years)
Celebrate Selenium's 20th birthday. An overview of the years.
[Read more: Selenium 20th birthday](https://testingbot.com/blog/selenium-20-years)
## Articles by Jochen
Sep 1, 2026
### [Parallel Testing: What Actually Scales](https://testingbot.com/resources/articles/parallel-testing)
Why adding workers stops helping, how to shard so they are not idle, and what breaks the first time a suite runs concurrently.
Aug 31, 2026
### [Automated Regression Testing in CI](https://testingbot.com/resources/articles/automated-regression-testing)
How to build a regression suite that stays fast and trustworthy in CI, what to leave out of it, and when to run which part.
Aug 31, 2026
### [Mobile App Test Automation at Scale](https://testingbot.com/resources/articles/mobile-app-test-automation)
How to choose between Appium, Espresso, XCUITest and Maestro, and what changes when you run the suite on real devices at scale.
Aug 31, 2026
### [AI in Software Testing: What Works](https://testingbot.com/resources/articles/ai-in-software-testing)
A category-by-category look at where AI genuinely helps a test suite, where it does not, and how to evaluate a tool before you buy it.
Aug 31, 2026
### [Playwright MCP for Test Automation](https://testingbot.com/resources/articles/playwright-mcp)
What Playwright MCP does, how driving the accessibility tree differs from screenshot agents, and where it fits in a real test workflow.
Sep 18, 2025
### [Top Web Development Tools in 2025](https://testingbot.com/resources/articles/web-development-tools)
The most popular web development tools in 2025, split into front-end, back-end and integrated, with what each one is actually good for.
Jun 22, 2025
### [Most popular Screen Resolutions](https://testingbot.com/resources/articles/common-screen-resolutions)
The most common desktop, mobile and tablet screen resolutions in 2025, and what they mean for responsive design and the breakpoints you test.
May 26, 2025
### [Selenium Python Tutorial](https://testingbot.com/resources/articles/python-selenium-web-automation-test)
Getting started with Selenium WebDriver in Python: a first test, locating elements, implicit and explicit waits, alerts and teardown.
May 22, 2025
### [How to Change Browser Settings](https://testingbot.com/resources/articles/how-to-change-browser-settings)
Step-by-step instructions for changing proxy, cookie and cache settings in Chrome, Edge and Firefox, on both Windows and macOS.
May 17, 2025
### [What is isolation testing?](https://testingbot.com/resources/articles/isolation-test)
Isolation Testing means verifying the functionality of a specific part of software on its own, free from outside influences.
## Get in touch with Jochen
Questions about cross-browser testing, mobile automation, or working with TestingBot? Reach out.
[Contact us](https://testingbot.com/contact/new)
---
URL: https://testingbot.com/contact/new
Talk to TestingBot
# Contact TestingBot
If you need support, have questions, or are interested in purchasing a plan, feel free to reach out, we'll get back to you promptly.
Name
Work email
How can we help?
The more context you share, the faster we can route your message to the right person.
By sending this message you agree to our [privacy policy](https://testingbot.com/privacy). We use your details only to reply to your request.
Send message
Need a faster path?
Skip the form for these common tasks.
[Book a 15-minute demo Live walkthrough by an engineer](https://testingbot.com/demo) [Browse documentation Setup guides for every framework](https://testingbot.com/support) [Check system status Live uptime & incidents](https://status.testingbot.com) [Join our Slack community Get help from devs and our team](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
Other ways to reach us
[Phone (US toll-free) +1 (855) 410-0024](tel:+18554100024) [Email info@testingbot.com](mailto:info@testingbot.com)
Address
TestingBotVlinderhof 18 9180 Moerbeke Belgium VAT BE 0842.864.474
## Frequently Asked Questions
Quick answers about contacting our team, demos, trials and security.
How quickly will TestingBot respond to my message?
Our team replies to most contact form messages within a few hours during European business hours, and within 24 hours otherwise. Paid customers get priority support around the clock.
Should I contact sales or support?
Use this form for sales, billing, partnership and general questions. For technical issues with a running test or your account, signed-in users can also reach support directly from the dashboard.
Can I try TestingBot before talking to sales?
Yes. You can sign up for a free account that includes 100 minutes of automated and manual testing on real browsers and devices, with no credit card required.
Where is TestingBot hosted and is my data safe?
TestingBot is built and hosted in Europe (Belgium). All test data, video recordings and logs stay on EU infrastructure. We are GDPR compliant and a CSA STAR Level 1 registered cloud provider.
Can I schedule a live demo?
Yes. Use the Book a Demo button on this page to pick a 15-minute slot. We will walk you through automated, manual and AI-driven testing on real browsers and devices.
Do you offer enterprise plans, dedicated devices or on-premise?
Yes. We offer enterprise plans with dedicated browsers and devices, single sign-on, custom contracts and a dedicated private cloud option. Mention your needs in the message and our enterprise team will reach out.
---
URL: https://testingbot.com/careers
# Careers
Join our team in maintaining, optimizing and improving our testing cloud.
## Open Positions
- [Technical Content Writer](https://testingbot.com#writer)
## Technical Content Writer
TestingBot is looking for a technical content writer, with experience in Selenium, Cypress, Puppeteer and Playwright.
You will be adding content to our [Blog](https://testingbot.com/blog) and [Resources](https://testingbot.com/resources) section.
### Responsibilities
- Write articles, tutorials and courses about Selenium, Cypress, Puppeteer and Playwright.
- Research interesting topics to write about, related to the online testing ecosystem.
- Add code examples, screenshots and tips & tricks to the articles you write.
### Key Skills
- English writer
- Previous content writing experience
- Faultless grammar and spelling
- IT technical knowledge an advantage, especially if you have knowledge about either of these topics: Selenium, Cypress, Puppeteer or Playwright
- Excellent attention to detail
### Benefits
- Competitive salary
- Flexible schedule
- Work from home (remotely)
Interested? Please send us a brief introduction and resume.
[Apply now](https://testingbot.com/contact/new)
---
URL: https://testingbot.com/terms
# Terms of Service
PLEASE READ CAREFULLY.
THESE TERMS AND CONDITIONS ("THE AGREEMENT") IS A LEGALLY BINDING AGREEMENT BETWEEN YOU ("YOU" OR "YOUR") AND TESTINGBOT. ("TESTINGBOT", "WE" OR "OUR"). BY CHECKING THE APPROPRIATE BOX INDICATING THAT YOU AGREE DURING REGISTRATION OR OTHERWISE ACCESSING THE TESTINGBOT WEBSITE, SERVICES, SOFTWARE OR MATERIALS (COLLECTIVELY THE "SERVICE"), YOU ARE INDICATING THAT YOU HAVE READ THIS AGREEMENT, THAT YOU UNDERSTAND IT, AND THAT YOU CONSENT TO BE BOUND BY ALL OF ITS TERMS AND CONDITIONS. IF YOU DO NOT AGREE WITH ALL OF THE FOLLOWING TERMS AND CONDITIONS, PLEASE DO NOT CONTINUE THE REGISTRATION PROCESS OR USE THE SERVICE.
IF YOU WISH TO USE TESTINGBOT AS AN EMPLOYEE, CONTRACTOR, OR AGENT OF A CORPORATION, PARTNERSHIP OR SIMILAR ENTITY, THEN YOU MUST BE AUTHORIZED TO SIGN FOR AND BIND THE ENTITY IN ORDER TO ACCEPT THE TERMS OF THIS AGREEMENT. THE LICENSES GRANTED AND SERVICES PROVIDED UNDER THIS AGREEMENT ARE EXPRESSLY CONDITIONED UPON ACCEPTANCE BY SUCH AUTHORIZED PERSONNEL.
This End User License Agreement ("Agreement") is entered into by and between TestingBot and the customer (individual or entity) that has procured the TestingBot products or services for use as an end user ("Customer"). "Effective Date" means the date on which accepts the terms and conditions of this Agreement. TestingBot reserves the right, at its sole discretion, to modify or replace this Agreement, or change, suspend, or discontinue any TestingBot Product(s) or Service Descriptions (as those terms are defined below), including without limitation, the availability of any feature, database, or content) at any time by posting a notice on the TestingBot Product(s) or by sending you an email. It is your responsibility to check this Agreement periodically for changes. Your continued use of any TestingBot Product following the posting of any changes to this Agreement constitutes acceptance of those changes.
## 1. Provision of Products
Welcome to TestingBot.com. We provide our customers with online software-as-a-service/platform-as-a-service products.
We use Selenium, an open-sourced browser testing framework to test websites in multiple browsers in an automated way.
We offer several plans according to the needs of the customer. Each plan is provided on a subscription basis for a set term designated at the time of purchase (each, as “Subscription Term”). Customer shall purchase and TestingBot shall provide the specific product and plan as specified at the time of purchase.
Some elements of the Service are available to any User, while other elements ("Subscription Services") are only available to Users who have purchased a subscription from TestingBot according to these terms and conditions. TestingBot may at any time, in its sole discretion, change a freely available element of the Service to a Subscription Service, and vice versa. TestingBot may, it its sole discretion, offer Users a one-time, free trial period during which you may use Subscription Service(s) for free. The time period of any such free trial period may vary from time to time, and will be indicated at the time you register as a User.
Please note that TestingBot does not provide any internet connectivity services, nor does TestingBot provide any computer equipment for Users. You are entirely responsible for securing and paying for any such services and equipment, and for ensuring that such services and equipment meet the minimum requirements for the use of the Services.
General Restrictions; Acceptable Use. Customer shall not: (a) rent, lease, copy, provide access to or sublicense any TestingBot Product to a third party; (b) use any TestingBot Product to provide or incorporate any TestingBot Product into any product or service provided to a third party, (c) reverse engineer, decompile, disassemble, or otherwise seek to obtain the source code or non-public APIs to TestingBot Products, including the Video Playback, except to the extent such restriction is prohibited by applicable law (and only upon advance notice to TestingBot), (d) modify any TestingBot Product or any Documentation, or create any derivative product from any of the foregoing, (e) remove or obscure any proprietary or other notices contained in any TestingBot Product (including any reports or data printed from TestingBot Products) or (f) publicly disseminate information regarding the performance of TestingBot Products.
## 2. Accounts/Customer Data
You may establish one (1) User account (the "Account") per email address you supply to TestingBot. You may not share the Account with anyone. You are liable for all uses of the Account.
If you would like to have several accounts for co-workers or friends, please consider using a sub-account.
Storage of Customer Data. TestingBot does not provide an archiving service. TestingBot agrees only that it shall not intentionally delete any Customer Data which is less than 30 days old from TestingBot Products prior to termination of Customer’s applicable Subscription except as directed by Customer or its Permitted Users. TestingBot expressly disclaims all other obligations with respect to storage. If specified at the time of purchase, TestingBot may charge Customer a fee for storing the Customer Data.
## 3. Registration
During the registration process prior to using the Service, you may be requested to provide certain information (e.g. your birthdate, gender, zip code, name, e-mail address, country or territory of residence, and for paid services, credit card and billing information, such as your telephone number), and you agree to provide true, accurate, current and complete information about yourself as prompted by such process (such information being "Personal Information"). If you open an Account, and any of your Personal Information changes (e.g., new credit card number, expiration date, new address or email address, new phone number, etc.), you must update that information in your Account.
## 4. Subscription Services
If you elect to subscribe to Subscription Services, the fee therefor will be the price specified at the time of purchase. You hereby agree to pay all charges incurred by the Account, including applicable taxes, in accordance with billing terms for access to the Service that are in effect at the time that the fee or charge becomes payable. Your right to access the Service is subject to any limits established by TestingBot or by your credit card issuer, billing service, or other payment methods authorized by TestingBot. TestingBot may, in its sole and absolute discretion, suspend or terminate your access to the Service and disable or delete the Account, if: (i) payment cannot be charged to your credit or debit card; or (ii) your charge is returned to TestingBot for any reason. If at any time you have a balance due on the Account, you agree that TestingBot can charge these unpaid fees to your credit card or debit them from your debit card, as applicable. You agree to reimburse TestingBot for all costs and expenses incurred by TestingBot in collecting payment due hereunder, including without limitation all bank or service charges, and any reasonable attorneys' fees.
By subscribing to any Subscription Service you are representing to us that you are authorized to use the credit card you submit for payment. You agree to take such steps as are appropriate to ensure such compliance and will indemnify and hold TestingBot and TestingBot's affiliates and distribution and syndication partners harmless from any breach of this Agreement.
Your subscription will last for the period chosen in the subscription process (the "Term"). Unless notifying us of your decision to terminate your subscription, your subscription will automatically be renewed at the end of each Term according to the then-current billing terms. TestingBot reserves the right to change the amount of, or basis for determining, any fees it charges, and to institute new fees, effective immediately upon notice to you. You may cancel your automatic membership renewal by contacting info@testingbot.com . Cancellation of your subscription will take effect upon the end of the then-current Term, at which time you will no longer have access to any Subscription Services.
Our payment provider (Stripe or FastSpring) will automatically renew your paid subscription. Payment details such as your credit card number are not saved on this website.
## 5. TestingBot Materials
The Service may require that you install plugins to report back test results to TestingBot. That software, and all new versions, updates, upgrades, bug fixes, modifications and derivative works that TestingBot may make available from time to time are referred to collectively as the "Software." All Software, designs, text, images, photographs, illustrations, audio clips, video clips, artwork, graphic materials, other copyrightable elements, and the selection and arrangements thereof, and all trademarks, service marks, trade dress, trade names, logos and designs and all intellectual property rights therein, provided by TestingBot via the Service (collectively the "Materials") are the property of TestingBot and/or its business partners, affiliates, assigns, licensors or other respective owners and are protected by intellectual property laws, including without limitation, U.S. and international patent, copyright and trademark laws. TestingBot may include in some Materials or in conjunction with your User Supplied Content third party advertisements that appear or are played when those Materials or User Supplied Content are viewed. You hereby consent to the display and playback of these advertisements in conjunction with your User Supplied Content. To the extent you elect to use or access TestingBot Materials or User-Supplied Content from other Users, you also consent to the display of advertisements in those Materials and that User Supplied Content. Content generated during testing like logs, screenshots and videos will stay available for 30 days. After this period, TestingBot will remove the content.
## 6. License from TestingBot
Subject to all of the terms and conditions of this Agreement, TestingBot hereby grants you a personal, limited, non-exclusive, non-transferable, revocable right to (a) view or play back Materials on, your compatible computer or device.
All rights not expressly granted to you are reserved. For the avoidance of doubt, your use of the Service and the Materials is under license and will terminate if this Agreement terminates, regardless of whether the relevant Service or Materials is free or you have paid for it; you will not obtain any ownership interest in any Materials through this Agreement or otherwise.
## 7. Usage Restrictions
You agree not to use, download, reproduce, modify, create derivative works from, display, perform, publish, distribute, disseminate, transmit, broadcast or circulate any Materials to any third party, except as expressly authorized by TestingBot in connection with the Service. The source code and/or object code to the Materials contains confidential and trade secret information belonging to TestingBot and its licensors, and you shall at all times take reasonable steps to protect the confidentiality of such information. You further agree that you will not disassemble, decompile or reverse engineer the Materials (except to the extent that this restriction is expressly prohibited by law, and then only upon providing TestingBot with prior written notice and opportunity to respond). Any unauthorized or prohibited use may subject the offender to civil liability and criminal prosecution under applicable federal and state laws.
As between you and TestingBot, TestingBot owns all elements of the Account. Except as specifically permitted and enabled within the Service, you may not transfer your Account, or any component of your User-Supplied Content to any other person. Through the use of the Service you may not impair or violate the rights of TestingBot or any other party with respect to trademarks, logos, or other intellectual property or rights. You may not display or perform Materials or User Supplied Content in a manner or in a context that is, in TestingBot's sole discretion, unlawful, misleading, defamatory, infringing, libelous, disparaging, obscene, an endorsement of any unrelated product or service, an unauthorized parody, or otherwise objectionable.
## 8. Code of Conduct
You may not: (i) post any User Supplied Content that is illegal, abusive, deceptive, pornographic, obscene, defamatory, slanderous, offensive, or otherwise inappropriate or harmful to any person (including any minor) or entity; (ii) sell, rent, lease, or otherwise assign any rights to any Materials or Service to any third party; (iii) remove any proprietary notices or labels on the Materials or Service; (iv) use the Materials or Service in conjunction with any device or service designed to circumvent technological measures employed to control access to, or the rights in, any content protected by the copyright laws of any jurisdiction; (v) use the Materials or Service for any commercial or illegal purpose or to harm minors in any way; (vi) use the Materials or Service to invade the privacy of, or obtain personal information about, any User, or to obtain a list of Users, or to access or use any User's User Supplied Content in a manner not specifically authorized by such User; (vii) except as expressly permitted by TestingBot, copy, modify, erase or damage any Materials or information contained on any TestingBot or third party servers; (viii) use the Materials or Service to violate any law or legal right of any third party, including any publicity or privacy right, copyright, other intellectual property right, or any contractual, fiduciary or confidentiality obligation or duty, or to take any action that is harassing, libelous, defamatory, abusive, tortious, threatening, harmful or otherwise objectionable; (ix) use the Materials or Service to post or transmit any unsolicited advertising, campaigning or promotional materials or to send any Materials from an anonymous or false address; (x) access or use any password protected, secure or non-public areas of the Service except as specifically authorized in writing by TestingBot (unauthorized individuals attempting to access these areas of the Service may be subject to prosecution); (xi) distribute any software viruses or other computer code designed to interfere with the functionality of the Service or any computer systems, software or hardware; (xii) use any data mining, robots, or similar data gathering, spoofing, or extraction tools in connection with the Materials Service; (xiii) mislead other Users or third parties as to your identity or to the origin of a message or content; or (xiv) frame or link to the Service or Materials except as expressly permitted in writing by TestingBot.
## 9. Objectionable or Explicit Content
TestingBot has the right, but not the obligation, to monitor, modify and/or remove any Materials and/or User Supplied Content in the Service. TestingBot may monitor, modify and remove any Materials and/or User Supplied Content in its sole discretion. TestingBot is not responsible to you for any content or materials constituting all or part of any Materials, any User Supplied Content, or any other aspect of the Service that you might find objectionable. By viewing the Service, you may be exposed to content that you consider offensive. You take sole responsibility for such exposure.
## 10. Privacy
For information regarding TestingBot's use of information collected in connection with the Service, please refer to the [TestingBot Privacy Policy.](https://testingbot.com/privacy)
## 11. Active Participation; Storage and Bandwidth Limits
If you do not have a currently active subscription to the Subscription Service, a condition of your access to the Service is your "Active Participation" in the Service. Active Participation is defined as accessing the Service, including displaying Materials or sending User Supplied Content through the Service, at least once every 180 days. If you do not maintain Active Participation, TestingBot may, with or without providing you with notice, terminate your Account (or any part thereof) and/or your use of the Service, and may remove and discard any content, including any of your User Supplied Content and any related Materials within the Service. Additionally, with or without providing you with notice, we may place a maximum limit on the amount of User Supplied Content that you are allowed to upload into the Service, or the maximum bandwidth that you may utilize each day. We may change these limits in our sole discretion upon providing you with notice. If you exceed these limits, we may refuse to accept and/or remove any User Supplied Content, or we may suspend certain functions of the Service for your Account.
## 12. Changes to the Agreement
We reserve the right, at our sole discretion, to change, modify or otherwise alter these terms and conditions at any time. Such changes and/or modifications shall become effective immediately upon posting. Please review the terms and conditions periodically. If you do not agree to be bound by (or cannot comply with) the Agreement as amended, your only remedy is to cancel your TestingBot Account and to cease using the Service. You will be deemed to have accepted the Agreement as amended thirty (30) days after the amended agreement is posted if you continue to use the Service. All other terms of this Agreement will continue in effect.
## 13. Notices
TestingBot may provide you with any notices under this Agreement by means of a posting on the Service, by e-mail, or by sending a message to you through the Service.
A. A physical or electronic signature of the owner (or person authorized to act on behalf of the owner) of an exclusive right that is allegedly infringed.
B. Specific identification of the copyrighted work claimed to have been infringed, or if multiple copyrighted works are covered by a single notification, a list of each copyrighted work claimed to have been infringed.
C. Information related to the work(s) reasonably sufficient for TestingBot to promptly locate the work (e.g. title of work, location within the Service, etc.).
D. Information reasonably sufficient to permit TestingBot to directly contact the complaining party, such as a complete name and address, telephone number, and an email address.
E. A statement that the complaining party has a good TestingBot belief that use of the work(s) in the manner complained of is not authorized by the copyright owner, its agent, or the law.
F. A statement requesting that TestingBot take a specific act with respect to the alleged infringement (e.g., removal, access restricted or disabled), and
G. A statement that the information in the notification is accurate, and under penalty of perjury, that the complaining party is authorized to act on behalf of the owner of an exclusive right that is allegedly infringed.
## 14. Trademarks
Except as expressly set forth in this Agreement, all trademarks, service marks, trade names, domain names, slogans, logos, and other indicia of origin that appear on or in connection with the Service and Materials are the property of TestingBot and/or its affiliates and licensors. You may not copy, display or use any of these marks without prior written permission of the mark owner.
## 15. International
You agree to abide by U.S and other applicable export control laws and not to transfer, by electronic transmission, the Service, or otherwise, any content or software subject to restrictions under such laws to a national destination or person prohibited under such laws.
## 16. Promotions and Advertising
TestingBot and/or its business partners may present advertisements and promotional materials on or through the Service. Your participation in any TestingBot promotional event is subject to the terms and conditions associated with that event. Your correspondence or business dealings with, or participation in promotions of, any third-party advertisers on or through the Service, including payment and delivery of related goods or services, and any other terms, conditions, warranties or representations associated with such dealings, are solely between you and such third-party. You agree that neither TestingBot nor its business partners shall be responsible or liable for any loss or damage of any sort incurred as the result of any such dealings or as the result of the presence of such third parties on the Service.
## 17. The Website and Links to Third Party Sites
The Website means this web site, as accessed by any of the following domain names: testingbot.com, \*.testingbot.com and any web site owned or controlled by TestingBot (except as otherwise provided on such other site). However, the Website does not refer to any other web site linked to or from the Website, or to any third party site on which the Service may appear. The Service may present links to retailers and/or other third-party websites not owned or operated by TestingBot. Neither TestingBot nor its business partners are responsible for the availability of these outside sites or their contents. You understand and agree that neither TestingBot nor its business partners are responsible or liable, directly or indirectly, for any damage or loss caused or alleged to be caused by or in connection with your use of or reliance on any content of any such site or goods or services available through any such site or from any such advertiser. You should direct any concerns regarding these third-party sites to those sites' administrators.
## 18. Modifications to Service and Materials
TestingBot may, without liability or obligation, in its sole discretion and at any time: (i) make modifications to, or discontinue, any Materials and/or the Service, including deleting any Downloaded Materials we may have placed on your computer, with or without providing you with notice (except as we may expressly agree to provide you with notice elsewhere in this Agreement); and/or (ii) require you to use modified Materials and/or Service (e.g. Software updates) in order to have continued access to the Service and Materials. The modifications may result in certain Materials being unavailable, or incompatible with certain devices. Other Users may similarly delete or cease offering their User Supplied Materials at any time without notice.
Upon TestingBot giving you notice, you shall promptly cease the use of, and destroy, all copies of any specified Materials, or otherwise modify your usage as directed by TestingBot. You agree that neither TestingBot nor the TestingBot Affiliates shall be liable to you or to any third party for any modification, suspension or discontinuance of the Service or Materials.
## 19. Remedies
You understand and agree that any unauthorized use of the Service or the Materials would result in irreparable injury to TestingBot and/or the TestingBot Affiliates for which money damages would be inadequate, and in such event TestingBot and the TestingBot Affiliates, shall have the right, in addition to other remedies available at law and in equity, to immediate injunctive relief against you. Nothing contained in this paragraph or elsewhere in this Agreement shall be construed to limit remedies or relief available pursuant to statutory or other claims that TestingBot and the TestingBot Affiliates may have under separate legal authority, including but not limited to, any claim for intellectual property infringement. You understand and agree that your cancellation of your Account is your sole right and remedy with respect to any dispute with TestingBot. You can cancel your Account by delivering notice to TestingBot at info@testingbot.com.
## 20. Indemnity
You agree to indemnify, and hold harmless, TestingBot, the TestingBot Affiliates, and each of their respective officers, directors, employees, agents, licensors, representatives, and third party providers from and against all claims, actions, demands, causes of action, and other proceedings, and all losses, expenses, fees, fines, damages and costs, including reasonable attorneys' fees and court costs, arising out of or relating to your breach of this Agreement or improper use of the Service or Materials. TestingBot reserves the right to assume, at its sole expense, the exclusive defense and control of any matter subject to indemnification by you, in which event you will fully cooperate with TestingBot in asserting any available defenses.
## 21. Suspension or Termination
TestingBot may, in its sole discretion, terminate this Agreement or suspend your Account at any time with or without notice to you in the event that you breach (or TestingBot reasonably suspects that you have breached) any provision of this Agreement. If TestingBot terminates this Agreement, or suspends your Account for any of the reasons set forth in this paragraph, TestingBot shall have no liability or responsibility to you, and you agree that you forfeit any amounts you have paid TestingBot. TestingBot may terminate this Agreement and your Account for any reason or no reason at the end of the then-current Term by providing notice to you.
You may cancel your Account (including Subscription Services) at any time by contacting info@testingbot.com. All fees paid and charges made prior to termination as provided herein are nonrefundable, including any advance charge or payment for the subscription term during which you terminate your subscription. Termination of your subscription shall not relieve you of any obligations to pay accrued charges.
Upon termination of this Agreement for any reason, all of your rights to use the Service and materials shall immediately terminate, and you shall immediately cease using and destroy all Materials.
## 22. Disclaimers
You understand and agree that your use of the Service and the Materials is at your own sole risk. TestingBot is not responsible or liable for any infections or contamination of your system, or delays, inaccuracies, errors, or omissions arising out of your use of the Service or Materials. The entire risk as to the quality, accuracy, adequacy, completeness, correctness and validity of any Materials rests with you. THE SERVICE AND THE MATERIALS ARE PROVIDED "AS IS" AND "AS AVAILABLE," AND WITHOUT WARRANTY BY TESTINGBOT AND THE TESTINGBOT AFFILIATES, AND, TO THE MAXIMUM EXTENT ALLOWED BY APPLICABLE LAW, TESTINGBOT AND THE TESTINGBOT AFFILIATES EXPRESSLY DISCLAIM ALL WARRANTIES, EXPRESS AND IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE AND ANY WARRANTY OF NONINFRINGEMENT. THERE IS NO WARRANTY THAT THE SERVICE OR THE MATERIALS WILL MEET YOUR REQUIREMENTS, OR THAT YOUR ACCESS TO THE SAME WILL BE UNINTERRUPTED OR ERROR-FREE. TESTINGBOT AND THE TestingBot AFFILIATES DO NOT WARRANT, GUARANTEE, OR MAKE ANY REPRESENTATIONS REGARDING THE USE OR THE RESULTS OF THE USE OF THE SERVICE OR THE MATERIALS WITH RESPECT TO PERFORMANCE, ACCURACY, RELIABILITY, SECURITY CAPABILITY, CURRENTNESS OR OTHERWISE. YOU WILL NOT HOLD TESTINGBOT AND/OR THE TESTINGBOT AFFILIATES RESPONSIBLE FOR ANY DAMAGES THAT RESULT FROM YOU ACCESSING THE SERVICE OR USING THE SERVICE AND/OR THE MATERIALS. NO ORAL OR WRITTEN INFORMATION OR ADVICE GIVEN BY ANY PERSON SHALL CREATE A WARRANTY IN ANY WAY WHATSOEVER RELATING TO TESTINGBOT AND/OR THE TESTINGBOT AFFILIATES.
UNDER NO CIRCUMSTANCES SHALL TESTINGBOT AND/OR THE TESTINGBOT AFFILIATES BE LIABLE FOR ANY UNAUTHORIZED USE OF THE SERVICE AND/OR THE MATERIALS.
## 23. LIMITATION OF LIABILITY
TO THE FULLEST EXTENT PERMISSIBLE PURSUANT TO APPLICABLE LAW, UNDER NO CIRCUMSTANCES SHALL TESTINGBOT AND/OR THE TESTINGBOT AFFILIATES BE LIABLE TO YOU FOR ANY CONSEQUENTIAL, INCIDENTAL, INDIRECT, SPECIAL OR SIMILAR DAMAGES (INCLUDING, WITHOUT LIMITATION, DAMAGES FOR LOSS OF BUSINESS PROFITS, BUSINESS INTERRUPTION, LOSS OF BUSINESS INFORMATION, AND THE LIKE) ARISING OUT OF THE USE OR INABILITY TO USE THE SERVICE AND/OR THE MATERIALS, EVEN IF TESTINGBOT AND/OR THE TESTINGBOT AFFILIATES, HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. BECAUSE SOME JURISDICTIONS DO NOT ALLOW THE EXCLUSION OR LIMITATION OF LIABILITY FOR CONSEQUENTIAL OR INCIDENTAL DAMAGES, THE ABOVE LIMITATION MAY NOT APPLY TO YOU. TO THE EXTENT THAT IN A PARTICULAR CIRCUMSTANCE ANY DISCLAIMER OR LIMITATION ON DAMAGES OR LIABILITY SET FORTH HEREIN IS PROHIBITED BY APPLICABLE LAW, THEN, INSTEAD OF THE PROVISIONS HEREOF IN SUCH PARTICULAR CIRCUMSTANCE, TESTINGBOT AND/OR THE TESTINGBOT AFFILIATES, SHALL BE ENTITLED TO THE MAXIMUM DISCLAIMERS AND/OR LIMITATIONS ON DAMAGES AND LIABILITY AVAILABLE AT LAW OR IN EQUITY BY SUCH APPLICABLE LAW IN SUCH PARTICULAR CIRCUMSTANCE, AND IN NO EVENT SHALL SUCH DAMAGES OR LIABILITY EXCEED TWENTY FIVE DOLLARS ($25). Notwithstanding the foregoing, in no event shall TestingBot's liability to you for any and all claims, damages, losses, and causes of action (whether in contract, tort or otherwise) exceed the amount paid by you, if any, for accessing the Service.
## 24. Claims and Arbitration
Any action hereunder by a User must be brought, if at all, within one (1) year from the accrual of the cause of action. You agree that the all disputes and claims arising out of, or relating to, this Agreement, the Service or Materials, your use of the Service or Materials (including the arbitrability of any claim or dispute and the enforceability of this paragraph), or to any other alleged act or omission by you or TestingBot, shall be determined exclusively by final and binding arbitration. Provided however, that you agree that the terms of this paragraph do not apply to any breach or alleged breach of the ownership or license rights of TestingBot or any of its licensors with respect to any Materials and/or Service, or your (or any third party's) misuse of the Materials and/or Service, and that, in the event of any such breach of alleged breach, TestingBot and its licensors will be irreparably harmed and entitled to equitable and injunctive relief in addition to all other remedies provided by this Agreement or available at law. Except as expressly set forth above in this paragraph, you and TestingBot may litigate in court only to compel arbitration under this Agreement or to confirm, modify, vacate or enter judgment on the award rendered by the arbitrators. To the extent that you have breached or have indicated your intention to breach this Agreement in any manner which violates or may violate TestingBot's or any of its licensor's intellectual property rights, or may cause continuing or irreparable harm to TestingBot (including, but not limited to, any breach that may impact TestingBot's or its licensor's intellectual property rights, or a breach by reverse engineering), TestingBot may seek injunctive relief, or any other appropriate relief, in any court of competent jurisdiction. To the fullest extent permitted by applicable law: no arbitration under this Agreement shall be joined to an arbitration involving any other current or former licensee of TestingBot, whether through class arbitration proceedings or otherwise; no finding or stipulation of fact in any other arbitration, judicial or similar proceeding may be given preclusive or collateral estoppel effect in any arbitration hereunder (unless determined in another proceeding between you and TestingBot); and no conclusion of law in any other arbitration may be given any weight in any arbitration hereunder (unless determined in another proceeding between you and TestingBot).
## 25. Miscellaneous
This Agreement and any other terms or documents referred to herein represent your entire agreement with TestingBot with respect to your use of the Service and Materials. As used in this Agreement the term "licensor" shall not refer to the person at that time entering into the Agreement. You understand and agree that, except as expressly set forth herein, this Agreement is not intended to confer and does not confer any rights or remedies upon any person other than the parties to this Agreement. If any part of this Agreement is held invalid or unenforceable, that portion shall be construed in a manner consistent with applicable law to reflect, as nearly as possible, the original intentions of the parties, and the remaining portions shall remain in full force and effect. TestingBot's failure to pursue any available claim or defense pursuant to this Agreement or otherwise will not be a waiver of such claim or defense. The headings used in this Agreement are for convenience only, and will have no effect on the interpretation or legal enforceability of the terms herein. This Agreement is personal to you and may not be transferred, assigned or delegated to anyone. Any attempt by you to assign, transfer or delegate this Agreement shall be null and void.
Last updated: May 9, 2017
---
URL: https://testingbot.com/privacy
# Privacy Policy
TestingBot knows that you care how information about you is used and shared, and we appreciate your trust that we will do so carefully and sensibly.
By visiting testingbot.com, you are accepting the practices described in this Privacy Policy.
TestingBot knows that you care how information about you is used and shared, and we appreciate your trust that we will do so carefully and sensibly. We process personal information contained in Customer Data on behalf of our customers under the terms and conditions of our [TestingBot Customer Data Processing Addendum](https://trust.testingbot.com).
- [What's your policy about Cookies?](https://testingbot.com#cookies)
- [What personal information does TestingBot gather?](https://testingbot.com#personal)
- [Which information do you share?](https://testingbot.com#information)
- [What Data Subject Rights Do I Have?](https://testingbot.com#rights)
- [How Long Do We Retain Your Personal Data?](https://testingbot.com#duration)
- [Is my information safe on this website?](https://testingbot.com#secure)
- [Which information can I access?](https://testingbot.com#access)
- [Are children allowed to use TestingBot?](https://testingbot.com#children)
- [How can I remove all my information from TestingBot?](https://testingbot.com#delete)
- [Who is the Data Controller?](https://testingbot.com#controller)
- [What Are Our Legal Bases for Processing Personal Data?](https://testingbot.com#legal)
- [Conditions of use, notices, and revisions](https://testingbot.com#changes)
- [Implemented Technologies](https://testingbot.com#technologies)
## What's your policy about Cookies?
Please consult our [Cookie Policy](https://testingbot.com/cookie-policy).
## What personal information does TestingBot gather?
The information we learn from customers helps us personalize and continually improve your experience at testingbot.com.
During signup, you are asked for your first name and last name, together with an e-mail address.
When upgrading your plan, you may be asked for a telephone number and address by our billing provider.
Information from Other Sources: We might receive information about you from other sources and add it to our account information.
**Fraud prevention at signup:** when an account is created, we record technical characteristics of the request so we can detect fraudulent signups and abuse of free trial credits. This covers the IP address and derived network information (country, network operator), the browser user agent, and a device identifier computed locally in your browser from its configuration. We do not use this information for advertising, we do not share it with third parties, and it is not used to track you across other websites. It is retained with your account. When an account is deleted, we keep these signup records, together with a one-way cryptographic hash of the email address (not the address itself) and the amount of free trial credit the account had left, for up to 12 months, so that deleting an account and registering again cannot be used to obtain a new free trial. They are then deleted.
You are not required to provide any personal data to us, but if you do not provide any personal data to us, you may not be able to use certain features of our Services...
## Which information do you share?
- **Subcontractors, sub-processors and service providers:** We share your firstname, lastname and email address with our payment providers (Stripe Inc. and Bright Market, LLC (FastSpring)) and with our customer support platform (Intercom). The complete, current list of sub-processors, including what each receives, is maintained at [trust.testingbot.com/sub-processor-updates](https://trust.testingbot.com/sub-processor-updates).
- A third party, in order to enforce or defend our rights, or to address financial or reputational risks.
- A third party to respond to requests relating to a criminal investigation or alleged or suspected illegal activity.
- Other recipients where we are authorised or required by law to do so.
- A purchaser or prospective purchaser of all or part of our assets or our business...
## What Data Subject Rights Do I Have?
- **Right of access:** You have the right to obtain from us confirmation as to whether your personal data is being processed...
- **Right to rectification:** You have the right to obtain from us the rectification of inaccurate personal data...
- **Right to erasure:** You have the right to ask us to erase your personal data...
- **Right to restriction of processing:** You have the right to request restriction of processing...
- **Right to data portability:** You have the right to receive your personal data in a structured, commonly used...
- **Right to object:** You have the right to object, on grounds relating to your particular situation...
- **Right to Submit Complaints:** You have a right to lodge a complaint with a supervisory authority.
## How Long Do We Retain Your Personal Data?
If you register for an account on our Services, we retain your personal data for as long as you have an account with us...
## Is my information safe on this website?
TestingBot uses SSL (HTTPs) on all parts of its service/website. SSL encrypts information between you and our website.
Payment data is handled by our payment providers (Stripe Inc. and Bright Market, LLC (FastSpring)); PayPal payments are processed through Stripe Checkout. Full card numbers never reach TestingBot servers; we store the last 4 digits only.
All sensitive data is encrypted in our database and in our log files. Passwords and security tokens are stripped before they are logged into our system.
We will notify serious data breaches...
We restrict access to Account-Related Information and other Personal Information...
## Are children allowed to use TestingBot?
We do not sell products for purchase by children. If you are under 18, you may use TestingBot only with the involvement of a parent or guardian.
## Which information can I access?
You can access and modify your personal information in our member area.
## How can I remove all my information from TestingBot?
Please log into our member area, click "Account" and click the "Remove account" button.
## Who is the Data Controller?
If you are using our Services, the data controller is Norento BV (trading as TestingBot), Vlinderhof 18, 9180 Moerbeke, Belgium, +1 (855) 410-0024...
## What Are Our Legal Bases for Processing Personal Data?
- **Contract Performance:** We process the personal data of users of our Services as necessary to perform our contractual obligations...
- **Legitimate Interests:** To provide users with a good user experience, to maintain and secure our Services...
- **Legal Obligations:** If we are subject to a lawful access request, engaged in a legal proceeding...
- **Consent:** If we are required to obtain your consent to send you marketing communications...
- **Vital Interests:** In extenuating circumstances, we may need to process your personal data to protect the vital interests...
## Conditions of use, notices, and revisions
If you choose to visit testingbot.com, your visit and any dispute over privacy is subject to this Notice...
We may periodically alter this policy and our Terms of service. Please check these pages frequently to see the most recent changes.
**Last updated: October 17, 2026** (described what we keep for up to 12 months after an account is deleted, to prevent repeated free trials)
## Implemented Technologies
If you have any questions, complaints or remarks about this Privacy Policy, you can contact our Data Protection Officer at [info@testingbot.com](mailto:info@testingbot.com).
TestingBot
Attn: Data Protection Officer
[info@testingbot.com](mailto:info@testingbot.com)
Telephone number: +1 (855) 410-0024
[Contact TestingBot](https://testingbot.com/contact/new)
---
URL: https://testingbot.com/cookie-policy
# Cookie Policy
TestingBot uses cookies to improve your user experience.
## What's a cookie?
A cookie is a small file which is placed on your computer to remember your preferences.
You can read more about cookies on [AllAboutCookies](https://allaboutcookies.org).
## How are you using cookies?
When you first visit our website, our partner Google Analytics will set a cookie to track which pages you visit.
Next to that, we will set a cookie to determine if you're logged in or not (session cookie). Once you log out, this cookie is destroyed.
We offer online chat as a way to help you with your questions. This 3rd party provider also sets cookies to enhance your user experience.
## Cookie List
Below is the list of cookies we use on TestingBot:
### Strictly Necessary Cookies
These cookies are necessary for the website to function and cannot be switched off in our systems...
#### Cookies used
| Cookie Name | Description | Expiration |
| --- | --- | --- |
| \_testingbot\_gsession | Keep track of whether you're signed in on TestingBot | Until you close your browser |
| remember\_user\_token | If you enabled "Remember me" during login, we will use this cookie to remember your session. | 14 days |
| \_\_stripe\_mid | Used by Stripe. More info on [Stripe's Cookie Policy](https://stripe.com/cookies-policy/legal). | 1 year |
### Performance Cookies
These cookies allow us to count visits and traffic sources so we can measure and improve the performance of our site...
#### Cookies used
| Cookie Name | Description | Expiration |
| --- | --- | --- |
| \_gid | Used by Google Analytics. [More info](https://www.en.advertisercommunity.com/t5/Google-Analytics-Code/Information-about-a-cookie-of-Google-Analytics-named-gid/td-p/1127468). | 30 days |
| \_ga | Used by Google Analytics to identify unique sessions. | 30 days |
### Targeting Cookies
These cookies may be set through our site by our advertising partners...
#### Cookies used
| Domain | Cookies |
| --- | --- |
| youtube.com | - SID - HSID - demographics - VISITOR\_INFO1\_LIVE - PREF - APISID - SSID - LOGIN\_INFO - YSC - SAPISID |
| google.com | - APISID - SSID - NID - PREF - SID - SAPISID - HSID |
### Social Media Cookies
These cookies are set by a range of social media services that we have added to the site...
#### Cookies used
| Cookie Name | Description | Expiration |
| --- | --- | --- |
| intercom-id-[app\_id] | Anonymous visitor identifier for the online chat functionality. | 9 months |
| intercom-session-[app\_id] | Keeps track of chat sessions. Gives users access to their previous conversations and identifies them in the Messenger. | 1 week |
| intercom-device-id-[app\_id] | Identifier for each unique device that interacts with the Messenger. | 9 months |
This policy was last updated on October 8, 2025.
Please see our [Terms & Conditions](https://testingbot.com/terms) and our [Privacy Policy](https://testingbot.com/privacy) for additional information.
---
URL: https://testingbot.com/features/smart-tv-testing
# Online Test Automation On Smart TVs
OTT test automation on real living room hardware: Apple TV 4K running tvOS, hosted in our cloud. Manual and automated Apple TV app testing on real tvOS devices, straight from your browser.
[Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo)
Trusted by some of the world's most innovative companies
## Automated Testing for Smart TV apps
Run Appium tests on TV apps running on real Apple TV 4K hardware with tvOS.
Get access to raw video and audio generated from the Smart TV device. Test DRM-protected content, by accessing a device via manual or automated testing.
[Read SmartTV Testing Documentation](https://testingbot.com/support/app-automate/smart-tv)
### Automated Testing with Appium
Upload your Smart TV app and run automated tests with Appium on the remote device.
### Connect to a remote TV device
Access live video and audio generated from the device, through your own browser and control the remote as if it was in your hand.
### Device Coverage
Test on real Apple TV 4K hardware running tvOS.
## Frequently Asked Questions
### What competitive edge does TestingBot offer with Smart TV testing, compared to its peers?
TestingBot offers both manual and automated testing of apps on Smart TVs. View DRM-protected content, hear live audio of the device under test and manually send remote control (IR signals) to the device.
### How can I test Smart TV applications on a remote Smart TV device?
Simply upload your app to TestingBot Storage and pick a device to test on. TestingBot provides a high quality video and audio stream of the device.
### Can I test Apple TV apps manually as well as with Appium?
Yes. Manually control a real Apple TV 4K through your own browser, including sending remote control signals, and run automated Appium tests against the same device.
### Which Apple TV devices and tvOS versions can I test on?
Testing runs on real Apple TV 4K hardware with tvOS 17.4, hosted in our European datacenter. Set deviceName to Apple TV 4K and platformName to tvOS in your capabilities. We do not currently offer Android TV, Fire TV or Roku.
### How do I run Apple TV automation testing with Appium?
Upload your tvOS app to TestingBot Storage, then point Appium at our hub with platformName tvOS and the app capability set to your uploaded build. Apple TV automation testing then behaves like any other Appium session, with video, logs and screenshots captured for every run.
### Sign up for a Free Trial
Start testing your apps with TestingBot.
[Start a free trial](https://testingbot.com/users/sign_up)
---
URL: https://testingbot.com/features/manual-browser-testing
Live cross-browser testing
# Cross browser testing online, on real browsers
Take control of any real Chrome, Safari, Firefox, Edge or Internet Explorer running on real macOS, Windows or Linux. No VM, no plugin, no install. Open a tab, click a browser, start testing in seconds.
[Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo)
- **Combinations:** 6100+
- **Platforms:** 5
- **Uptime:** 99.9%
_A 2 by 2 grid of remote machines running in parallel in the TestingBot cloud: macOS with Safari, Windows 11 with Edge, Linux with Firefox, and Windows 7 with Internet Explorer 11. Each is opened in a browser tab, each shows DevTools inspecting the same web page._
Trusted by some of the world's most innovative companies
30-second tour
## Why teams pick TestingBot for live browser testing
Real, never emulated
Every session runs on a real macOS, Windows or Linux VM with the real browser binary. No headless shims, no fake Internet Explorer. The DOM, the JS engine and the cipher suite are exactly what your customer sees.
From IE 8 to the latest beta
Chrome, Safari, Firefox, Edge, Opera and every Internet Explorer back to IE 8. macOS Tahoe through Yosemite, Windows 11 through Windows 7, and Linux. The full matrix, always patched.
Zero install, zero setup
No browser extensions, no Java applet, no agent on your machine. Click a browser, our cloud streams a live, interactive desktop into the tab you already have open.
Capabilities
## Everything a real browser session gives you
A live cross-browser test should feel like sitting at the customer's machine. Native dev tools, real input, real audio, real network conditions.
- ### Native DevTools
Open the same DevTools your developers use locally. Chrome DevTools, Safari Web Inspector, Firefox DevTools and the F12 tools in Edge and IE, fully interactive over the live stream.
- ### Streamed audio
Hear video playback, autoplay policies, Web Audio output and HTML5 media events the way the customer would. Audio streams from the remote VM straight into your tab.
- ### Real keyboard and mouse
Type, scroll, drag, right-click and use modifier keys. The remote browser sees the same input events as a local one, so hover menus and keyboard traps behave normally.
- ### Two-way clipboard
Paste credentials, JSON fixtures or long URLs from your machine into the remote browser. No plugin to install. Copy data back out the same way.
- ### Any resolution, on the fly
Switch between 1280x720, 1920x1080, 4K and custom resolutions mid-session to validate responsive breakpoints without restarting the browser.
- ### Geolocation testing
Run a session that exits from the United States, the EU, the UK, Australia, Japan or other regions. Validate currency, language detection, GDPR banners and CDN behaviour.
- ### Network throttling
Simulate Edge, 2G, 3G, 4G and offline conditions. Catch perceived-performance bugs before they hit users on a train or in an elevator.
- ### Mobile browsers
Live sessions on real iPhones, iPads and Android phones from Samsung, Google, Xiaomi and OnePlus. Mobile Safari, Chrome on Android, Samsung Internet, no emulator in sight.
- ### Screenshots & video
Every session records a full video and lets you snap, annotate and share screenshots in two clicks. Send bugs straight to Jira, Slack or GitHub.
- ### Beta and dev builds
Reproduce bugs on Chrome Beta, Firefox Nightly, Safari Technology Preview and Edge Dev within hours of upstream release.
Browser and OS coverage
## A real cross-browser matrix, not a wish list
Real Chrome, Safari, Firefox, Edge, IE and Opera on real macOS, Windows and Linux. Live, on demand, with versions going back as far as IE 8 for legacy regression coverage.
### Desktop browsers
-  Chrome
-  Safari
-  Firefox
-  Edge
-  Opera
-  IE 11
-  IE 10
-  IE 9
-  IE 8
-  Chrome Beta
Updated within hours of every Chromium and Firefox stable release. Beta and Nightly channels available on request.
[View the full browser matrix](https://testingbot.com/support/web-automate/browsers)
### Operating systems
-  macOS Golden Gate (27)
-  macOS Tahoe (26)
-  macOS Sequoia (15)
-  macOS Sonoma (14)
-  macOS Ventura (13)
-  macOS Monterey (12)
-  macOS Big Sur (11)
-  Windows 11
-  Windows 10
-  Windows 8.1
-  Windows 7
-  Linux
macOS Yosemite through macOS Tahoe, Windows 7 through 11, plus Linux images for headless-style regression.
Real browsers vs. local VMs
## Why developers stop running their own browser farm
Maintaining your own VirtualBox image of IE 11 on Windows 7 was already painful in 2018. Here is what teams get back when they move to TestingBot.
| Capability | TestingBot | Local VM / BrowserStack snapshot |
| --- | --- | --- |
| Browser engine | Real Chrome, Safari, Firefox, Edge or IE running on a real OS | VM image that drifts out of date the moment you snapshot it |
| Version coverage | IE 8 to the latest stable, plus Beta and Dev channels | Whatever you had time to download and license |
| Setup | Sign in, click a browser, you are testing | VM tooling, license keys, network bridging, disk space |
| Legal IE licensing | Covered by TestingBot's Microsoft licensing | You source and license each Windows image yourself |
| Sharing a repro | One link, video, screenshots and logs included | Screen-record locally, upload to a chat tool, hope it plays |
| Clean state | VM is wiped after every session, no carry-over cookies | Whatever the previous test left behind |
Native browsers
## Real browsers on real macOS, Windows and Linux
TestingBot does not ship a custom-patched Chrome or a "Chromium-based" Safari clone. Every browser runs on the real operating system it shipped for, with the real binary from Apple, Microsoft, Mozilla and Google. The CSS engine, the JS engine, the TLS stack, even font rendering match what your customer sees.
- Real Chrome on real macOS, not Chromium on Linux
- Real Safari on real macOS Tahoe through Yosemite
- Real Internet Explorer 8, 9, 10 and 11 on the right Windows release
- Real Edge on Windows, Linux and macOS
 Chrome
 Safari
 Firefox
 Edge
 IE 8 to 11
 Opera
macOS Yosemite to Golden Gate
Windows 7 to 11
Linux Ubuntu
Mobile Safari
iPhone, iPad
Chrome for Android
Google, Samsung, OnePlus
Samsung Internet
Galaxy S, A
Firefox for Android
Optional
Real devices and simulators/emulators Tap, pinch, rotate, scroll
Mobile browsers
## Real iPhones, iPads and Android phones from the same tab
Switch from a desktop Chrome session to a real iPhone running Mobile Safari, or a real Galaxy running Chrome on Android, without changing tools. Tap, scroll, pinch-zoom, rotate, and inspect the page with the real device's Safari Web Inspector or Chrome DevTools.
[iPhone testing](https://testingbot.com/test-on-iphone) [iPad testing](https://testingbot.com/test-on-ipad) [Galaxy testing](https://testingbot.com/test-on-galaxy)
Live exit nodes
10+ countries
United States
United Kingdom
Germany
France
Spain
Belgium
Australia
China
Japan
Network throttling sits alongside geolocation: simulate Edge, 2G, 3G, 4G or offline in the same session.
Geolocation testing
## Test from where your customers actually are
Route a live session through an exit node in the United States, the UK, Germany, France, Japan, Australia or other regions. Validate currency, language detection, ad bidding, GDPR consent flows and edge CDN behaviour without flying anywhere.
Secure tunnel
## Test staging and internal apps behind your firewall
TestingBot Tunnel opens a secure, encrypted route from our cloud browsers to _your_ local server, staging environment or internal network. No VPN, no firewall changes, full control of the proxy and policies.
Start the tunnel from your laptop or your CI runner. The session loads your staging URL exactly the way the user will see it in production, with cookies, auth and SSO intact.
[Read more](https://testingbot.com/support/tunnel)
Security
## Pristine VMs, wiped after every session
Each live test gets its own isolated virtual machine. We do not share state between customers. The instant you close the session, the disk is destroyed and the next test starts on a clean image.
TestingBot is GDPR compliant with EU-only hosting, and supports SAML SSO on enterprise plans.
- One customer per VM, never co-tenanted
- Datacenter-hosted, not residential proxies
- Disk and memory wiped at session end
1 customer per VM
Datacenter hosted
Wiped after each session
CSA STAR Level 1
GDPR compliant
SAML SSO enterprise
From any OS
## Need Safari from Windows? IE from a Mac? Just open a tab.
Designers on Macs need to check Internet Explorer. Windows devs need to check Safari. Linux engineers need to check Edge. TestingBot gives every machine in your company access to every browser, without anyone installing a VM.
The browser runs in our datacenter. You interact with it through a normal browser tab using HTML5. There is nothing to download.
[Start free](https://testingbot.com/users/sign_up)
Your machine
 macOS
 Windows
 Linux
TestingBot cloud
 Chrome
 Safari
 IE 11
Collaboration
## Report bugs without leaving the session
Take and annotate screenshots, share the live session with a co-worker, and download the automatic video recording when the test ends. Each bug ships with the metadata an engineer actually needs: browser, OS, screen size, network conditions and console output.
Jira Slack GitHub
_A TestingBot bug report draft: annotated screenshot, capture strip, console output and network log, ready to share to Jira or Slack_
FAQ
## Frequently Asked Questions
Do I need to install anything?
No. Live browser sessions run entirely in your existing browser tab using HTML5. There is no agent, no Java applet, no browser extension and no Flash. Sign in, click a browser, and the live desktop streams straight into the tab.
Can I hear audio from the remote browser?
Yes. Audio from the remote VM is streamed over the same WebRTC connection as the video, so video playback, autoplay policies, Web Audio output and media events behave the way they do on a customer's machine.
Can I change the screen resolution mid-test?
Yes. Switch between common breakpoints (mobile, tablet, laptop, 1080p, 4K) or set a custom resolution at any point in the session. The remote browser resizes without restarting, so responsive design work does not lose context.
Which browser versions are available?
TestingBot covers Chrome, Safari, Firefox, Edge, Opera and Internet Explorer 8, 9, 10 and 11 across macOS Yosemite through Tahoe, Windows 7 through 11, and Linux. New stable releases land within hours of upstream availability. Beta, Dev and Nightly channels are available on request.
Do I get the native developer tools?
Yes. Chrome DevTools, Safari Web Inspector, Firefox DevTools and the F12 tools in Edge and IE are fully interactive over the live stream. You inspect, profile, edit and debug exactly the way you do locally.
Can I test a site running on my laptop or staging?
Yes. Start [TestingBot Tunnel](https://testingbot.com/support/tunnel) on your machine or in CI. The cloud browsers see your localhost, staging URL or internal-network site over an encrypted route, with no VPN or firewall change required.
Does TestingBot record the session?
Yes. Every live session is recorded as a downloadable video, plus screenshots, console logs and network HARs. Sharing a bug is one link, not a re-run.
Is my data safe?
Each session uses its own isolated VM that is destroyed after you disconnect. TestingBot is GDPR compliant with EU-only hosting. Enterprise plans add SAML SSO and dedicated infrastructure.
Do I need a credit card to try it?
No. The free trial does not require a credit card. Sign up with an email address and you can start a live browser session immediately.
Is this free browser testing, and what does the trial include?
Online browser testing on TestingBot starts on the free plan: no credit card, nothing to install, and a recent version of each browser. The 7-day full-access trial asks for a card, charges nothing during the trial, and unlocks every browser version, with live sessions, screenshots and the secure tunnel included. Paid plans start after that.
### More reasons to love TestingBot
### 24/7 support
Got questions? Ask in the 24/7 in-app customer [chat support](https://testingbot.com#) or [send us an email](https://testingbot.com/contact/new). We are here to help you!
TestingBot Support
Hey 👋 How can we help?
I need support
### Trusted by over 14,000 teams
6M+
Tests
120k+
Users
400+
Enterprises
106
Countries
[Request a demo](https://testingbot.com/demo)
### Developer Documentation
Everything you need to integrate TestingBot into your workflow. Browse guides, API references and SDKs for web and mobile testing.
- [Quickstart tutorials for Selenium](https://testingbot.com/support/web-automate/selenium)
- [Appium Examples](https://testingbot.com/support/app-automate/appium)
- [Integrations](https://testingbot.com/support/integrations)
- [List of browsers & devices](https://testingbot.com/support/web-automate/browsers)
[Explore Documentation](https://testingbot.com/support)
### Enterprise-grade Security
STAR Level 1
Certified
GDPR
Compliant
SAML SSO
Single Sign-On
99.9%
Uptime SLA
[Visit TestingBot Trust Center](https://trust.testingbot.com/)
Ready in 60 seconds
## Start a live browser session in 60 seconds
No credit card. No install. Real Chrome, Safari, Firefox, Edge and IE running in our cloud, ready when you are.
No credit card No install, no VMs Cancel anytime
[Start a free trial](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo)
---
URL: https://testingbot.com/features/manual-mobile-testing
Live mobile app testing
# Test Android and iOS apps online, on real devices
Manually control [a physical mobile device](https://testingbot.com/support/app-automate/devices) in our cloud to test your mobile app. Upload an `.apk` or `.ipa`, or install straight from Google Play or the App Store, and test it online on a real Android phone or iPhone, directly from your own browser. Nothing to install locally, no Android SDK, no emulator setup.
[Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo)
- **Real devices:** 40
- **Platforms:** iOS & Android
- **Uptime:** 99.9%
Trusted by some of the world's most innovative companies
## How to test an APK online
You can test an APK online in your browser by uploading the .apk file to TestingBot and opening it on a physical Android phone in our datacenter. There is no Android SDK to install, no emulator image to download and nothing to configure on your own machine. The same four steps work for iOS - upload an .ipa and it opens on a real iPhone or iPad.
1. Sign in to the TestingBot dashboard and open **Live App Testing**.
2. Under **Select Source** , choose **Uploaded Apps** , then click **Upload App** and pick your build. You can also install straight from Google Play, the App Store or TestFlight instead of uploading a file.
3. Choose a real Android phone, iPhone or tablet from the device list.
4. The app installs and launches. Tap, swipe, type and rotate from your browser, exactly as you would on the phone in your hand.
Supported formats are `.apk` and `.aab` for Android, `.ipa` for physical iOS devices and `.zip` for iOS simulators. Builds can also be pushed from a CI pipeline with a single call to the [Storage API](https://testingbot.com/support/api/storage#upload).
[Start free](https://testingbot.com/users/sign_up)
## Reliable Device Farm
Get instant access to a fast, reliable device infrastructure.
### 24/7 device availability
Physical iOS and Android devices, available at any time of the day.
### Zero setup & Maintenance
Focus on testing instead of worrying about setup and maintenance of a mobile device farm.
### Security
Devices located in a datacenter in Europe. Each test runs on a secure and pristine device.
### Real Android and iOS devices
Test across real iPhone, iPads, Google, OnePlus, Xiaomi and Samsung phones and tabs.
### Variety of devices
Test on any device, flagship or budget, with support for the newest and legacy iOS and Android versions.
### Performance
Our infrastructure is built for performance to eliminate test flakiness and latency.
## The devices you can test on
TestingBot runs a lab of physical phones and tablets in a datacenter in Belgium. These are unmodified retail devices - nothing is jailbroken or rooted - and each one is wiped and re-imaged between sessions, so your build never inherits someone else's state.
We currently keep **40 physical devices** online for app testing.
Browse the full lineup of [iPhones and iPads](https://testingbot.com/test-on-iphone) and [Samsung Galaxy devices](https://testingbot.com/test-on-galaxy), or see [every device and OS version](https://testingbot.com/support/app-automate/devices) we keep online.
iPhone 13 Pro
Pixel 8
Pixel 11
iPhone 17 Pro
iPhone 17 Pro Max
iPhone 16 Pro
iPad (9th generation)
Galaxy S26
Huawei P40
iPhone Air
Pixel 10
iPhone 17
## Take control of a device
Interact with the remote mobile device, just as you would with a device in hand. Tap, scroll, zoom, swipe and more.
## Test native device features
Access and modify device features to test your app in various real-world conditions.
### Network Simulation
Test for scenarios like connectivity, fluctuating network & many more.
### Localization testing
Test your apps across different timezones, languages and geolocations.
### Dark Mode
Test your app's UI with a dark background to ensure a smooth user experience.
### App Orientation
Test your app in the landscape or portrait mode. Switch between orientations to test your app's response.
### Audio
Record and hear in realtime the audio generated on the device.
### Settings
Control various settings on the device.
## Control device
See realtime generated logs from the remote iOS/Android device.
Rotate the device, change its GPS location, modify the system language and more.
Inspect the logs for errors, crashes or stacktraces.
[Start free](https://testingbot.com/users/sign_up)
## Inspect your app
Find locators to use in your Automated Tests. TestingBot highlights the area in your layout while you go over the different elements in the layout hierarchy.
Easily locate XPath, XCUITest and Espresso locators which you can then use for Automated Testing.
## TestingBot Storage
Upload your native mobile app, as an .apk or .ipa, to [TestingBot Storage](https://testingbot.com/support/api#upload) and start testing immediately.
[Start free](https://testingbot.com/users/sign_up)
## Your very own Mobile Devices
Choose your own iOS and Android devices. We will purchase these devices, install and configure them in our secure datacenter.
These devices will be only available to you, 24x7.
Private devices can be customised to your needs. Have us install specific profiles and apps, or change specific settings of the device. You decide.
[Learn more](https://testingbot.com/enterprise/private-device-cloud)
Secure Tunnel
## Test Staged Websites
Use the secure TestingBot Tunnel to easily test your staging and development environments across all browsers, ensuring seamless access behind firewalls or private networks.
Test data and websites from behind your firewall on the TestingBot cloud.
[Read more](https://testingbot.com/support/tunnel)
## Debug the app while you are testing it
A bug you hit by hand should not have to be reproduced twice. Every live session on TestingBot records what happened while you were poking at the app, so you can hand a colleague the evidence instead of a description.
Device logs stream in real time. Android logcat and iOS syslog appear next to the device while you use it, so a crash shows its stacktrace the moment it happens.
A screen recording of the whole session is kept with the test, alongside any screenshots you take.
The Mobile Inspector shows the live layout hierarchy and hands you XPath, XCUITest and Espresso locators you can paste straight into an automated test.
Change GPS location, system language, orientation and network conditions mid-session, then keep testing without restarting the device.
## Emulator or real device?
Both are useful, and TestingBot gives you both. The honest answer is that they answer different questions.
An Android emulator or iOS simulator is fast, cheap and fine for checking that a screen renders and a flow does not crash. It runs your app against a desktop CPU and a synthetic GPU, so it will not tell you how the app behaves on a mid-range phone with two years of wear, a throttled radio and a battery at 12%.
A physical device tells you about the things that only exist in hardware - camera and sensor behaviour, biometrics, push notifications, real network handover, thermal throttling and the specific rendering quirks of a particular vendor's Android skin.
Most teams we work with run emulators in CI on every commit and physical devices before a release goes out. If you want the longer version of this argument, we wrote it up on [real device testing](https://testingbot.com/real-device-testing).
More on the Android side specifically: [running an Android emulator online](https://testingbot.com/android-emulator-online).
## Frequently Asked Questions
### Should I use a real device or an emulator?
TestingBot provides both emulators/simulators, as well as physical devices.
Emulators are good if you want to test basic functionality, but they do not provide the same experience as a real device.
We recommend using both real devices and emulators, depending on your needs.
### Where are the devices located?
TestingBot offers physical devices located in Belgium (Europe).
### Which apps can I test?
You can test any app that is available in the Apple App Store or Google Play Store.
You can also upload your own app (.ipa or .apk) and test it on a real device.
### Do I need the Android SDK to test an Android app online?
No. The device runs in our datacenter and you drive it from a browser tab, so there is no Android SDK, no emulator image and no local toolchain to keep up to date. Uploading a build and opening it on a phone is the whole setup.
### Can I test an APK online for free?
You can start without paying anything. Signing up is free and needs no credit card or local install. Full access to every real iOS and Android device comes with the 7-day trial, which asks for a card but charges nothing during the trial. Paid plans start after that.
### Can I test an iOS app on a real iPhone from my browser?
Yes. Upload an `.ipa`, or install the app from the App Store or TestFlight, and it opens on a physical iPhone or iPad in our datacenter. You control it from a browser tab - no Mac, no Xcode and no provisioning of your own hardware.
### Which file formats can I upload?
For Android, upload an `.apk` or an `.aab`. For iOS, upload an `.ipa` for a physical iPhone or iPad, or a `.zip` for an iOS simulator.
You do not have to upload anything at all if your app is already published - you can install it on the device straight from Google Play, the App Store or TestFlight.
### Can I test an app that talks to my staging server?
Yes. Start [TestingBot Tunnel](https://testingbot.com/support/tunnel) on your machine or a build agent and the device in our datacenter reaches your local or internal environment over an encrypted connection, without you exposing anything publicly.
### More reasons to love TestingBot
### 24/7 support
Got questions? Ask in the 24/7 in-app customer [chat support](https://testingbot.com#) or [send us an email](https://testingbot.com/contact/new). We are here to help you!
TestingBot Support
Hey 👋 How can we help?
I need support
### Trusted by over 14,000 teams
6M+
Tests
120k+
Users
400+
Enterprises
106
Countries
[Request a demo](https://testingbot.com/demo)
### Developer Documentation
Everything you need to integrate TestingBot into your workflow. Browse guides, API references and SDKs for web and mobile testing.
- [Live Mobile App Testing documentation](https://testingbot.com/support/app-live)
- [Upload your mobile app](https://testingbot.com/support/app-automate/help/upload)
- [Preparing your app for testing](https://testingbot.com/support/app-automate/help/prepare)
- [List of mobile devices](https://testingbot.com/support/app-automate/devices)
[Explore Documentation](https://testingbot.com/support)
### Enterprise-grade Security
STAR Level 1
Certified
GDPR
Compliant
SAML SSO
Single Sign-On
99.9%
Uptime SLA
[Visit TestingBot Trust Center](https://trust.testingbot.com/)
### Sign up for a Free Trial
Start testing your apps with TestingBot.
[Start a free trial](https://testingbot.com/users/sign_up)
---
URL: https://testingbot.com/features/ai-testing
# AI Test Automation
Create automated tests in a fast and easy way. Generate tests in natural language with Gen AI. Generate your tests, replay on 6100+ browsers and mobile devices.
[Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo)
Trusted by some of the world's most innovative companies
## AI Testing
Describe in natural language what your test should do and let Gen AI create the test for you. Save time and resources from your developers and QA engineers by using AI to run tests against your website.
The AI Testing Agent will convert your prompt into an automated test, which you can then schedule to run periodically on remote browsers.
[Start free](https://testingbot.com/users/sign_up)
## Example AI usecases to Test Your Website
Use natural language prompts to check critical elements and flows on your site, using Generative AI.
### Shopping Cart
Prompt: Add a product to the shopping cart. Verify on the checkout page that the product is in the shopping cart.
### Handle OCR/Captcha's
Use the AI to handle OCR and captcha's. The AI will perform the necessary actions and validations.
### Skip Locators
The AI will not depend on locators such as XPath or CSS selectors. Simply define elements by their text, such as "Click the button that says 'Submit'".
### Conditional Actions
The AI will be able to perform conditional actions, such as "If the login is successful, check if the user is redirected to the dashboard page".
### Verify video playback
Check if a video is playing correctly on the page. The AI agent will verify the video playback and check for any errors.
### Graph Check
Prompt: Check that the page contains an image of a graph showing positive growth.
### Logo Presence
Prompt: Verify that the company's logo is visible in the page header.
### CTA Button Check
Prompt: Confirm the presence of a “Get Started” button that links to the signup page.
### Form Validation
Prompt: Ensure that submitting an empty form displays an error message.
## AI Chat
Chat with the AI Test Agent and instruct it to perform actions, such as clicking buttons, filling out forms, and verifying elements on the page.
Ask the AI to generate automated test scripts from the actions you specified. Export to Java, Kotlin, Python, C#, Selenium IDE, JavaScript, Ruby and more.
## Record tests
Import your recorded tests from Selenium IDE. TestingBot will run the tests periodically on the remote browsers you specified.
[Start free](https://testingbot.com/users/sign_up)
## AI Test Automation
Run tests without writing any code on the TestingBot platform.
### Browsers & OS
Select one or more browsers running on a specific operating system. Specify browser versions to target specific test environments.
### Schedule
Run tests periodically; every minute, hour, day or at a custom interval.
### Alert
When a test fails, you will receive email notifications or text messages.
### Reports
Receive daily reports on the progress of your tests.
### Settings
Set specific settings such as geolocation, screen resolution and more.
### Performance
The tests will run in parallel on TestingBot's performant browser grid.
## Schedule Tests
Schedule your tests to run on specific days and intervals.
Set a cron format to define the exact days and hours you want your tests to run in parallel on our browsers.
## Frequently Asked Questions
### How can I use AI to create automated tests?
TestingBot allows you to use LLMs to create automated tests in natural language. Simply describe what you want to test, and the AI will generate the test for you.
The AI will generate the test and instruct a remote browser to perform the necessary actions and validations.
### How can I record automated tests?
You can use the official Selenium IDE browser plugin to record your test flow. The recorded test can be imported into TestingBot for execution.
Once imported, you can choose the browsers and devices you want to run the test on, and schedule it to run periodically.
### What are some example AI prompts I can use to create tests?
You can use prompts like "Create a test that logs into my website" or "Create a test that checks if the homepage loads correctly".
Or check if an item can be added to a shopping cart - "Create a test that adds an item to the shopping cart and checks if the cart is updated correctly".
### Why should I use AI to create automated tests?
You can write tests in plain English. Some customers even import their user stories from other platforms such as TestRail, PractiTest or Zephyr into TestingBot AI. It is very easy to explain and understand what is being tested.
Because the TestingBot AI tests are not dependent on XPath or other brittle locators, they are ultra-stable. Customers are running hundreds of tests multiple times a day without a single failure.
### Ready to start AI Testing?
Check out the [codeless testing documentation](https://testingbot.com/support/web-automate/codeless-automation/add-test) to get started.
[View documentation](https://testingbot.com/support/web-automate/codeless-automation/add-test)
Resources you may like
[](https://testingbot.com/blog/testingbot-ai-agent)
### [Generative AI test agent](https://testingbot.com/blog/testingbot-ai-agent)
TestingBot has launched an AI Test Agent that will help you to create automated tests
[Read more: Generative AI test agent](https://testingbot.com/blog/testingbot-ai-agent)
[](https://testingbot.com/blog/codeless-automation-ios-android)
### [Codeless Web Automation on iOS and Android](https://testingbot.com/blog/codeless-automation-ios-android)
We now offer Codeless Automation for websites on iOS Simulators and Android Emulators.
[Read more: Codeless Web Automation on iOS and Android](https://testingbot.com/blog/codeless-automation-ios-android)
### More reasons to love TestingBot
### 24/7 support
Got questions? Ask in the 24/7 in-app customer [chat support](https://testingbot.com#) or [send us an email](https://testingbot.com/contact/new). We are here to help you!
TestingBot Support
Hey 👋 How can we help?
I need support
### Trusted by over 14,000 teams
6M+
Tests
120k+
Users
400+
Enterprises
106
Countries
[Request a demo](https://testingbot.com/demo)
### Developer Documentation
Everything you need to integrate TestingBot into your workflow. Browse guides, API references and SDKs for web and mobile testing.
- [Quickstart tutorials for Selenium](https://testingbot.com/support/web-automate/selenium)
- [Appium Examples](https://testingbot.com/support/app-automate/appium)
- [Integrations](https://testingbot.com/support/integrations)
- [List of browsers & devices](https://testingbot.com/support/web-automate/browsers)
[Explore Documentation](https://testingbot.com/support)
### Enterprise-grade Security
STAR Level 1
Certified
GDPR
Compliant
SAML SSO
Single Sign-On
99.9%
Uptime SLA
[Visit TestingBot Trust Center](https://trust.testingbot.com/)
---
URL: https://testingbot.com/features/accessibility-testing
# Web Accessibility Testing
TestingBot provides automated accessibility testing to help ensure your web applications comply with WCAG standards and are accessible to all users.
Schedule recurring tests, receive detailed reports with actionable insights, and improve your application's accessibility with ease.
[Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo)
Trusted by some of the world's most innovative companies
## Automated Accessibility Testing
Identify and fix accessibility issues across your websites and web apps.
Ensure WCAG compliance and deliver an inclusive experience for all users.
### WCAG Compliance
Test your web applications against Web Content Accessibility Guidelines (WCAG) to ensure compliance.
### Scheduled Testing
Configure one-time or recurring tests on a daily, weekly, or monthly basis to catch accessibility issues early.
### Detailed Reports
Receive comprehensive reports with detailed information about each accessibility violation and recommended fixes.
### Customizable Alerts
Set up notifications via email, webhooks, or SMS when accessibility issues are detected.
### Severity Classification
Prioritize accessibility issues with clear severity labels to focus on the most critical problems first.
### Secure Testing
Test locally hosted and authenticated pages securely using the [TestingBot Tunnel](https://testingbot.com/support/tunnel).
## Accessibility Testing is Essential
Make your web applications accessible to all users and comply with regulations.
### Legal Compliance
Meet legal requirements, such as the European Accessibility Act (EAA) to avoid any penalties.
### Broader Audience
Reach more users by making your applications accessible to people with disabilities, who represent about 15% of the global population.
### Improved SEO
Boost your search engine rankings as accessibility improvements often align with SEO best practices.
### Better User Experience
Create a better experience for all users by implementing accessibility best practices that often improve usability for everyone.
## Detailed Accessibility Reports
Receive comprehensive reports that identify accessibility issues with detailed information about each violation, its severity and recommendations for fixing the issue.
[Start free](https://testingbot.com/users/sign_up)
## WCAG Compliance Testing
Automatically test your web applications against WCAG guidelines to ensure they are accessible to users with disabilities and comply with international standards.
[Read more](https://testingbot.com/support/accessibility/web/rules)
## Scheduled Accessibility Tests
Configure one-time or recurring accessibility tests on a daily, weekly, or monthly basis to continuously monitor your web applications for accessibility issues.
[Read more](https://testingbot.com/support/accessibility/web/scheduled)
## Integrate into your existing tests
With a few lines of code, you can check for accessibility issues during your automated Selenium tests. No custom SDK, library or framework required.
[Read more](https://testingbot.com/support/accessibility/web/selenium)
### Sign up for a Free Trial
Start testing your apps with TestingBot.
[Start a free trial](https://testingbot.com/users/sign_up)
Resources you may like
[](https://testingbot.com/resources/articles/why-accessibility-testing)
### [Why is accessibility testing important in web design?](https://testingbot.com/resources/articles/why-accessibility-testing)
Why accessible design matters, what the four POUR principles ask of you, and how to check a site against them as part of normal testing.
[Read more: Why is accessibility testing important in web design?](https://testingbot.com/resources/articles/why-accessibility-testing)
[](https://testingbot.com/blog/automated-accessibility-testing)
### [Automated Accessibility Testing with TestingBot](https://testingbot.com/blog/automated-accessibility-testing)
Start testing for accessibility issues with TestingBot and Axe.
[Read more: Automated Accessibility Testing with TestingBot](https://testingbot.com/blog/automated-accessibility-testing)
---
URL: https://testingbot.com/features/headless
# Headless Browsers
Connect AI agents and test scripts to remote, headless browsers.
Perform automated browser actions, simulate user interactions or perform data-driven tasks with full browser capabilities and zero setup.
[Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo)
Trusted by some of the world's most innovative companies
## How it works
TestingBot takes care of the setup and maintenance of the headless browsers. Run your sessions in parallel on our cloud-based grid, optimized for speed and performance.
Take advantage of a high performance platform, with low latency and fast browser startup times.
Extract HTML, bypass bot detectors and use user-in-the-loop functionality with Hybrid Automations.
[Start free](https://testingbot.com/users/sign_up)
## Browser Frameworks
These automation frameworks support running headless sessions.
[](https://testingbot.com/features/automation/selenium)
### [Selenium](https://testingbot.com/features/automation/selenium)
Popular Browser Automation Framework, using WebDriver, which supports all major browsers.
[](https://testingbot.com/features/automation/puppeteer)
### [Puppeteer](https://testingbot.com/features/automation/puppeteer)
Chrome DevTools based Automation Framework by Google. Supports Chrome and Edge browsers.
[](https://testingbot.com/features/automation/playwright)
### [Playwright](https://testingbot.com/features/automation/playwright)
Similar to Puppeteer, built by Microsoft, with support for Chrome and Edge.
## Instrument native, remote browsers with speed.
Take advantage of increased performance and speed with headless browsers. Test, scrape or fetch information from websites with real browsers. Feed data to your AI agents.
[Start free](https://testingbot.com/users/sign_up)
## Parallel Sessions
Drastically improve your workflow by running sessions simultaneously. TestingBot provides single-use machines on Windows, Linux and macOS, optimized for speed and stability.
[Start free](https://testingbot.com/users/sign_up)
## Video, screenshots & logs
Every browser comes with access to generated logs, monitoring, videos and screenshots.
[Start free](https://testingbot.com/users/sign_up)
## Remote Browser Functions
Use TestingBot Functions to easily scrape URLs, automate actions, generate PDFs/screenshots and more.
[Read more](https://testingbot.com/support/functions)
### More reasons to love TestingBot
### 24/7 support
Got questions? Ask in the 24/7 in-app customer [chat support](https://testingbot.com#) or [send us an email](https://testingbot.com/contact/new). We are here to help you!
TestingBot Support
Hey 👋 How can we help?
I need support
### Trusted by over 14,000 teams
6M+
Tests
120k+
Users
400+
Enterprises
106
Countries
[Request a demo](https://testingbot.com/demo)
### Developer Documentation
Everything you need to integrate TestingBot into your workflow. Browse guides, API references and SDKs for web and mobile testing.
- [Quickstart for Playwright](https://testingbot.com/support/web-automate/playwright)
- [Run Puppeteer on TestingBot](https://testingbot.com/support/web-automate/puppeteer)
- [Functions: scrape, take screenshots and serverless functions](https://testingbot.com/support/functions)
- [Playwright Recorder](https://testingbot.com/support/web-automate/playwright/recorder)
[Explore Documentation](https://testingbot.com/support)
### Enterprise-grade Security
STAR Level 1
Certified
GDPR
Compliant
SAML SSO
Single Sign-On
99.9%
Uptime SLA
[Visit TestingBot Trust Center](https://trust.testingbot.com/)
### Ready to start your first headless browser session?
Please see the [Playwright documentation](https://testingbot.com/support/web-automate/playwright) to get started.
[View documentation](https://testingbot.com/support/web-automate/playwright)
Resources you may like
[](https://testingbot.com/resources/articles/automated-testing-with-puppeteer)
### [Automated Testing with Puppeteer](https://testingbot.com/resources/articles/automated-testing-with-puppeteer)
Puppeteer combined with a test framework provides a great way to run automated browser tests. Follow this guide for more information.
[Read more: Automated Testing with Puppeteer](https://testingbot.com/resources/articles/automated-testing-with-puppeteer)
[](https://testingbot.com/resources/articles/playwright-visual-regression-testing)
### [Visual Testing with Playwright](https://testingbot.com/resources/articles/playwright-visual-regression-testing)
Playwright provides automated browser testing. It offers a built-in feature to perform visual regression testing for your website.
[Read more: Visual Testing with Playwright](https://testingbot.com/resources/articles/playwright-visual-regression-testing)
[](https://testingbot.com/blog/puppeteer-cloud-testing)
### [Puppeteer Testing in the Cloud](https://testingbot.com/blog/puppeteer-cloud-testing)
Start running Puppeteer tests in the cloud with TestingBot.
[Read more: Puppeteer Testing in the Cloud](https://testingbot.com/blog/puppeteer-cloud-testing)
---
URL: https://testingbot.com/features/parallel-testing
# Parallel testing
Parallel testing is running several automated tests at the same time instead of one after another. Each test gets its own isolated browser or device session, so a suite that takes an hour sequentially can finish in a few minutes. The number you can run at once is your concurrency.
[See plans by concurrency](https://testingbot.com/pricing)
## How concurrency works on TestingBot
Every test you start takes one parallel session for as long as it runs, then releases it. Your plan sets how many sessions can be open at once. Browser sessions and real-device sessions draw on the same pool, so a Selenium run and an Appium run compete for the same slots.
If your CI job starts more tests than your plan allows, the extra ones queue instead of failing. TestingBot holds them and starts each as a slot frees up, so a burst above your concurrency still completes. It just takes longer, which is usually the first sign that it is time to size up.
### Parallel sessions by plan
| Plan | Parallel sessions available |
| --- | --- |
| Automated | 1, 2, 4, 8, 16, 24 |
| Pro | 1, 2, 4, 8, 16, 24 |
| Live (manual) | 1, 2, 4, 10, 25 |
Concurrency is the axis these plans scale on. For what each tier costs, see [pricing](https://testingbot.com/pricing).
## What speedup to actually expect
A suite of 200 tests averaging one minute each takes about 200 minutes end to end. Split across parallel sessions it drops roughly in proportion, until the point where your slowest individual test becomes the floor. Nothing finishes faster than its longest single test, however many sessions you buy.
| Parallel sessions | 200-minute suite finishes in about |
| --- | --- |
| 1 | 200 min |
| 2 | 100 min |
| 4 | 50 min |
| 8 | 25 min |
| 16 | 13 min |
| 24 | 9 min |
Indicative only. Real suites have uneven test durations, so a run ends when the slowest worker finishes rather than when the average one does. Treat these as an upper bound and expect to land slightly above each figure.
To size it against your own suite rather than this example, use the [parallel test calculator](https://testingbot.com/support/parallel-calculator).
## Running your framework in parallel
Parallelism is a test-runner setting, not a rewrite, as long as your tests are independent of each other. Each framework spells it differently:
- [Playwright parallel testing](https://testingbot.com/support/web-automate/playwright/playwright-parallel-testing) — set workers in your Playwright config and shard across CI machines.
- [Selenium parallel testing with JUnit](https://testingbot.com/support/web-automate/selenium/java/parallel-junit) — thread-safe WebDriver instances and parallel JUnit execution.
- [Puppeteer parallel testing](https://testingbot.com/support/web-automate/puppeteer/puppeteer-parallel-testing) — run multiple browser connections concurrently against the grid.
- [Parallel test calculator](https://testingbot.com/support/parallel-calculator) — work out how many sessions your suite needs to hit a target run time.
Framework overviews: [Selenium](https://testingbot.com/features/automation/selenium), [Playwright](https://testingbot.com/features/automation/playwright), [Cypress](https://testingbot.com/features/automation/cypress) and [Appium](https://testingbot.com/features/automation/appium).
## Frequently asked questions
- **What is parallel testing?:** Parallel testing is running several automated tests at the same time instead of one after another. Each test gets its own isolated browser or device session, so a suite that takes an hour sequentially can finish in a fraction of that. The number of tests you can run at once is called your concurrency, or your number of parallels.
- **How many parallel tests do I need?:** Divide your total suite runtime by the wall-clock time you are willing to wait, then round up. A 200-minute suite that must finish inside 20 minutes needs roughly 10 parallels. Adding parallels past that point stops helping, because your slowest single test becomes the floor.
- **Does doubling my parallels halve my test run?:** Close to it at first, but not exactly, and not forever. Real suites have uneven test durations, so the run ends when the slowest worker finishes rather than when the average one does. Speedup is close to linear while you have many more tests than parallels, and flattens as the two numbers converge.
- **What happens if I start more tests than my plan allows?:** They queue rather than fail. TestingBot holds the extra sessions until a slot frees up, then starts them automatically, so a CI job that bursts above your concurrency still completes without errors. It simply takes longer.
- **Do I need to change my test code to run in parallel?:** You change your test runner configuration, not your tests, as long as your tests are independent. Each framework has its own switch: workers in Playwright, threads in TestNG or pytest-xdist, and the parallel flag in Cypress. Tests that share state or fixed test data need isolating first.
- **Does parallel testing work with real mobile devices?:** Yes. Appium, XCUITest, Espresso and Maestro sessions consume the same concurrency as browser sessions, so the same plan covers both. Each parallel session gets its own dedicated device.
---
URL: https://testingbot.com/features/screenshot-testing
# Automated Visual UI Regression Testing
Detect and fix visual issues on your webpages and mobile apps before they impact users. Leverage TestingBot's codeless visual UI testing to get instant alerts whenever a visual discrepancy is detected on your site.
[Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo)
Trusted by some of the world's most innovative companies
## Effortless Visual Regression Testing
Enable visual regression testing in your existing test framework with a single line of code. Quickly detect layout shifts, UI inconsistencies and visual regressions during your test.
Or enter the URLs you would like to monitor. TestingBot will take screenshots on regular intervals and alert you when a change is detected.
[Start free](https://testingbot.com/users/sign_up)
## Review and Approve Visual Regressions
You and your team can review visual test results, approving or rejecting specific UI changes. Use a slider to compare the baseline and new screenshots or analyze a diff image highlighting the differences.
## Codeless Visual UI Testing
Start identifying UI defects on your website in minutes. Just enter your URL, select the browsers you want to test on, and set the testing frequency. Customize alerts to get notified instantly when issues are detected.
When a visual change occurs, you'll receive an instant alert with a screenshot and a diff image highlighting the differences. Keep your UI and UX flawless by catching unexpected changes early.
[Start free](https://testingbot.com/users/sign_up)
## Optimized for accuracy
We built visual testing with accuracy in mind, preventing false-positives during the UI verification process.
### Freeze animations and videos
Animations and videos are temporarily frozen during the visual snapshot, to prevent false positives.
### Ignore UI elements
We ignore certain UI elements that are prone to change and are irrelevant to the screenshot.
### Pixel Perfect Screenshots
Take full-page, above the fold or chromeless screenshots of your website across hundreds of browsers and devices at once.
## Test Automation Frameworks
TestingBot's UI testing seamlessly integrates with Selenium, Appium, Puppeteer and Playwright. Just add a single line of code to enhance your existing tests with automated visual regression testing.
[](https://testingbot.com/support/visual-testing/automated/selenium)
### [Selenium](https://testingbot.com/support/visual-testing/automated/selenium)
Popular Browser Automation Framework, using WebDriver, which supports all major browsers.
[](https://testingbot.com/support/visual-testing/automated/appium)
### [Appium](https://testingbot.com/support/visual-testing/automated/appium)
Mobile Automation Framework, using WebDriver, runs on physical devices and simulators/emulators.
[](https://testingbot.com/support/visual-testing/automated/playwright)
### [Playwright](https://testingbot.com/support/visual-testing/automated/playwright)
Similar to Puppeteer, built by Microsoft, with support for Chrome and Edge.
[](https://testingbot.com/support/visual-testing/automated/puppeteer)
### [Puppeteer](https://testingbot.com/support/visual-testing/automated/puppeteer)
Chrome DevTools based Automation Framework by Google. Supports Chrome and Edge browsers.
## Frequently Asked Questions
### How can I use this to spot visual defects?
TestingBot performs automated pixel-by-pixel comparisons between an image that you marked as correct and a new screenshot that was just taken. If the amount of different pixels exceeds the threshold, the test is considered as failed.
This can happen when you deploy a new version of your website that has an incorrect CSS styling, positioning or is missing from the page. Catch visual bugs before these cause harm to the UX of your webpages.
### Why can’t functional tests cover visual problems?
You can test for dimensions, positioning and styling with regular functional tests. The disadvantage is that it will soon become a huge burden of checkpoints in your code.
Instead of doing the visual checks in your code, it is much more efficient to take snapshots and compare these. The visual results are also much more easily interpreted by developers and QA.
### When Should I Use Visual Testing?
Ideally you would run visual checks as part of your CI/CD pipeline. It could be run after each check-in of front-end code, making sure new code does not negatively impact your product.
### Can AI be used for visual testing?
Machine learning algorithms can be trained to discover visual differences. AI depends on trained data, so it may report false positives/negatives. It is not a Not a One-Size-Fits-All solution and often fails in comparison with exact pixel matching.
### How does Snapshot Testing work?
Snapshot Testing in the context of visual testing for UI on webpages, captures screenshots of the initial state of a webpage. It then compares them against subsequent renders. Tests are considered failed when one or more changes in visual appearance occur. This helps to ensure that UI elements remain consistent.
### Sign up for a Free Trial
Start visually testing your apps and websites with TestingBot.
[Start a free trial](https://testingbot.com/users/sign_up)
Resources you may like
[](https://testingbot.com/resources/articles/python-visual-testing)
### [Visual Regression Testing with Python](https://testingbot.com/resources/articles/python-visual-testing)
How to add visual regression testing to a Python suite: creating a baseline, comparing against it, and keeping screenshots stable over time.
[Read more: Visual Regression Testing with Python](https://testingbot.com/resources/articles/python-visual-testing)
[](https://testingbot.com/resources/articles/angular-ui-testing)
### [Angular UI Testing](https://testingbot.com/resources/articles/angular-ui-testing)
How to add visual regression testing to an Angular app with Cypress, comparing screenshots against a baseline to catch unintended UI changes.
[Read more: Angular UI Testing](https://testingbot.com/resources/articles/angular-ui-testing)
[](https://testingbot.com/blog/visual-regression-testing)
### [Visual Regression Testing](https://testingbot.com/blog/visual-regression-testing)
Learn about TestingBot's visual testing feature, allowing you to easily compare layouts across browsers and devices.
[Read more: Visual Regression Testing](https://testingbot.com/blog/visual-regression-testing)
---
URL: https://testingbot.com/features/automation/puppeteer
Puppeteer cloud
# Run Puppeteer on a real browser cloud
Point your existing Puppeteer scripts at TestingBot's grid and run them across real Chrome and Edge on Windows, macOS and Linux. Massive parallelism, video & log artifacts, GDPR-compliant with EU data residency.
Choosing a framework? See [Selenium vs Puppeteer](https://testingbot.com/selenium-vs-puppeteer) for a head-to-head breakdown, or [Playwright vs Selenium](https://testingbot.com/playwright-vs-selenium).
[Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo)
- **Browsers & OSes:** Chrome · Edge
- **Parallel sessions:** 100
- **Uptime SLA:** 99.9%
Trusted by some of the world's most innovative companies
30-second answer
## Why run Puppeteer on TestingBot?
Real Chrome and Edge
TestingBot runs your Puppeteer scripts against real Chrome and Edge on real Windows, macOS and Linux machines. No more "works in headless Chromium on my laptop" surprises.
Massive parallelism
Fan out to up to 100 concurrent Puppeteer sessions, each in its own single-use VM. Headless or headful, your choice per session.
Recorder & extras
Use the Puppeteer Recorder to generate scripts without writing a line of code. Test browser extensions, web scrape and run cross-browser parallel suites from the same dashboard.
What is Puppeteer?
## Puppeteer in 60 seconds
### Puppeteer
Released 2017 · Chrome DevTools team · Apache 2.0
Puppeteer is a Node.js library from the Chrome DevTools team that drives Chrome and Chromium through the DevTools Protocol. It was the original "headless Chrome" automation library and remains the fastest way to script Chrome from JavaScript.
Puppeteer ships first-class APIs for screenshots, PDFs, network interception, performance traces and page metrics. It is the canonical choice for Chrome-first end-to-end testing, web scraping and headless rendering pipelines.
- Node.js library, JavaScript / TypeScript
- Chrome, Chromium and Edge via the DevTools Protocol
- First-class screenshots, PDFs and network interception
- Headless and headful, single API
- Maintained by the Chrome DevTools team
5-line setup
## Point your scripts at the TestingBot grid
Reuse your existing Puppeteer code. Connect to TestingBot's WebSocket endpoint and your scripts run on real Chrome and Edge in parallel.
connect.js step 1
```javascript
// Connect Puppeteer to TestingBot's cloud
const puppeteer = require('puppeteer-core');
const caps = {
browserName: 'chrome',
browserVersion: 'latest',
platform: 'WIN11',
'tb:options': { key: 'KEY', secret: 'SECRET' },
};
const browser = await puppeteer.connect({
browserWSEndpoint: `wss://cloud.testingbot.com?caps=${
encodeURIComponent(JSON.stringify(caps))}`,
});
```
login.spec.js step 2
```javascript
// Your existing Puppeteer script, unchanged
const page = await browser.newPage();
await page.goto('https://app.example.com/login');
await page.type('#user', 'jane');
await page.type('#pwd', '••••');
await Promise.all([
page.click('#go'),
page.waitForNavigation(),
]);
await page.screenshot({ path: 'dashboard.png' });
await browser.close();
```
See the full [Puppeteer on TestingBot documentation](https://testingbot.com/support/web-automate/puppeteer) for capabilities, video recording and parallel options.
Local vs cloud
## Local Puppeteer vs TestingBot
Puppeteer is fast on a laptop until you need scale, multiple OSes or real Edge. Here is how a hosted cloud compares.
| Capability | TestingBot | Local Puppeteer |
| --- | --- | --- |
| Browser coverage | Real Chrome and Edge across versions | Whatever you have installed locally |
| Operating systems | Real Windows, macOS and Linux | Your single dev machine |
| Parallel scaling | Up to 100 concurrent sessions, on demand | Capped by your CPU and RAM |
| Video & logs | HD video, console logs, screenshots included | Roll your own recording & log capture |
| Test private & staging environments | Built-in secure tunnel | Direct access only, no secure remote sharing |
| EU data residency | Yes, GDPR-compliant | Whatever country your laptop is in |
| Reliability | 99.9% uptime SLA, 24/7 monitoring | Crashes when you close the lid |
Test artifacts
## Video, screenshots and logs out of the box
Every Puppeteer session ships with a full HD video, browser console log and screenshots. Inspect a failure without re-running the test.
Mark sessions as passed or failed from your script, fetch artifacts via our [REST API](https://testingbot.com/support/api), and ship results back to Jira, Slack or your CI of choice.
- Full HD video for every session
- Console and DevTools logs
- Screenshots and PDFs on demand
- Custom screen resolution, timezone and locale
- Browser startup flags and [many more options](https://testingbot.com/support/web-automate/puppeteer/options)
Languages & runners
## Puppeteer integrations for every stack
Puppeteer is Node-first, but the same scripts run from Jest, WebDriverIO and CodeceptJS, plus Python via Pyppeteer and Go via chromedp. All point at the same TestingBot endpoint.
- ### [Node.js](https://testingbot.com/support/web-automate/puppeteer/capabilities)
- [Quickstart](https://testingbot.com/support/web-automate/puppeteer)
- [Capabilities](https://testingbot.com/support/web-automate/puppeteer/capabilities)
- [Browser flags](https://testingbot.com/support/web-automate/puppeteer/options)
- [Test results](https://testingbot.com/support/web-automate/puppeteer/test-results)
- ### [Jest](https://testingbot.com/support/web-automate/puppeteer/jest)
- [Jest with Puppeteer](https://testingbot.com/support/web-automate/puppeteer/jest)
- ### [WebDriverIO](https://testingbot.com/support/web-automate/puppeteer/webdriverio)
- [WebDriverIO + Puppeteer](https://testingbot.com/support/web-automate/puppeteer/webdriverio)
- ### [CodeceptJS](https://testingbot.com/support/web-automate/puppeteer/codeceptjs)
- [CodeceptJS setup](https://testingbot.com/support/web-automate/puppeteer/codeceptjs)
- ### [Python · PyTest](https://testingbot.com/support/web-automate/puppeteer/pytest)
- [PyTest with Pyppeteer](https://testingbot.com/support/web-automate/puppeteer/pytest)
- ### [Go · chromedp](https://testingbot.com/support/web-automate/puppeteer/chromedp)
- [chromedp setup](https://testingbot.com/support/web-automate/puppeteer/chromedp)
Looking for use-case guides? Read our [Puppeteer Recorder](https://testingbot.com/support/web-automate/puppeteer/recorder), [web scraping](https://testingbot.com/support/web-automate/puppeteer/scraping), [browser extension testing](https://testingbot.com/support/web-automate/puppeteer/browser-extension-testing) and [parallel testing](https://testingbot.com/support/web-automate/puppeteer/puppeteer-parallel-testing) guides.
Massive parallelism
## Cut Puppeteer suite time from hours to minutes
Run hundreds of Puppeteer sessions in parallel across single-use VMs. Headless or headful, same scripts, a fraction of the wall-clock time.
- Up to 100 concurrent Puppeteer sessions
- Sessions start in seconds, no warm-up
- Headless or headful, switchable per session
[Read more](https://testingbot.com/support/web-automate/puppeteer/puppeteer-parallel-testing)
Live exit nodes
10+ countries
United States
United Kingdom
Germany
France
Spain
Belgium
Australia
China
Japan
Geolocation testing
## Test Puppeteer scripts from 10+ countries
Run Puppeteer scripts from real exit nodes around the world. Verify **geo-targeting** , **geo-blocking** and **localisation** from the United States, the United Kingdom, Germany, France, Spain, Belgium, Australia, China, Japan and more.
Secure infrastructure
## Real machines. Single-use. Zero leaks.
Each Puppeteer session runs in a brand-new, single-use virtual machine on a real operating system. The VM is destroyed the moment your script ends: nothing is reused, nothing leaks across customers.
GDPR-compliant with EU data residency, SAML SSO and audit logs available for enterprise teams.
- Single-use VMs, wiped after every session
- GDPR-compliant, EU data residency available
- SAML SSO, audit logs and SCIM provisioning
[Visit TestingBot Trust Center](https://trust.testingbot.com/)
1. 1
Script connects to `cloud.testingbot.com`
A pristine VM is provisioned in seconds, just for your session.
2. 2
Puppeteer drives a real Chrome or Edge
Real Windows, macOS or Linux, never simulated, never shared.
3. 3
VM destroyed the moment your script ends
Cookies, files, browser state, all wiped. Nothing is ever reused.
Continuous integration
## Plug Puppeteer into your CI/CD in minutes
TestingBot integrates with every major CI/CD tool. Trigger Puppeteer runs on every commit and stream videos and logs back into your build dashboards.
### [Jenkins](https://testingbot.com/support/integrations/ci-cd/jenkins)
### [Bamboo](https://testingbot.com/support/integrations/ci-cd/bamboo)
### [TeamCity](https://testingbot.com/support/integrations/ci-cd/teamcity)
### [CircleCI](https://testingbot.com/support/integrations/ci-cd/circleci)
### [GitLab CI](https://testingbot.com/support/integrations/ci-cd/gitlab)
### [GitHub](https://testingbot.com/support/integrations/ci-cd/github-actions)
Secure Tunnel
## Test internal & staging apps securely
Use our [secure tunnel](https://testingbot.com/support/tunnel) to run Puppeteer scripts against firewalled staging sites, internal dashboards or apps still on localhost.
Encrypted end-to-end, no inbound firewall rules required, ready out of the box for popular CI providers.
[Read more](https://testingbot.com/support/tunnel)
FAQ
## Puppeteer on TestingBot — frequently asked questions
Why run Puppeteer on TestingBot instead of locally?
Locally you are capped by your laptop or CI runner. TestingBot gives you up to 100 parallel Puppeteer sessions on real Chrome and Edge across real Windows, macOS and Linux machines, with HD video and console logs collected automatically.
Which Puppeteer versions are supported?
TestingBot supports every supported Puppeteer release. You can pin a specific Puppeteer version per-test using capabilities.
Can I run Puppeteer tests in parallel?
Yes. Plans on TestingBot include up to 100 parallel Puppeteer sessions. Each session runs in its own single-use VM, so scripts do not interfere with one another.
Can I use Puppeteer with Jest, WebDriverIO or CodeceptJS?
Yes. We have dedicated guides for [Jest](https://testingbot.com/support/web-automate/puppeteer/jest), [WebDriverIO](https://testingbot.com/support/web-automate/puppeteer/webdriverio) and [CodeceptJS](https://testingbot.com/support/web-automate/puppeteer/codeceptjs), plus Python via [PyTest](https://testingbot.com/support/web-automate/puppeteer/pytest) and Go via [chromedp](https://testingbot.com/support/web-automate/puppeteer/chromedp).
How is Puppeteer different from Selenium and Playwright?
Puppeteer is Chrome-first and Node-first, with a smaller, snappier API than Selenium and a narrower scope than Playwright. Pick Puppeteer when you only need Chrome/Edge and want speed, screenshots, PDFs and network control with minimum ceremony. TestingBot supports all three on the same grid.
Can I record Puppeteer scripts without writing code?
Yes, use the [Puppeteer Recorder](https://testingbot.com/support/web-automate/puppeteer/recorder) to capture clicks, types and assertions, then run the resulting script on the TestingBot grid.
Can I test internal or staged sites?
Yes, install our [secure tunnel](https://testingbot.com/support/tunnel) and your Puppeteer scripts can reach staging environments, localhost or anything else behind your firewall.
Is the Puppeteer cloud GDPR / SOC 2 compliant?
TestingBot is not SOC 2 or ISO 27001 certified. We are GDPR-compliant with EU-only data residency, registered at CSA STAR Level 1 (a published self-assessment), and our control set is modelled on ISO 27001 Annex A. Our datacenter operators hold ISO 27001. We offer SAML SSO, SCIM and audit logs on enterprise plans.
## Ready to scale your Puppeteer testing?
Read our [Puppeteer Cloud Testing](https://testingbot.com/support/web-automate/puppeteer) documentation and run your first script in under 5 minutes.
[Start a free trial](https://testingbot.com/users/sign_up)
Resources you may like
[](https://testingbot.com/resources/articles/automated-testing-with-puppeteer)
### [Automated Testing with Puppeteer](https://testingbot.com/resources/articles/automated-testing-with-puppeteer)
Puppeteer combined with a test framework provides a great way to run automated browser tests. Follow this guide for more information.
[Read more: Automated Testing with Puppeteer](https://testingbot.com/resources/articles/automated-testing-with-puppeteer)
[](https://testingbot.com/blog/puppeteer-cloud-testing)
### [Puppeteer Testing in the Cloud](https://testingbot.com/blog/puppeteer-cloud-testing)
Start running Puppeteer tests in the cloud with TestingBot.
[Read more: Puppeteer Testing in the Cloud](https://testingbot.com/blog/puppeteer-cloud-testing)
---
URL: https://testingbot.com/features/automation/playwright
Playwright cloud
# Run Playwright testing on a real browser cloud
Point your existing Playwright suite at TestingBot's grid for cross-browser Playwright testing across 6100+ real browsers and Android devices. Massive parallelism, test artifacts, GDPR-compliant with EU data residency.
Choosing a framework? See [Playwright vs Selenium](https://testingbot.com/playwright-vs-selenium) or [Cypress vs Playwright](https://testingbot.com/cypress-vs-playwright).
[Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo)
- **Browsers & devices:** 6100+
- **Parallel sessions:** 20+
- **Uptime SLA:** 99.9%
Trusted by some of the world's most innovative companies
30-second answer
## Why run Playwright on TestingBot?
Real browsers, not just Chromium
TestingBot runs your Playwright suite against real Chrome, Firefox, Edge and WebKit on real Windows, macOS and Linux machines. Microsoft Playwright Testing caps you at Chromium on Linux.
Predictable pricing
Plans include unlimited test minutes. No per-minute meter, no separate reporting fee, no surprise bills the morning after a flaky CI run.
Real Android devices
Move past Playwright's mobile emulation and run your flows on real Android phones in our cloud, alongside the same desktop suite.
What is Playwright?
## Playwright in 60 seconds
### Playwright
Released 2020 · Microsoft · Apache 2.0
Playwright is an end-to-end testing framework built by Microsoft that drives Chromium, Firefox and WebKit through one API. It was created by the team that originally built Puppeteer, with cross-browser support and modern web app needs in mind.
Auto-waiting, network interception, multi-tab support and browser contexts come out of the box. The official @playwright/test runner ships with the framework, so you do not need to bolt on Mocha, Jest or PyTest unless you want to. Teams use Playwright for end-to-end testing, cross-browser testing, visual regression testing, API testing, accessibility testing and component testing across a single codebase.
- TypeScript / JavaScript / Python / Java / .NET bindings
- Auto-wait, network interception, fixtures, traces
- Chrome, Firefox, WebKit · headed and headless
- Built-in test runner, parallel workers, retries
- Video, screenshots and logs
5-line setup
## Point your tests at the TestingBot grid
Reuse your existing Playwright code. Set the wsEndpoint to TestingBot's cloud and your tests run on real browsers, real OSes, in parallel.
playwright.config.ts step 1
```typescript
// Connect to TestingBot's Playwright cloud
import { defineConfig } from '@playwright/test';
export default defineConfig({
use: {
connectOptions: {
wsEndpoint: 'wss://cloud.testingbot.com/playwright?
key=KEY&secret=SECRET&browser=chrome',
},
},
workers: 10,
trace: 'on',
});
```
login.spec.ts step 2
```typescript
// Your existing Playwright test, unchanged
import { test, expect } from '@playwright/test';
test('login redirects to dashboard', async ({ page }) => {
await page.goto('https://app.example.com/login');
await page.getByLabel('Username').fill('jane');
await page.getByLabel('Password').fill('••••');
await page.getByRole('button').click();
await expect(page).toHaveURL(/\/dashboard/);
});
```
See the full [Playwright on TestingBot documentation](https://testingbot.com/support/web-automate/playwright) for capabilities, video recording and options.
TestingBot vs MPT
## TestingBot vs Microsoft Playwright Testing
Microsoft Playwright Testing (MPT) is a useful service if you live entirely in the Microsoft stack. For everything else, TestingBot is the more complete cloud.
| Capability | TestingBot | Microsoft Playwright Testing |
| --- | --- | --- |
| Browsers | Real Chrome, Firefox, Edge, WebKit on real OSes | Chromium on Linux only |
| Operating systems | Real Windows, macOS and Linux | Windows and Linux only |
| Pricing | Flat plans with unlimited test minutes | Per-minute, separate reporting fees |
| Real mobile devices | Real Android phones, on demand | Emulation only |
| Test private & staging environments | Built-in secure tunnel | Not available |
| EU data residency | Yes, GDPR-compliant | US-region only |
| Vendor history | Cross-browser cloud since 2012 | New product within Microsoft |
Test artifacts
## Video, screenshots and logs out of the box
Every Playwright session ships with a full HD video, Playwright logs, browser console log and network HAR.
Mark sessions as passed or failed from your test code, fetch artifacts via our [REST API](https://testingbot.com/support/api), and ship results back to Jira, Slack or your CI of choice.
- Full HD video for every test
- Built for speed and reliability
- Console, network HAR and Playwright server logs
- Custom screen resolution, timezone and locale
- Browser startup flags and [many more options](https://testingbot.com/support/web-automate/playwright/options)
Languages & runners
## Playwright integrations for every stack
Playwright ships official client libraries for JavaScript, TypeScript, Python, Java and .NET. Drop them in alongside your test runner of choice.
- ### [TypeScript / JavaScript](https://testingbot.com/support/web-automate/playwright/playwright-test)
- [@playwright/test](https://testingbot.com/support/web-automate/playwright/playwright-test)
- [Jest](https://testingbot.com/support/web-automate/playwright/jest)
- [Mocha](https://testingbot.com/support/web-automate/playwright/mocha)
- [CodeceptJS](https://testingbot.com/support/web-automate/playwright/codeceptjs)
- ### [Python](https://testingbot.com/support/web-automate/playwright/pytest)
- [PyTest](https://testingbot.com/support/web-automate/playwright/pytest)
- [Robot Framework](https://testingbot.com/support/web-automate/playwright/robotframework)
- ### [Java](https://testingbot.com/support/web-automate/playwright/java)
- [Java quickstart](https://testingbot.com/support/web-automate/playwright/java)
- ### [.NET](https://testingbot.com/support/web-automate/playwright/dotnet)
- [.NET quickstart](https://testingbot.com/support/web-automate/playwright/dotnet)
- ### [Ruby](https://testingbot.com/support/web-automate/playwright/ruby)
- [Ruby quickstart](https://testingbot.com/support/web-automate/playwright/ruby)
- ### [Recorder & extras](https://testingbot.com/support/web-automate/playwright/recorder)
- [Recorder](https://testingbot.com/support/web-automate/playwright/recorder)
- [Visual regression](https://testingbot.com/support/web-automate/playwright/visual-regression-testing)
- [Performance](https://testingbot.com/support/web-automate/playwright/performance)
Don't write tests manually? Use the [Playwright Recorder](https://testingbot.com/support/web-automate/playwright/recorder) to generate scripts and run them on the TestingBot grid.
Massive parallelism
## Cut Playwright suite time from hours to minutes
Fan out your Playwright workers across hundreds of single-use VMs running in parallel. Same test code, a fraction of the wall-clock time, no flakiness.
More on how concurrency is sized and priced: [parallel testing](https://testingbot.com/features/parallel-testing).
- Up to 100 concurrent Playwright sessions
- Sessions start in seconds, no warm-up
- Auto-retry, native Playwright --shard support and per-worker artifacts
[Start free](https://testingbot.com/users/sign_up)
Live exit nodes
10+ countries
United States
United Kingdom
Germany
France
Spain
Belgium
Australia
China
Japan
Geolocation testing
## Test Playwright flows from 10+ countries
Run Playwright tests from real exit nodes around the world. Verify **geo-targeting** , **geo-blocking** and **localisation** from the United States, the United Kingdom, Germany, France, Spain, Belgium, Australia, China, Japan and more.
Secure infrastructure
## Real machines. Single-use. Zero leaks.
Each Playwright session runs in a brand-new, single-use virtual machine on a real operating system. The VM is destroyed the moment the test ends, so nothing is reused and nothing leaks across customers.
GDPR-compliant with EU data residency, SAML SSO and audit logs available for enterprise teams.
- Single-use VMs, wiped after every session
- GDPR-compliant, EU data residency available
- SAML SSO, audit logs and SCIM provisioning
[Visit TestingBot Trust Center](https://trust.testingbot.com/)
1. 1
Test connects to `cloud.testingbot.com`
A pristine VM is provisioned in seconds, just for your session.
2. 2
Playwright drives a real browser
Real Chrome, Firefox, Edge or WebKit, never simulated, never shared.
3. 3
VM destroyed the moment your test ends
Cookies, files, browser state, all wiped. Nothing is ever reused.
Continuous integration
## Plug Playwright into your CI/CD in minutes
TestingBot integrates with every major CI/CD tool. Trigger Playwright suites on every commit and stream traces, videos and logs back into your build dashboards.
### [Jenkins](https://testingbot.com/support/integrations/ci-cd/jenkins)
### [Bamboo](https://testingbot.com/support/integrations/ci-cd/bamboo)
### [TeamCity](https://testingbot.com/support/integrations/ci-cd/teamcity)
### [CircleCI](https://testingbot.com/support/integrations/ci-cd/circleci)
### [GitLab CI](https://testingbot.com/support/integrations/ci-cd/gitlab)
### [GitHub](https://testingbot.com/support/integrations/ci-cd/github-actions)
Real mobile
## Playwright mobile testing on real Android devices
Skip the emulators. Run Playwright mobile testing against Chrome on real Android devices in our cloud, with the same Playwright APIs, real hardware and real sensors.
[View available devices](https://testingbot.com/support/web-automate/playwright/mobile)
 
Same Playwright APIs, real Android Chrome.
Secure Tunnel
## Test internal & staging apps securely
Use our [secure tunnel](https://testingbot.com/support/tunnel) to run Playwright tests against firewalled staging sites, internal dashboards or apps still on localhost.
Encrypted end-to-end, no inbound firewall rules required, ready out of the box for popular CI providers.
[Read more](https://testingbot.com/support/tunnel)
FAQ
## Playwright on TestingBot, frequently asked questions
Why run Playwright on TestingBot instead of locally?
Locally you are capped by your laptop or CI runner, usually 2–4 parallel browsers and only the OSes you have installed. TestingBot gives you up to 100 parallel Playwright sessions across real Chrome, Firefox, Edge and WebKit on real Windows, macOS and Linux machines, with the video and logs collected automatically.
How is TestingBot different from Microsoft Playwright Testing (MPT)?
TestingBot runs Playwright on real Chrome, Firefox, Edge and WebKit on real Windows, macOS and Linux. MPT runs Chromium on Linux only. TestingBot offers flat plans with unlimited test minutes, real Android devices and EU data residency. See [the comparison](https://testingbot.com#mpt) on this page.
Which Playwright versions does TestingBot support?
TestingBot supports every supported Playwright version, including the latest releases the day they ship. You can pin a specific Playwright version per-test using capabilities.
Can I run Playwright tests in parallel?
Yes. Plans on TestingBot include up to 100 parallel Playwright sessions. Each session runs in its own single-use VM, so tests do not interfere with one another.
Does TestingBot provide test assets?
Fully. Every session comes with a video recording, screenshots and logs.
How do I migrate my existing Playwright tests?
Set the `wsEndpoint` in your `playwright.config.ts` to `wss://cloud.testingbot.com/playwright`, add your TestingBot credentials, and pick a browser, OS or device through capabilities. See our [migration guide](https://testingbot.com/support/web-automate/playwright#update) for full examples.
Can I test internal or staged sites?
Yes, install our [secure tunnel](https://testingbot.com/support/tunnel) and your Playwright tests can reach staging environments, localhost or anything else behind your firewall, with no inbound rules required.
Is the Playwright grid GDPR / SOC 2 compliant?
TestingBot is not SOC 2 or ISO 27001 certified. We are GDPR-compliant with EU-only data residency, registered at CSA STAR Level 1 (a published self-assessment), and our control set is modelled on ISO 27001 Annex A. Our datacenter operators hold ISO 27001. We offer SAML SSO, SCIM and audit logs on enterprise plans.
What is Playwright?
Playwright is an open-source end-to-end testing framework built by Microsoft that drives Chromium, Firefox and WebKit through a single API. It supports JavaScript, TypeScript, Python, Java and .NET, with auto-waiting, network interception, fixtures and a built-in test runner. Teams use Playwright for cross-browser testing, visual regression testing, API testing, accessibility testing and mobile-web testing. TestingBot runs Playwright on real browsers, real operating systems and real Android devices in the cloud, so you can scale Playwright testing without managing infrastructure.
Is Playwright free?
Yes, Playwright itself is free. Microsoft releases it under the Apache 2.0 license, so you can install it, write tests and run them locally at no cost. The cost question for teams is usually how to run Playwright at scale across real browsers, devices and parallel workers, which is what TestingBot provides on flat plans with unlimited test minutes.
Is Playwright open source?
Playwright is open source. The framework is maintained by Microsoft on GitHub under the Apache 2.0 license, with a public roadmap and active community. TestingBot complements the open-source framework with a managed cloud grid for real-browser, real-device and parallel Playwright testing.
Can Playwright be used for visual, accessibility and API testing?
Yes. Playwright is a general-purpose automation framework, so the same scripts that drive end-to-end UI tests can also be used for visual regression testing, accessibility audits, API testing and performance checks. On TestingBot you get full HD video, Playwright traces, console and network HAR logs for every session, which makes visual and behaviour comparisons easier.
Is TestingBot Playwright as a service?
Yes. TestingBot is hosted Playwright, or Playwright as a service: you keep your own test code and config, and we run the browsers. There is no grid to provision, no images to patch and no capacity to plan for. Point @playwright/test at our endpoint and video, traces and logs come back with every run.
Can I use TestingBot for Playwright automation in CI?
Yes, and that is the common setup. Playwright automation runs the same way locally and in CI: the same config, credentials read from environment variables, and up to 100 concurrent sessions so a full suite finishes in minutes. Worked examples exist for GitHub Actions, GitLab CI, Jenkins, CircleCI and Azure Pipelines.
## Ready to scale your Playwright testing?
Read our [Playwright Cloud Testing](https://testingbot.com/support/web-automate/playwright) documentation and run your first test in under 5 minutes.
[Start a free trial](https://testingbot.com/users/sign_up)
Resources you may like
[](https://testingbot.com/resources/articles/playwright-visual-regression-testing)
### [Visual Testing with Playwright](https://testingbot.com/resources/articles/playwright-visual-regression-testing)
Playwright provides automated browser testing. It offers a built-in feature to perform visual regression testing for your website.
[Read more: Visual Testing with Playwright](https://testingbot.com/resources/articles/playwright-visual-regression-testing)
[](https://testingbot.com/resources/articles/playwright-recorder)
### [Playwright Codegen: Record Tests in VS Code](https://testingbot.com/resources/articles/playwright-recorder)
How to record Playwright scripts from Visual Studio Code rather than writing them by hand, then run the recorded tests on a browser grid.
[Read more: Playwright Codegen: Record Tests in VS Code](https://testingbot.com/resources/articles/playwright-recorder)
---
URL: https://testingbot.com/features/automation/selenium
Selenium WebDriver
# Selenium Cloud Automation
Run your Selenium WebDriver tests on 6100+ real browsers and physical iOS & Android devices. Massive parallelism, video & log artifacts, and battle-tested infrastructure with a 99.9% uptime SLA.
Comparing frameworks? See [Playwright vs Selenium](https://testingbot.com/playwright-vs-selenium), [Cypress vs Selenium](https://testingbot.com/cypress-vs-selenium) or [Selenium vs Puppeteer](https://testingbot.com/selenium-vs-puppeteer).
[Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo)
- **Browsers & devices:** 6100+
- **Selenium versions:** All
- **Uptime SLA:** 99.9%
_Selenium tests running on Chrome, Firefox, Safari, Edge, Opera, Samsung, iOS and Android via TestingBot's cloud hub_
Trusted by some of the world's most innovative companies
5-line setup
## Point your existing tests at our hub
One URL change is all it takes. Reuse your current Selenium WebDriver code, run it on our cloud, and pick any browser, OS or device through capabilities.
hub.testingbot.com Copy
JavaScript Python Java Ruby C# PHP
```javascript
// Run a Selenium test on the TestingBot grid
const wd = require('selenium-webdriver');
const driver = await new wd.Builder()
.usingServer('https://hub.testingbot.com/wd/hub')
.withCapabilities({
'tb:options': { name: 'Checkout flow' },
browserName: 'chrome',
browserVersion: 'latest',
platformName: 'Windows 11'
})
.build();
await driver.get('https://example.com');
await driver.quit();
```
```python
# Run a Selenium test on the TestingBot grid
from selenium import webdriver
options = webdriver.ChromeOptions()
options.set_capability('tb:options', {'name': 'Checkout flow'})
options.set_capability('browserVersion', 'latest')
options.set_capability('platformName', 'Windows 11')
driver = webdriver.Remote(
command_executor='https://hub.testingbot.com/wd/hub',
options=options
)
driver.get('https://example.com')
driver.quit()
```
```java
// Run a Selenium test on the TestingBot grid
DesiredCapabilities caps = new DesiredCapabilities();
caps.setCapability("browserName", "chrome");
caps.setCapability("browserVersion", "latest");
caps.setCapability("platformName", "Windows 11");
WebDriver driver = new RemoteWebDriver(
new URL("https://hub.testingbot.com/wd/hub"),
caps
);
driver.get("https://example.com");
driver.quit();
```
```ruby
# Run a Selenium test on the TestingBot grid
require 'selenium-webdriver'
caps = Selenium::WebDriver::Remote::Capabilities.chrome(
browserVersion: 'latest',
platformName: 'Windows 11',
'tb:options' => { name: 'Checkout flow' }
)
driver = Selenium::WebDriver.for(:remote,
url: 'https://hub.testingbot.com/wd/hub',
capabilities: caps
)
driver.get 'https://example.com'
driver.quit
```
```csharp
// Run a Selenium test on the TestingBot grid
var options = new ChromeOptions();
options.AddAdditionalOption("browserVersion", "latest");
options.AddAdditionalOption("platformName", "Windows 11");
IWebDriver driver = new RemoteWebDriver(
new Uri("https://hub.testingbot.com/wd/hub"),
options
);
driver.Navigate().GoToUrl("https://example.com");
driver.Quit();
```
```php
// Run a Selenium test on the TestingBot grid
$capabilities = DesiredCapabilities::chrome();
$capabilities->setCapability('browserVersion', 'latest');
$capabilities->setCapability('platformName', 'Windows 11');
$driver = RemoteWebDriver::create(
'https://hub.testingbot.com/wd/hub',
$capabilities
);
$driver->get('https://example.com');
$driver->quit();
```
Connected · [View documentation](https://testingbot.com/support/web-automate/selenium/nodejs) [View documentation](https://testingbot.com/support/web-automate/selenium/python) [View documentation](https://testingbot.com/support/web-automate/selenium/java) [View documentation](https://testingbot.com/support/web-automate/selenium/ruby) [View documentation](https://testingbot.com/support/web-automate/selenium/csharp) [View documentation](https://testingbot.com/support/web-automate/selenium/php)
Test Results
## Rich test results out of the box
Every Selenium session produces a full HD video, step-by-step command log, browser & Selenium logs and screenshots.
Mark sessions as passed or failed from your test code, fetch artifacts via our [REST API](https://testingbot.com/support/api), and ship results back to Jira, Slack or your CI of choice.
Every Selenium session is fully customizable, including:
- All Selenium versions supported (3.x, 4.x, BiDi)
- Custom screen resolution
- Custom timezone
- Change geolocation
- Browser startup flags and [many more options](https://testingbot.com/support/web-automate/selenium/test-options)
Languages & frameworks
## Selenium integrations for every stack
TestingBot integrates with every major Selenium binding and test framework. Drop in your existing tests, no rewrites needed.
- ### [Java](https://testingbot.com/support/web-automate/selenium/java)
- [JUnit](https://testingbot.com/support/web-automate/selenium/java/junit)
- [Parallel JUnit](https://testingbot.com/support/web-automate/selenium/java/parallel-junit)
- [TestNG](https://testingbot.com/support/web-automate/selenium/java/testng)
- [Selenide](https://testingbot.com/support/web-automate/selenium/java/selenide)
- ### [Python](https://testingbot.com/support/web-automate/selenium/python)
- [PyTest](https://testingbot.com/support/web-automate/selenium/python/pytest)
- [PyUnit](https://testingbot.com/support/web-automate/selenium/python/pyunit)
- [Behave](https://testingbot.com/support/web-automate/selenium/python/behave)
- [Helium](https://testingbot.com/support/web-automate/selenium/python/helium)
- ### [JavaScript](https://testingbot.com/support/web-automate/selenium/nodejs)
- [WebDriverIO](https://testingbot.com/support/web-automate/selenium/nodejs/webdriverio)
- [Nightwatch](https://testingbot.com/support/web-automate/selenium/nodejs/nightwatch)
- [Protractor](https://testingbot.com/support/web-automate/selenium/nodejs/protractor)
- [TestCafe](https://testingbot.com/support/web-automate/selenium/nodejs/testcafe)
- ### [Ruby](https://testingbot.com/support/web-automate/selenium/ruby)
- [RSpec](https://testingbot.com/support/web-automate/selenium/ruby/rspec)
- [Cucumber](https://testingbot.com/support/web-automate/selenium/ruby/cucumber)
- [Minitest](https://testingbot.com/support/web-automate/selenium/ruby/minitest)
- [Test::Unit](https://testingbot.com/support/web-automate/selenium/ruby/testunit)
- ### [C#](https://testingbot.com/support/web-automate/selenium/csharp)
- [NUnit](https://testingbot.com/support/web-automate/selenium/csharp/nunit)
- [MSTest](https://testingbot.com/support/web-automate/selenium/csharp/mstest)
- [xUnit](https://testingbot.com/support/web-automate/selenium/csharp/xunit)
- [SpecFlow](https://testingbot.com/support/web-automate/selenium/csharp/specflow)
- ### [PHP](https://testingbot.com/support/web-automate/selenium/php)
- [PHPUnit](https://testingbot.com/support/web-automate/selenium/php/phpunit)
- [Codeception](https://testingbot.com/support/web-automate/selenium/php/codeception)
- [Behat](https://testingbot.com/support/web-automate/selenium/php/behat-mink)
- [SimpleTest](https://testingbot.com/support/web-automate/selenium/php/simpletest)
Looking to run tests without writing code? Record tests with [Selenium IDE](https://testingbot.com/support/web-automate/codeless-automation/add-test) and upload them to our Codeless Automation for scheduled execution.
Massive parallelism
## Cut suite time from hours to minutes
Fan out your Selenium suite across hundreds of single-use VMs running in parallel. Same test code, a fraction of the wall-clock time, all on Windows, Linux and macOS optimised for speed and stability.
More on how concurrency is sized and priced: [parallel testing](https://testingbot.com/features/parallel-testing).
- Up to 100 concurrent Selenium sessions
- No queue, no warm-up. Sessions start in seconds
- Auto-retry, and your test runner's own parallelism works unchanged
[Start free](https://testingbot.com/users/sign_up)
Live exit nodes
10+ countries
United States
United Kingdom
Germany
France
Spain
Belgium
Australia
China
Japan
Geolocation testing
## Test from 10+ countries
Run Selenium tests from real exit nodes around the world. Verify **geo-targeting** , **geo-blocking** and **localisation** from Australia, Belgium, the United Kingdom, the United States, Germany, France, Spain, China and more.
Secure infrastructure
## Real machines. Single-use. Zero leaks.
Each Selenium session runs in a brand-new, single-use virtual machine on a real operating system. The VM is destroyed the moment the test ends, nothing is reused, nothing leaks across customers.
GDPR-compliant, with SAML SSO and audit logs for teams.
- Single-use VMs, wiped after every session
- GDPR-compliant, EU data residency available
- SAML SSO, audit logs and SCIM provisioning
[Visit TestingBot Trust Center](https://trust.testingbot.com/)
1. 1
Test connects to `hub.testingbot.com`
A pristine VM is provisioned in seconds, just for your session.
2. 2
Your Selenium commands run on real hardware
Real Windows, macOS or Linux — never simulated, never shared.
3. 3
VM destroyed the moment your test ends
Cookies, files, browser state — all wiped. Nothing is ever reused.
Continuous integration
## Plug into your CI/CD in minutes
TestingBot integrates with every major CI/CD tool. Trigger Selenium suites on every commit and stream results back into your build dashboards.
### [Jenkins](https://testingbot.com/support/integrations/ci-cd/jenkins)
### [Bamboo](https://testingbot.com/support/integrations/ci-cd/bamboo)
### [TeamCity](https://testingbot.com/support/integrations/ci-cd/teamcity)
### [CircleCI](https://testingbot.com/support/integrations/ci-cd/circleci)
### [GitLab CI](https://testingbot.com/support/integrations/ci-cd/gitlab)
### [GitHub](https://testingbot.com/support/integrations/ci-cd/github-actions)
Mobile Selenium
## Mobile web Selenium on real devices
Drive Mobile Safari on real iPhones and iPads, and Chrome on real Android devices using the same Selenium WebDriver protocol. Real hardware, real sensors, accurate results.
[View all devices](https://testingbot.com/support/app-automate/devices)
iPhone 13 Pro
Pixel 8
Pixel 11
iPhone 17 Pro
iPhone 17 Pro Max
iPhone 16 Pro
Secure Tunnel
## Test internal & staging apps securely
Use our [secure tunnel](https://testingbot.com/support/tunnel) to run Selenium tests against firewalled staging sites, internal dashboards or apps still on localhost.
Encrypted end-to-end, no inbound firewall rules required, and built-in support for popular CI providers.
[Read more](https://testingbot.com/support/tunnel)
Buy vs build
## TestingBot vs a self-hosted Selenium grid
Running your own Selenium grid sounds cheap until you account for the maintenance, the flakiness and the missing devices. Here is how a hosted Selenium cloud compares.
| Capability | TestingBot | Self-hosted Selenium Grid |
| --- | --- | --- |
| Browser coverage | 6100+ browsers, all major versions, refreshed weekly | Limited to what you provision and patch yourself |
| Time to first test | Minutes, change one URL in your code | Days to weeks of grid setup and tuning |
| Real iOS & Android | Real physical devices on demand | Requires a separate device lab |
| Parallel scaling | Up to 100 concurrent sessions, on demand | Capped by your hardware budget |
| Video & logs | Video, network, browser & Selenium logs included | Roll your own recording & log aggregation |
| Reliability | 99.9% uptime SLA, 24/7 monitoring | You are the on-call engineer |
| Support | 24/7 human support and dedicated CSM | Stack Overflow and your own logs |
FAQ
## Selenium on TestingBot: frequently asked questions
What is a cloud-based Selenium grid?
A cloud-based Selenium grid is a managed pool of browser nodes that you connect to via the `https://hub.testingbot.com/wd/hub` endpoint. Your existing Selenium WebDriver code stays the same, only the remote URL and a few capabilities change. TestingBot handles the OS images, browser installs, scaling and recording for you.
Which Selenium versions does TestingBot support?
TestingBot supports every Selenium version still in active use, from Selenium 2.x and 3.x through Selenium 4 and Selenium BiDi. You can pin a specific server version per-test using the `se:options` or `tb:options` capability.
Can I run Selenium tests in parallel?
Yes. Plans on TestingBot include up to 100 parallel Selenium sessions. Each session runs in its own single-use VM, so tests do not interfere with one another and parallelism does not introduce flakiness.
Does TestingBot work with Selenium 4 and BiDi?
Fully. Our hub speaks Selenium 4 W3C WebDriver and supports the new [Selenium BiDi protocol](https://testingbot.com/support/web-automate/selenium/selenium-bidi) for advanced features like real-time logs, network interception and authentication handling.
How do I migrate existing Selenium tests to TestingBot?
Change the remote URL to `https://hub.testingbot.com/wd/hub`, add your TestingBot credentials, and pick a browser/OS via capabilities. See our [migration guide](https://testingbot.com/support/web-automate/selenium/migrate-existing-tests) for step-by-step examples in every language.
Can I test internal or staged sites?
Yes, install our [secure tunnel](https://testingbot.com/support/tunnel) and your Selenium tests can reach staging environments, localhost servers or anything else behind your firewall, with no inbound rules required.
Does TestingBot record videos and logs of my Selenium tests?
Every Selenium session is recorded as full HD video alongside step-by-step Selenium commands, browser console logs, network HAR files and screenshots. Artifacts are accessible from the dashboard or via the REST API for CI integrations.
Is the Selenium grid SOC 2 / GDPR compliant?
TestingBot is not SOC 2 or ISO 27001 certified. We are GDPR-compliant with EU-only data residency, registered at CSA STAR Level 1 (a published self-assessment), and our control set is modelled on ISO 27001 Annex A. Our datacenter operators hold ISO 27001. We offer SAML SSO, SCIM and audit logs on enterprise plans.
Do I still need to run my own Selenium Grid?
No. A cloud Selenium Grid gives you the same hub-and-node model without any hub, nodes or driver upgrades to maintain. You point your existing WebDriver code at our endpoint, and we handle browser installs, OS images, scaling and cleanup between sessions.
Is TestingBot a Selenium testing tool or hosted infrastructure?
Both. As a Selenium testing tool it adds video, console and network artifacts, live debugging and a REST API on top of plain WebDriver. As infrastructure it runs the grid itself, so the tooling and the machines come from one place.
## More reasons to love TestingBot
### 24/7 support
Got questions? Ask in the 24/7 in-app customer [chat support](https://testingbot.com#) or [send us an email](https://testingbot.com/contact/new). We are here to help you!
TestingBot Support
Hey 👋 How can we help?
I need support
### Trusted by over 14,000 teams
6M+
Tests
120k+
Users
400+
Enterprises
106
Countries
[Request a demo](https://testingbot.com/demo)
### Developer Documentation
Everything you need to integrate TestingBot into your workflow. Browse guides, API references and SDKs for web and mobile testing.
- [Selenium Quickstart Documentation](https://testingbot.com/support/web-automate/selenium)
- [Selenium 4](https://testingbot.com/support/web-automate/selenium/selenium4)
- [Selenium BiDi](https://testingbot.com/support/web-automate/selenium/selenium-bidi)
- [CI/CD integration](https://testingbot.com/support/integrations/ci-cd)
[Explore Documentation](https://testingbot.com/support)
### Enterprise-grade Security
STAR Level 1
Certified
GDPR
Compliant
SAML SSO
Single Sign-On
99.9%
Uptime SLA
[Visit TestingBot Trust Center](https://trust.testingbot.com/)
## Ready to scale your Selenium testing?
Read our [Selenium Cloud Testing](https://testingbot.com/support/web-automate/selenium) documentation and run your first test in under 5 minutes.
[Start a free trial](https://testingbot.com/users/sign_up)
Resources you may like
[](https://testingbot.com/resources/articles/generative-ai-selenium)
### [Selenium and Generative AI](https://testingbot.com/resources/articles/generative-ai-selenium)
How to use generative AI to produce realistic test data for Selenium runs, and why varied data uncovers bugs that fixed fixtures never will.
[Read more: Selenium and Generative AI](https://testingbot.com/resources/articles/generative-ai-selenium)
[](https://testingbot.com/blog/selenium-4)
### [Selenium 4: what's new](https://testingbot.com/blog/selenium-4)
Learn more about what is new Selenium 4.
[Read more: Selenium 4: what's new](https://testingbot.com/blog/selenium-4)
[](https://testingbot.com/blog/webdriver-bidi)
### [Selenium WebDriver BiDi](https://testingbot.com/blog/webdriver-bidi)
WebDriver BiDi support is now available on TestingBot. Learn how to use this exciting new improvement with your Selenium tests.
[Read more: Selenium WebDriver BiDi](https://testingbot.com/blog/webdriver-bidi)
---
URL: https://testingbot.com/features/automation/cypress
Cypress cloud
# Run Cypress on a real browser cloud
Point your existing Cypress suite at TestingBot's grid and run it across real Chrome, Firefox and Edge on Windows, macOS and Linux. Massive parallelism, video & log artifacts, GDPR-compliant with EU data residency.
Comparing frameworks? See [Cypress vs Selenium](https://testingbot.com/cypress-vs-selenium) and [Cypress vs Playwright](https://testingbot.com/cypress-vs-playwright).
[Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo)
- **Browsers & OSes:** Chrome · Firefox · Edge
- **Parallel sessions:** 100
- **Uptime SLA:** 99.9%
Trusted by some of the world's most innovative companies
30-second answer
## Why run Cypress on TestingBot?
Real cross-browser coverage
Cypress runs on real Chrome, Firefox and Edge in our cloud, on real Windows, macOS and Linux. Same specs, every browser your customers use.
Massive parallelism
Fan out to up to 100 concurrent Cypress sessions, each in its own single-use VM. Cut suite time from hours to minutes.
Drop-in CLI swap
Install our open-source `testingbot-cypress-cli`, point it at your `cypress.config.js`, and run.
What is Cypress?
## Cypress in 60 seconds
### Cypress
Open source · MIT · JavaScript / TypeScript
Cypress is a JavaScript end-to-end testing framework that runs in the browser, in the same JavaScript runtime as your app. Tests get direct access to every object, the DOM, network requests, timers and the framework's own state, without the cross-process bridge that WebDriver-based tools use.
That gives Cypress its trademark fast feedback loop, automatic waiting, time-travel debugging and built-in spy / stub APIs. It is the canonical choice for JavaScript-first teams that want a fast feedback loop and modern DX.
- JavaScript / TypeScript-first, runs in-browser
- Real Chrome, Firefox and Edge
- Auto-wait, time-travel debugger, network stubbing
- Built-in spy, stub and clock APIs
- Parallel via the testingbot-cypress-cli
Two-line CLI
## Run Cypress on the cloud with one command
Install our open-source CLI, run testingbot-cypress init to generate a config, then testingbot-cypress run to fan out across real browsers in parallel.
install & init step 1
```bash
# Install the open-source CLI
$ npm install testingbot-cypress-cli
# Generate testingbot.json (one-off)
$ testingbot-cypress init
# Edit the generated testingbot.json:
# "cypress_project_dir": "./cypress",
# "browsers": ["chrome", "firefox", "edge"],
# "build_name": "ci-build-${BUILD_NUMBER}"
```
run on the cloud step 2
```bash
# Run your specs on real cloud browsers
$ testingbot-cypress run --parallel 10
# Pin a specific supported Cypress version
$ testingbot-cypress run \
--cypress-version "14.2.0"
# → live results in your TestingBot dashboard
# → video and console logs via REST API
```
See the full [Cypress on TestingBot documentation](https://testingbot.com/support/web-automate/cypress) for advanced flags, parallel options and CI integration.
Local vs cloud
## Local Cypress vs TestingBot
Cypress is fast on a laptop until you need scale, multiple browsers or real WebKit. Here is how a hosted cloud compares.
| Capability | TestingBot | Local Cypress |
| --- | --- | --- |
| Browser coverage | Real Chrome, Firefox and Edge | Whatever you have installed locally |
| Operating systems | Real Windows, macOS and Linux | Your single dev machine |
| Parallel scaling | Up to 100 concurrent sessions, on demand | Capped by your CPU and RAM |
| Video & logs | HD video and console logs included | Built-in video, but no central dashboard |
| Test private & staging environments | Built-in secure tunnel | Direct access only |
| EU data residency | Yes, GDPR-compliant | Whatever country your laptop is in |
| Reliability | 99.9% uptime SLA, 24/7 monitoring | Crashes when you close the lid |
Cypress Cloud alternative
## Looking for a Cypress Cloud alternative?
Cypress Cloud, formerly the Cypress Dashboard, records and orchestrates runs on machines you provide. TestingBot provides the machines too: the same parallel runs, video and logs, plus the browsers and operating systems to run them on.
| What you are comparing | TestingBot | Cypress Cloud |
| --- | --- | --- |
| What it provides | Browsers, operating systems and the recording | Recording, orchestration and analytics |
| Where your tests run | On our single-use VMs, wiped after every run | On CI runners you supply and maintain |
| How it is billed | By the parallel sessions on your plan | Per recorded test result |
| Cross-browser coverage | Chrome, Firefox, Edge and WebKit on Windows and macOS | Whatever your own runners have installed |
### When Cypress Cloud is the better fit
If you already have all the CI capacity you need, and what you want is Test Replay, flake detection and Cypress-specific analytics layered on top of it, Cypress Cloud is built for exactly that and we do not try to replicate it. Teams tend to come to us when the constraint is machines and browser coverage rather than reporting.
Cypress Cloud details from cypress.io/pricing, checked 20 August 2026.
Test artifacts
## Video, screenshots and logs out of the box
Every Cypress run ships with a full HD video, browser console log and screenshots. Inspect a failure without re-running the suite.
Mark sessions as passed or failed from your tests, fetch artifacts via our [REST API](https://testingbot.com/support/api), and ship results back to Jira, Slack or your CI of choice.
- Full HD video for every spec run
- Browser console output and Cypress logs
- Screenshots on demand
- Custom screen resolution, timezone and locale
- Browser startup flags and [many more capabilities](https://testingbot.com/support/web-automate/cypress/capabilities)
Capabilities
## Everything Cypress needs in the cloud
Beyond running tests, TestingBot exposes the full set of Cypress capabilities you would set in your config, plus cloud-only features like geolocation and timezone.
- ### [Capabilities](https://testingbot.com/support/web-automate/cypress/capabilities)
Browser, OS, version, screen resolution and every capability the cloud accepts.
- ### [Authentication](https://testingbot.com/support/web-automate/cypress/authentication)
Pass credentials, environment variables and login fixtures into your specs.
- ### [Specs & filters](https://testingbot.com/support/web-automate/cypress/specs)
Pick which specs run per build, group by tag or shard automatically.
- ### [TypeScript](https://testingbot.com/support/web-automate/cypress/typescript)
Run TypeScript-based Cypress specs out of the box, no extra setup.
- ### [Geolocation & timezone](https://testingbot.com/support/web-automate/cypress/geolocation)
Spoof GPS and timezone per session for App Store, store-front and back-end checks.
- ### [Reporting](https://testingbot.com/support/web-automate/cypress/reporting)
JUnit XML, mochawesome and integrations with Jenkins, GitHub and GitLab.
Massive parallelism
## Cut Cypress suite time from hours to minutes
Run hundreds of Cypress sessions in parallel across single-use VMs. Same specs, a fraction of the wall-clock time, with smart spec sharding handled by the CLI.
More on how concurrency is sized and priced: [parallel testing](https://testingbot.com/features/parallel-testing).
- Up to 100 concurrent Cypress sessions
- Sessions start in seconds, no warm-up
- Smart spec sharding via testingbot-cypress-cli
[Start free](https://testingbot.com/users/sign_up)
Live exit nodes
10+ countries
United States
United Kingdom
Germany
France
Spain
Belgium
Australia
China
Japan
Geolocation testing
## Test Cypress flows from 10+ countries
Run Cypress tests from real exit nodes around the world. Verify **geo-targeting** , **geo-blocking** and **localisation** from the United States, the United Kingdom, Germany, France, Spain, Belgium, Australia, China, Japan and more.
Secure infrastructure
## Real machines. Single-use. Zero leaks.
Each Cypress session runs in a brand-new, single-use virtual machine on a real operating system. The VM is destroyed the moment your run ends, nothing is reused, nothing leaks across customers.
GDPR-compliant with EU data residency, SAML SSO and audit logs available for enterprise teams.
- Single-use VMs, wiped after every run
- GDPR-compliant, EU data residency available
- SAML SSO, audit logs and SCIM provisioning
[Visit TestingBot Trust Center](https://trust.testingbot.com/)
1. 1
Test connects to the TestingBot grid
A pristine VM is provisioned in seconds, just for your session.
2. 2
Cypress drives a real browser
Real Chrome, Firefox or Edge, never simulated, never shared.
3. 3
VM destroyed the moment your run ends
Cookies, files, browser state, all wiped. Nothing is ever reused.
Continuous integration
## Plug Cypress into your CI/CD in minutes
TestingBot integrates with every major CI/CD tool. Trigger Cypress runs on every commit and stream videos and logs back into your build dashboards.
### [Jenkins](https://testingbot.com/support/integrations/ci-cd/jenkins)
### [Bamboo](https://testingbot.com/support/integrations/ci-cd/bamboo)
### [TeamCity](https://testingbot.com/support/integrations/ci-cd/teamcity)
### [CircleCI](https://testingbot.com/support/integrations/ci-cd/circleci)
### [GitLab CI](https://testingbot.com/support/integrations/ci-cd/gitlab)
### [GitHub](https://testingbot.com/support/integrations/ci-cd/github-actions)
Secure Tunnel
## Test internal & staging apps securely
Use our [secure tunnel](https://testingbot.com/support/tunnel) to run Cypress tests against firewalled staging sites, internal dashboards or apps still on localhost.
Encrypted end-to-end, no inbound firewall rules required, ready out of the box for popular CI providers.
[Read more](https://testingbot.com/support/tunnel)
FAQ
## Cypress on TestingBot: frequently asked questions
Why run Cypress on TestingBot instead of locally?
Locally Cypress runs on whatever browser you have installed. TestingBot gives you up to 100 parallel Cypress sessions on real Chrome, Firefox and Edge across real Windows, macOS and Linux machines, with HD video and console logs collected automatically.
Which Cypress versions are supported?
TestingBot currently supports Cypress `14.2.0`, `13.17.0`, `13.3.3` (default), `12.5.1`, `10.0.0`, `9.1.0` and `8.0.0`. Pin a specific version per-run via the `cypress_version` field in your `testingbot.json`.
Can I run Cypress in parallel?
Yes. Plans on TestingBot include up to 100 parallel Cypress sessions, with automatic spec sharding handled by the testingbot-cypress-cli. Each session runs in its own single-use VM.
How is TestingBot different from Cypress Cloud?
TestingBot runs Cypress on real Chrome, Firefox and Edge across real Windows, macOS and Linux. Cypress Cloud is a results dashboard, not a browser cloud. The two can coexist, many teams record to Cypress Cloud and execute on TestingBot.
Does TestingBot support TypeScript and the Cypress component runner?
Yes, see the [TypeScript guide](https://testingbot.com/support/web-automate/cypress/typescript). Component runs work the same way as e2e specs.
How do I migrate my existing Cypress suite?
Install our open-source `testingbot-cypress-cli`, run `testingbot-cypress init`, point it at your `cypress.config.js`, and run `testingbot-cypress run`. Your specs, plugins and fixtures stay unchanged.
Can I test internal or staged sites?
Yes, install our [secure tunnel](https://testingbot.com/support/tunnel) and your Cypress tests can reach staging environments, localhost or anything else behind your firewall.
Is the Cypress grid GDPR / SOC 2 compliant?
TestingBot is not SOC 2 or ISO 27001 certified. We are GDPR-compliant with EU-only data residency, registered at CSA STAR Level 1 (a published self-assessment), and our control set is modelled on ISO 27001 Annex A. Our datacenter operators hold ISO 27001. We offer SAML SSO, SCIM and audit logs on enterprise plans.
Is TestingBot a Cypress Cloud alternative?
For the parts most teams pay for, yes. TestingBot runs your specs in parallel, records video and logs and gives you a dashboard, and it supplies the browsers and machines as well, which Cypress Cloud does not. It is billed by parallel sessions rather than per recorded test result. If you specifically want Test Replay or Cypress-native flake analytics on your own CI, Cypress Cloud remains the right tool.
Which browsers can I run Cypress on?
Chrome, Chromium, Firefox, Edge, Electron and WebKit, on Windows and macOS. That covers cross-browser Cypress testing you cannot easily get on a single CI runner, including WebKit for Safari-like rendering, and headless or headed runs of any of them.
## Ready to scale your Cypress testing?
Read our [Cypress Cloud Testing](https://testingbot.com/support/web-automate/cypress) documentation and run your first spec in under 5 minutes.
[Start a free trial](https://testingbot.com/users/sign_up)
Resources you may like
[](https://testingbot.com/resources/articles/how-to-debug-cypress-tests)
### [Tutorial on debugging Cypress tests](https://testingbot.com/resources/articles/how-to-debug-cypress-tests)
How to debug a failing Cypress test using the debugger, console logs, the stack trace, and the debug and pause commands.
[Read more: Tutorial on debugging Cypress tests](https://testingbot.com/resources/articles/how-to-debug-cypress-tests)
[](https://testingbot.com/resources/articles/cypress-cucumber-testing)
### [Cypress and Cucumber Browser Testing](https://testingbot.com/resources/articles/cypress-cucumber-testing)
How to combine Cypress with Cucumber so end-to-end tests read as plain language, including a worked example and the TypeScript setup.
[Read more: Cypress and Cucumber Browser Testing](https://testingbot.com/resources/articles/cypress-cucumber-testing)
[](https://testingbot.com/blog/testingbot-cypress)
### [Run Cypress tests on TestingBot's Browser Grid](https://testingbot.com/blog/testingbot-cypress)
Run Cypress Automated tests on TestingBot's Browser grid.
[Read more: Run Cypress tests on TestingBot's Browser Grid](https://testingbot.com/blog/testingbot-cypress)
---
URL: https://testingbot.com/features/automation/espresso
Espresso cloud
# Run Espresso on real Android devices
Run your Espresso suite on real Galaxy, Pixel, OnePlus and Xiaomi devices from Android 4.4 through Android 15 with a single CLI command. Massive parallelism, video & log artifacts and a 99.9% uptime SLA.
Choosing a framework? See [Appium vs Espresso](https://testingbot.com/appium-vs-espresso), or the [full automation framework lineup](https://testingbot.com/features).
[Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo)
- **Real Android devices:** 200+
- **Android versions:** 4.4 → 15
- **Uptime SLA:** 99.9%
Trusted by some of the world's most innovative companies
30-second answer
## Why run Espresso on TestingBot?
Real Android devices
Run on a fleet of real Galaxy, Pixel, OnePlus and Xiaomi devices, not just emulators. Latest flagships on day one, plus a deep legacy fleet from Android 4.4.
Replace your in-house Android lab
Skip the cabinet of dusty phones, the swollen batteries and the OEM update treadmill. TestingBot has run Android tests on real devices for 5+ years.
Native Java & Kotlin
Use the same Espresso framework Google ships with Android Studio. Same APIs, same JUnit assertions, real native performance.
What is Espresso?
## Espresso in 60 seconds
### Espresso
Google · built into Android Studio · Java / Kotlin
Espresso is Google's official UI testing framework for Android, built into Android Studio and the Android SDK. It uses the AndroidJUnitRunner instrumentation runner, and lets you script taps, swipes, scrolls, intents and view assertions against your app's real UI.
Because Espresso runs in the same process as your app, tests are fast and reliable, with automatic synchronisation between the test and the UI thread. It is the canonical choice for fast, in-process Android UI tests.
- Native Google framework, built into Android Studio
- Java and Kotlin bindings
- Real Galaxy, Pixel, OnePlus, Xiaomi and emulators
- View matchers, view actions, view assertions, intents
- AndroidJUnitRunner, JUnit XML output for CI
One-line CLI
## Run Espresso on real devices with one command
Install the TestingBot CLI, point it at your app APK and your test APK, pick a device — done. Uploads, capabilities and result polling are all handled for you.
install the CLI step 1
```bash
# Install the TestingBot CLI
$ npm install -g testingbot-cli
# Authenticate (once) — set your KEY and SECRET
$ export TESTINGBOT_KEY=your-key
$ export TESTINGBOT_SECRET=your-secret
# Build app + Espresso test APKs (one-off)
$ ./gradlew assembleDebug \
assembleDebugAndroidTest
```
run Espresso step 2
```bash
# Run Espresso on a real Pixel — one command
$ testingbot espresso app.apk app-test.apk \
--device "Pixel 8" \
--platform-version "14" \
--real-device
# Or, on a Galaxy
$ testingbot espresso app.apk app-test.apk \
--device "Galaxy S24" \
--real-device
# → live results in your TestingBot dashboard
# → JUnit XML, video and logcat via REST API
```
See the full [Espresso on TestingBot documentation](https://testingbot.com/support/app-automate/espresso) for advanced flags, parallel runs and CI integration.
Buy vs build
## TestingBot vs an in-house Android device lab
Maintaining your own Android device lab is expensive. Devices break, OEMs ship updates on their own schedule, and Samsung's Knox controls keep changing. Here is how a hosted Android device cloud compares.
| Capability | TestingBot | In-house Android lab |
| --- | --- | --- |
| Device coverage | Galaxy, Pixel, OnePlus, Xiaomi from Android 4.4 to 15 | Limited to what you can buy and rack |
| Time to first test | Minutes, one CLI command and you are running | Weeks of provisioning, ADB, MDM and tethering |
| Parallel scaling | Up to 100 concurrent Espresso sessions | Capped by the number of devices you own |
| Video & logs | Video, logcat, screenshots and JUnit XML | Roll your own recording & log aggregation |
| Reliability | 99.9% uptime SLA, 24/7 monitoring | You replace dead batteries and broken screens |
| Security | Devices wiped between every session | Manual cleanup, leftover state risks |
| Support | 24/7 human support and dedicated CSM | Stack Overflow and your own logs |
Capabilities
## Everything Espresso needs in the cloud
Beyond running tests, TestingBot exposes the full set of Espresso capabilities you would set in Android Studio, plus cloud-only features like geolocation and network throttling.
- ### [Instrumentation](https://testingbot.com/support/app-automate/espresso/instrumentation)
Configure AndroidJUnitRunner instrumentation arguments per run.
- ### [Test filters](https://testingbot.com/support/app-automate/espresso/filter)
Run a subset of your Espresso suite without rebuilding the bundle.
- ### [Test reports](https://testingbot.com/support/app-automate/espresso/test-reports)
Video, logcat, screenshots and JUnit XML for every run.
- ### [Screenshots](https://testingbot.com/support/app-automate/espresso/screenshots)
Take in-test screenshots and pull them via the REST API.
- ### [Network throttling](https://testingbot.com/support/app-automate/espresso/simulate-network-conditions)
Simulate 3G, 4G, lossy WiFi and offline conditions per session.
- ### [Localization & geo](https://testingbot.com/support/app-automate/espresso/set-localization-options)
Set device locale, language and GPS coordinates for store-front and back-end checks.
Real Android devices
## Galaxy, Pixel, OnePlus & Xiaomi on demand
A fleet of real Android devices spanning Android 4.4 through Android 15, refreshed every release. Latest flagship phones available the day OEMs ship them.
[View all devices](https://testingbot.com/support/app-automate/devices)
Pixel 8
Pixel 11
Galaxy S26
Huawei P40
Pixel 10
Pixel 9
Test reports
## Video, screenshots and JUnit XML for every run
Every Espresso run ships with a video screencast, logcat output, viewport screenshots and a JUnit XML report compatible with every major CI parser.
Mark sessions as passed or failed from your tests, fetch artifacts via our [REST API](https://testingbot.com/support/api), and ship results into Jira, Slack or your CI dashboard.
- Full HD video for every test run
- logcat output, app logs and crash reports
- Viewport screenshots on demand
- JUnit XML for Jenkins, GitHub Actions, GitLab CI
Continuous integration
## Plug Espresso into your CI/CD in minutes
TestingBot integrates with every major CI/CD tool. Trigger Espresso runs on every build and stream device videos and JUnit reports back into your build dashboards.
### [Jenkins](https://testingbot.com/support/integrations/ci-cd/jenkins)
### [Bamboo](https://testingbot.com/support/integrations/ci-cd/bamboo)
### [TeamCity](https://testingbot.com/support/integrations/ci-cd/teamcity)
### [CircleCI](https://testingbot.com/support/integrations/ci-cd/circleci)
### [GitLab CI](https://testingbot.com/support/integrations/ci-cd/gitlab)
### [GitHub](https://testingbot.com/support/integrations/ci-cd/github-actions)
Spoof device GPS
20+ regions
United States
United Kingdom
Germany
France
Spain
Belgium
Australia
China
Japan
Geolocation testing
## Test your Android app from 20+ regions
Spoof the device GPS to anywhere in the world and verify **localisation** , **geo-targeting** and **Play Store store-front rules** from any country your customers live in.
Secure devices
## Real devices. Wiped after every session.
Each Espresso session locks one real Android device for your test only. The moment your test ends, the device is wiped, all installed apps are deleted, caches are cleared, and the device is returned to the pool ready for the next customer.
GDPR-compliant, with SAML SSO and audit logs available for enterprise teams.
- One real device per session, never shared
- Wiped between every session
- Encrypted .apk uploads with per-app retention controls
[Visit TestingBot Trust Center](https://trust.testingbot.com/)
1. 1
Your Espresso run locks one real device
A pristine Galaxy, Pixel, OnePlus or Xiaomi is reserved for your session only.
2. 2
Tests run natively on real Android hardware
Real sensors, real fingerprint reader, real OEM customisations.
3. 3
Device wiped on release
Apps removed, caches cleared, accounts wiped. Returned clean to the pool.
Secure Tunnel
## Test apps that talk to internal backends
Use our [secure tunnel](https://testingbot.com/support/tunnel) so Espresso runs can reach private APIs, staging environments and services running on localhost.
Encrypted end-to-end, no inbound firewall rules required, ready out of the box for popular CI providers.
[Read more](https://testingbot.com/support/tunnel)
FAQ
## Espresso on TestingBot — frequently asked questions
What is Espresso and why run it on TestingBot?
Espresso is Google's native UI testing framework for Android. TestingBot runs your Espresso builds on a fleet of real Galaxy, Pixel, OnePlus and Xiaomi devices, so you do not have to maintain devices, OEM updates and ADB tunnels in-house. Upload your `.apk` and `.apk` test bundle through the TestingBot CLI and trigger runs in seconds.
Which Android versions are supported?
TestingBot supports real Android devices from Android 4.4 through Android 15, refreshed every Google and OEM release. We carry the latest flagship Galaxy, Pixel and OnePlus phones plus a deep legacy fleet for backwards-compat coverage.
Can I run on real devices and emulators?
Yes. Plans on TestingBot include both real Android devices and Android emulators with Google Play APIs. Toggle per-test with the --real-device flag in the TestingBot CLI.
Can I run Espresso in parallel?
Yes. Plans on TestingBot include up to 100 parallel Espresso sessions on real devices and emulators. Each session locks a dedicated device, so tests do not interfere with one another.
Can I filter which tests run?
Yes — use the [test filters API](https://testingbot.com/support/app-automate/espresso/filter) to run a subset of your Espresso suite without rebuilding the bundle.
How do I see results?
Live in the TestingBot dashboard with HD video, logcat output and screenshots. Programmatically via the [REST API](https://testingbot.com/support/api). Plus a JUnit XML report compatible with every major CI parser.
Can Espresso reach my staging backend?
Install our [secure tunnel](https://testingbot.com/support/tunnel) and your Espresso runs can reach staging APIs, internal services and localhost without any inbound firewall rules.
Is the device cloud GDPR / SOC 2 compliant?
TestingBot is not SOC 2 or ISO 27001 certified. We are GDPR-compliant with EU-only data residency, registered at CSA STAR Level 1 (a published self-assessment), and our control set is modelled on ISO 27001 Annex A. Our datacenter operators hold ISO 27001. Devices are wiped between sessions, app uploads are encrypted, and SAML SSO and audit logs are available on enterprise plans.
## Ready to scale your Espresso runs?
Read our [Espresso documentation](https://testingbot.com/support/app-automate/espresso) and run your first test on a real Android device in under 5 minutes.
[Start a free trial](https://testingbot.com/users/sign_up)
Resources you may like
[](https://testingbot.com/resources/articles/android-espresso-testing)
### [Android Espresso Tutorial](https://testingbot.com/resources/articles/android-espresso-testing)
Why Espresso suits Android UI testing, how to add it to a project and write a first test, and how it compares with the alternatives.
[Read more: Android Espresso Tutorial](https://testingbot.com/resources/articles/android-espresso-testing)
---
URL: https://testingbot.com/features/automation/maestro
Maestro Cloud alternative, drop-in CLI
# Maestro Cloud Testing on Real iOS and Android Devices
TestingBot Maestro Cloud runs your Maestro flow files on real iOS and Android devices, simulators and emulators. Every test executes 2x faster than Maestro Cloud at a third of the cost, with a single CLI swap.
[Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo)
30 days free for Maestro users
- **Real devices & sims:** 200+
- **Faster than Maestro Cloud:** 2x
- **Cost vs Maestro Cloud:** 1/3
https://www.youtube.com/embed/MMWCkBgXFOU
Trusted by some of the world's most innovative companies
Drop-in CLI swap
## A drop-in replacement for Maestro Cloud
Switch from Maestro Cloud CLI to TestingBot in seconds. Replace `maestro cloud` with `testingbot maestro`. Same flows, same YAML, same arguments.
Maestro Cloud
`maestro cloud android.apk flows/`
TestingBot
`testingbot maestro android.apk flows/`
### Quick start
```bash
$ npm install -g testingbot-cli
$ testingbot maestro app.apk ./flows \
--device "Pixel 8" \
--deviceVersion "14"
```
### Advanced options
```bash
$ testingbot maestro app.apk ./flows \
-e API_URL=https://staging.example.com \
--include-tags "smoke,critical" \
--real-device
```
### CLI options
| Option | Description |
| --- | --- |
| --device | Device name (e.g. "Pixel 8", "iPhone 15") |
| --real-device | Target a physical device (iOS or Android) |
| --deviceVersion | OS version (e.g. "14" for Android, "17.0" for iOS) |
| --maestro-version | Pin a Maestro version (e.g. "2.0.10") |
| -e, --env | Pass environment variables to your tests |
| --include-tags | Run only flows with the given tags |
| --exclude-tags | Skip flows with the given tags |
| --name | Custom name for your test run |
| --locale | Set the device locale (e.g. "de\_DE", "fr\_FR") |
[View all CLI options](https://testingbot.com/support/app-automate/maestro/options)
[Get started free](https://testingbot.com/users/sign_up)
How it works
## From local CLI to cloud results in three steps
1
### Upload your mobile app
Push your .apk, .aab or .ipa via the TestingBot CLI or REST API. Builds are stored in your account for fast re-runs.
2
### Point at your Maestro flows
Reuse the same Maestro flow YAML you run locally. No rewrites, no plugins.
3
### Watch results stream in
Flows run in parallel across real devices and surface in your TestingBot dashboard with live video, logs and step-by-step screenshots.
Side-by-side
## TestingBot vs Maestro Cloud vs DeviceCloud
A side-by-side look at the leading Maestro cloud providers. TestingBot is the cheaper, faster real device cloud with iPhones, iPads, Galaxy and Pixel devices on demand.
| Feature | TestingBot ✨ Best value | Maestro Cloud | DeviceCloud |
| --- | --- | --- | --- |
| 💰 Pricing | 1/3 the cost | 3x more expensive | Pay per test run |
| 📱 Real physical devices (iOS and Android) | | | |
| 📲 iPhone & iPad simulators | | | Pay extra for specific iOS versions |
| 🤖 Android emulators with Google Play APIs | Full support | Limited | Pay extra |
| 💳 In-app purchase testing | Available on Android | | |
| 🔄 Portrait & landscape mode | Full control | Basic | |
| 🌍 Geolocation testing | Any location | | |
| 🌐 Device locale testing | 20+ locales | Limited | 9 locales |
| 📹 Real-time video streaming | Live streaming | Post-test only | Live streaming |
| 📊 Real-time logs | Live streaming | Basic | Live streaming |
| ⚡ Performance | 2x faster (Mac Silicon + ARM) | Standard speed | Standard speed |
[Start testing with TestingBot](https://testingbot.com/users/sign_up)
[Request a demo](https://testingbot.com/demo) or [view pricing](https://testingbot.com/pricing)
Deep dive: [Maestro Cloud alternative](https://testingbot.com/maestro-cloud-alternative) · [devicecloud.dev alternative](https://testingbot.com/devicecloud-alternative)
Massive parallelism
## Cut Maestro suite time from hours to minutes
Run hundreds of Maestro flows in parallel across real devices, simulators and emulators. Same flows, a fraction of the wall-clock time.
- Up to 100 concurrent Maestro sessions
- Sessions start in seconds, no device queue
- Auto-retry, and your own flow sharding works unchanged
[Start free](https://testingbot.com/users/sign_up)
Real-time insights
## Watch flows execute in real time
Stream live video, see steps as they run, and pull logs from the TestingBot dashboard or via the REST API. No waiting for tests to finish.
[Start free](https://testingbot.com/users/sign_up)
Spoof device GPS
20+ regions
United States
United Kingdom
Germany
France
Spain
Belgium
Australia
China
Japan
Geolocation testing
## Test your app from 20+ regions
Spoof the device GPS to anywhere in the world and verify **localisation** , **geo-targeting** and **store-front rules** from any country your customers live in.
Real device cloud
## A real device cloud for Maestro flows
TestingBot is a real device cloud built for Maestro. Test on real iPhones, iPads, Galaxy and Pixel devices, plus iOS Simulators and Android Emulators with Google Play APIs. iOS 16 to 26, Android 10 to 16. Latest flagships on day one.
[View all devices](https://testingbot.com/support/app-automate/devices)
iPhone 13 Pro
Pixel 8
Pixel 11
iPhone 17 Pro
iPhone 17 Pro Max
iPhone 16 Pro
Continuous integration
## Plug Maestro into your CI/CD in minutes
Trigger Maestro flows on every commit and stream device videos and logs back into Jenkins, GitHub Actions, GitLab CI, CircleCI and more.
### [Jenkins](https://testingbot.com/support/integrations/ci-cd/jenkins)
### [Bamboo](https://testingbot.com/support/integrations/ci-cd/bamboo)
### [TeamCity](https://testingbot.com/support/integrations/ci-cd/teamcity)
### [CircleCI](https://testingbot.com/support/integrations/ci-cd/circleci)
### [GitLab CI](https://testingbot.com/support/integrations/ci-cd/gitlab)
### [GitHub](https://testingbot.com/support/integrations/ci-cd/github-actions)
Secure Tunnel
## Test apps that talk to internal backends
Use our [secure tunnel](https://testingbot.com/support/tunnel) so Maestro flows can reach private APIs, staging environments and services running on localhost.
Encrypted end-to-end, no inbound firewall rules required, ready out of the box for popular CI providers.
[Read more](https://testingbot.com/support/tunnel)
FAQ
## Maestro on TestingBot, frequently asked questions
What is Maestro and why run it on TestingBot?
Maestro is the open-source mobile UI testing framework from mobile.dev, used for native iOS, Android, Flutter and React Native automation testing. TestingBot runs the same Maestro flows you run locally on real iPhones, iPads and Android devices in the cloud. Swap `maestro cloud` for `testingbot maestro` and your existing flows just work.
Is TestingBot really a drop-in replacement for Maestro Cloud?
Yes. The CLI signature is identical (app + flows + flags), so any script or CI job that calls Maestro Cloud can call TestingBot instead. Same YAML flows, same arguments, same exit codes.
Which Maestro versions are supported?
TestingBot supports every recent Maestro release, and you can pin a specific Maestro server version per-test using the `--maestro-version` flag. Newer versions are typically available within days of release.
Can I run Maestro flows in parallel?
Yes. Plans on TestingBot include up to 100 parallel Maestro sessions on real devices, simulators and emulators. Use `--include-tags` / `--exclude-tags` to shard intelligently and the dashboard will roll the results back together.
Real devices, simulators or both?
All three. TestingBot runs Maestro flows on real iPhones, iPads, Galaxy and Pixel devices, plus iOS Simulators on Apple Silicon and Android Emulators with Google Play APIs. Pick per-flow with the `--real-device` flag.
How do I see results?
Live in the TestingBot dashboard with video streaming, step-by-step screenshots and Maestro logs. Programmatically via our [REST API](https://testingbot.com/support/api). Or piped into Jenkins, GitHub Actions, GitLab CI and any CI that consumes JSON.
Can Maestro flows reach my staging backend?
Install our [secure tunnel](https://testingbot.com/support/tunnel) and your Maestro flows can reach staging APIs, internal services and localhost without any inbound firewall rules.
Is the device cloud GDPR / SOC 2 compliant?
TestingBot is not SOC 2 or ISO 27001 certified. We are GDPR-compliant with EU-only data residency, registered at CSA STAR Level 1 (a published self-assessment), and our control set is modelled on ISO 27001 Annex A. Our datacenter operators hold ISO 27001. Devices are factory-reset between sessions, app uploads are encrypted, and SAML SSO and audit logs are available on enterprise plans.
What is Maestro?
Maestro is an open-source mobile UI testing framework built by mobile.dev. It lets you write declarative YAML flows that drive native iOS and Android apps, plus Flutter and React Native, with built-in retries, auto-waiting and screenshots. Teams use Maestro for end-to-end testing, smoke tests and regression coverage. TestingBot runs the same Maestro flows on real iPhones, iPads, Galaxy and Pixel devices, simulators and emulators in the cloud, with parallel execution and CI/CD integration.
Is Maestro free?
Yes, the Maestro CLI and framework are free. Mobile.dev releases Maestro under the Apache 2.0 license, so you can install it, write flows and run them locally at no cost. The cost question for teams is usually how to run Maestro at scale across real devices and parallel workers, which is what TestingBot provides on flat plans at a third of the cost of Maestro Cloud.
Is Maestro open source?
Maestro is open source. The framework is maintained by mobile.dev on GitHub under the Apache 2.0 license, with a public roadmap and active community. TestingBot complements the open-source framework with a managed device cloud for real iPhones, iPads and Android devices, parallel runs and detailed reporting.
Which platforms and frameworks does Maestro support?
Maestro supports native iOS apps, native Android apps, Flutter and React Native, and works on real devices, iOS Simulators and Android Emulators. The same YAML flow runs across platforms with minor selector differences. TestingBot exposes all of these targets through the standard Maestro CLI, so you can pick a real iPhone, an iOS Simulator on Apple Silicon, an Android Emulator with Google Play APIs or a real Galaxy or Pixel device per flow.
What is the best Maestro Cloud alternative?
Maestro Cloud (run by Mobile Dev Inc. at maestro.dev) is the official managed cloud for Maestro flows. TestingBot is the leading independent Maestro Cloud alternative, with an identical CLI, real iPhones, iPads, Galaxy and Pixel devices alongside iOS Simulators and Apple Silicon Android Emulators, roughly a third of the cost per run on flat plans with no per-test surprises, plus parallel execution, geolocation testing in 20+ countries, secure tunnel access for staging environments, real-time video streaming and CI/CD integrations out of the box.
How does TestingBot compare to DeviceCloud?
DeviceCloud is a Maestro-focused cloud that runs flows on parallel iOS Simulators and Android Emulators only, with no real-device support, on subscription plans where extra runs are billed per test. TestingBot offers a true real device cloud, with real iPhones, iPads, Galaxy and Pixel devices alongside the same iOS Simulators and Android Emulators, on flat plans with predictable pricing, real-time video and logs, geolocation testing in 20+ countries, device locale testing in 20+ locales, and Apple Silicon performance for iOS Simulators. Both expose a Maestro-compatible CLI, so migrating between them is a one-line change.
## More reasons to love TestingBot
### 24/7 support
Got questions? Ask in the 24/7 in-app customer [chat support](https://testingbot.com#) or [send us an email](https://testingbot.com/contact/new). We are here to help you!
TestingBot Support
Hey 👋 How can we help?
I need support
### Trusted by over 14,000 teams
6M+
Tests
120k+
Users
400+
Enterprises
106
Countries
[Request a demo](https://testingbot.com/demo)
### Developer Documentation
Everything you need to integrate TestingBot into your workflow. Browse guides, API references and SDKs for web and mobile testing.
- [Maestro Quickstart](https://testingbot.com/support/app-automate/maestro)
- [Maestro CLI options](https://testingbot.com/support/app-automate/maestro/options)
- [CI/CD integration guides](https://testingbot.com/support/integrations/ci-cd)
[Explore Documentation](https://testingbot.com/support)
### Enterprise-grade Security
STAR Level 1
Certified
GDPR
Compliant
SAML SSO
Single Sign-On
99.9%
Uptime SLA
[Visit TestingBot Trust Center](https://trust.testingbot.com/)
## Ready to scale your Maestro testing?
Read our [Maestro Cloud Testing](https://testingbot.com/support/app-automate/maestro) documentation and run your first flow in under 5 minutes.
[Start a free trial](https://testingbot.com/users/sign_up)
---
URL: https://testingbot.com/maestro-cloud-alternative
TestingBot vs Maestro Cloud
# The real-device Maestro Cloud alternative
Run your existing Maestro flow files on real iPhones, iPads, Android phones and tablets, plus simulators and emulators. Flat monthly pricing instead of negotiated quotes. EU data residency for procurement.
Switching from Maestro Cloud? Maestro users get a **30-day free trial** , no credit card required.
[Start your free trial](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo)
- **Real devices & sims:** 200+
- **EU hosted since:** 2012
- **Frameworks:** 8+
Trusted by some of the world's most innovative companies
At a glance
## Why teams pick TestingBot over Maestro Cloud
Six concrete differences that show up in the first sprint after switching.
- ### Real iPhones and Android devices
Run flows on actual hardware with biometrics, push, camera and sensors. Maestro Cloud's public offering is configurable models, not your real device matrix.
- ### Every framework, one place
Same account runs Maestro, Appium, XCUITest, Espresso, Playwright and Selenium. Maestro Cloud is Maestro-only by design.
- ### Published flat pricing
Public monthly tiers by concurrent sessions. No per-test meter, no sales-led negotiation. Maestro Cloud pricing is quote-only.
- ### EU hosted, GDPR ready
Test data, video and logs stay in Europe (EU and Switzerland). GDPR, STAR L1, ISO 27001 datacenters and SAML SSO included. Maestro Cloud is US-hosted.
- ### Tunnel to internal backends
TestingBot Tunnel exposes staging and pre-prod backends to the cloud devices over a secure connection. Maestro Cloud has no documented equivalent.
- ### Drop-in CLI swap
Same flow files, same YAML, same arguments. Replace maestro cloud with testingbot maestro and your suite runs on TestingBot in minutes.
Compare
## TestingBot vs Maestro Cloud, feature by feature
Capabilities side by side, based on publicly documented Maestro Cloud features as of October 2026.
TestingBot vs Maestro Cloud, feature comparison.| Capability | Real devices TestingBot | Maestro Cloud |
| --- | --- | --- |
| Starting price | Published tiers See [pricing](https://testingbot.com/pricing) | Not published Sales-led quote |
| Real iPhones and iPads | | Not publicly offered |
| Real Android phones and tablets | | Limited |
| iOS simulators and Android emulators | | |
| Appium, XCUITest, Espresso | | – |
| Playwright and Selenium for web | | – |
| Local tunnel to internal backends | TestingBot Tunnel | – |
| Biometrics, camera, push, sensors | Real device support | Not on simulators |
| CI/CD integrations | GitHub, GitLab, Jenkins, CircleCI, Bitbucket | GitHub Actions, Bitrise |
| Pricing transparency | Published monthly tiers | Sales-led quote |
| Free trial | 30 days, no card | 7 days |
| EU data residency | | – |
| Compliance | GDPR, STAR L1, ISO 27001 datacenters, SAML SSO | SOC 2 (per public claims) |
Pricing
## What Maestro testing costs on each platform
List prices as published by each vendor, checked on 19 August 2026. Shown in the currency each vendor publishes, without conversion.
### TestingBot
Billed per plan, every framework included.
- €50 / month, billed annually, for 1 parallel test and 1,000 automated minutes
- €90 / month, billed annually, for **unlimited** automated minutes
- Higher parallelism on the same plans: 2, 4, 8, 16 or 24 parallel tests
- Appium, XCUITest, Espresso and Selenium included, not priced separately
See the [full pricing page](https://testingbot.com/pricing) for month-to-month rates.
### Maestro Cloud
Billed per concurrent device.
- Free for the Local tier, running on your own devices with the open-source CLI
- $250 / device / month for Cloud, priced on maximum concurrent executions, with unlimited hosted runs
- Enterprise Cloud is listed as custom pricing with no public figure
- Pricing for web and Chromium devices is not published
Source: [maestro.dev/pricing](https://maestro.dev/pricing), 19 August 2026.
The closest like-for-like comparison is a single concurrent slot with unlimited runs: **€90 per month on TestingBot Automated Pro** against **$250 per device per month on Maestro Cloud**. The two are priced in different currencies and structured differently, so compare the tier that matches your concurrency rather than the headline number. Maestro's Local tier remains free if you are happy running on hardware you own and maintain.
Migrate in under an hour
## Switch from Maestro Cloud in four steps
Your flow files do not change. The migration is a CLI swap and a credentials update. The [Maestro Cloud migration guide](https://testingbot.com/support/app-automate/maestro/migrate-from-maestro-cloud) maps every `maestro cloud` flag to its TestingBot equivalent.
1. 1
### Create a free account
Sign up at testingbot.com and grab your TB\_KEY and TB\_SECRET from the account dashboard.
2. 2
### Install the CLI
`npm install -g @testingbot/cli`. Works alongside your existing Maestro CLI install.
3. 3
### Swap the run command
Replace `maestro cloud` with `testingbot maestro`. Add `--real-device --device "iPhone 15"` to target physical hardware.
4. 4
### Run your suite
Trigger a smoke test, watch the live video, fetch results via the [REST API](https://testingbot.com/support/app-automate/maestro/api) or dashboard.
Coverage
## Real iPhones, Pixels and Galaxys ready for your flows
A grid of physical iPhones, iPads, Android phones and tablets sits behind every `testingbot maestro --real-device` command. Pin a specific model or use a regex like `".*Pixel.*"` to allocate any available match.
Simulators and emulators are included on the same account for fast pre-merge smoke runs. Drop `--real-device` and the same flow runs in seconds on a clean simulator.
[See supported devices](https://testingbot.com/support/app-automate/devices)
- **Real iOS devices:** iPhone 6 → 17 Pro Max
- **Real Android devices:** Pixel, Galaxy, Huawei
- **iOS simulators:** iOS 12 → 26
- **Android emulators:** API 24 → 36
Frameworks
## Maestro is one of many on TestingBot
Engineering managers consolidate Maestro, Appium, XCUITest, Espresso, Playwright and Selenium in one place. Maestro Cloud is Maestro-only by design.
[](https://testingbot.com/features/automation/maestro)
### [Maestro](https://testingbot.com/features/automation/maestro)
YAML-driven mobile UI testing. Run flows on real devices, simulators and emulators.
[](https://testingbot.com/features/automation/appium)
### [Appium](https://testingbot.com/features/automation/appium)
Mobile automation on real iOS and Android devices for deeper white-box scenarios.
[](https://testingbot.com/features/automation/xcuitest)
### [XCUITest](https://testingbot.com/features/automation/xcuitest)
Apple's native iOS UI testing framework. Runs against real iPhones and iPads.
[](https://testingbot.com/features/automation/espresso)
### [Espresso](https://testingbot.com/features/automation/espresso)
Google's instrumentation framework for native Android apps. Same device fleet.
[](https://testingbot.com/features/automation/playwright)
### [Playwright](https://testingbot.com/features/automation/playwright)
Cross-browser automation with auto-waits, network interception and trace viewer.
[](https://testingbot.com/features/automation/selenium)
### [Selenium](https://testingbot.com/features/automation/selenium)
Industry-standard browser automation. Java, Python, Ruby, JavaScript, C# and more.
[](https://testingbot.com/features/automation/cypress)
### [Cypress](https://testingbot.com/features/automation/cypress)
Modern JavaScript end-to-end testing on Chrome, Firefox and Edge in parallel.
[](https://testingbot.com/features/automation/puppeteer)
### [Puppeteer](https://testingbot.com/features/automation/puppeteer)
Headless Chrome and Edge automation, ideal for fast scripted browser tasks.
Speed
## Parallel Maestro runs by concurrent sessions
Cut total Maestro suite duration by sharding flows across concurrent sessions. Each shard runs on its own device or simulator, fully isolated. The TestingBot CLI handles `--shard-split` and `--shard-all` identically to the Maestro CLI.
Predictable flat tiers, not per-test credits. Add concurrency by upgrading the plan, not by buying ad-hoc credit packs.
[See pricing](https://testingbot.com/pricing)
parallel-shards.sh
```
$ testingbot maestro app.apk ./flows \
--real-device \
--device "Pixel 8" \
--shard-split 4
shard 1 of 4 → Pixel 8 ✓ 12 flows · 3m 24s
shard 2 of 4 → Pixel 8 ✓ 12 flows · 3m 18s
shard 3 of 4 → Pixel 8 ✓ 12 flows · 3m 31s
shard 4 of 4 → Pixel 8 ✓ 12 flows · 3m 20s
Total: 48 flows · 3m 31s wall clock
```
Why switch
## Why teams move from Maestro Cloud to TestingBot
### They need real hardware
Payment, KYC, biometrics, push, camera. Simulators report synthetic results. Real iPhones and Android devices surface the bugs that ship to App Store and Play Store users.
### They need more than Maestro
Legacy Appium and XCUITest suites still exist. Running them on the same vendor as Maestro flows simplifies billing, security review and SSO.
### Procurement asks for EU
GDPR, CSA STAR and ISO 27001 datacenters are table stakes for European fintech, health and govtech buyers. TestingBot has been hosted in Europe since 2012.
### Pricing predictability
Flat monthly tiers that scale by concurrency, not per-test credits. Finance can budget. Engineering can scale without raising a ticket.
### Tunnel to internal backends
Maestro suites that need to hit staging or pre-prod backends behind the firewall use TestingBot Tunnel without exposing services publicly.
### Documentation that ships
Working YAML examples for GitHub Actions, GitLab CI, Jenkins, CircleCI and Bitbucket Pipelines, plus a full REST API reference and a step-by-step Maestro guide.
FAQ
## Frequently asked questions
What is the difference between TestingBot and Maestro Cloud?
Maestro Cloud is the cloud service operated by mobile-dev-inc, the maintainers of the Maestro framework. It executes Maestro flows in their own infrastructure with configurable device models. TestingBot is an independent cloud test platform that runs Maestro flows on real iPhones and Android devices, simulators and emulators, alongside Appium, XCUITest, Espresso and Playwright, on a flat monthly plan with EU data residency.
Can I keep my Maestro .flow files when moving to TestingBot?
Yes. TestingBot supports the standard Maestro YAML flow syntax. Your existing .flow files run unchanged. The migration is a CLI swap: replace maestro cloud with testingbot maestro and your suite executes on TestingBot's grid.
Does TestingBot support real iPhones and iPads for Maestro?
Yes. TestingBot provides physical iPhones, iPads, Android phones and tablets, plus iOS simulators and Android emulators, all from the same CLI. Use --real-device to target physical hardware, or omit the flag to run on a simulator or emulator.
How does pricing compare to Maestro Cloud?
TestingBot publishes flat monthly plans based on the number of concurrent sessions you need. There is no per-test charge and no execution-minute meter. Maestro Cloud pricing is not published publicly; teams typically negotiate per workspace. The TestingBot pricing page lists the current concurrent-session tiers.
What about Appium, XCUITest and other frameworks?
On TestingBot, the same account covers Maestro plus Appium, XCUITest, Espresso, Playwright and Selenium. Engineering managers running mixed mobile and web suites can consolidate on one vendor instead of one per framework.
Is TestingBot hosted in the EU? Is it GDPR compliant?
Yes. TestingBot is operated from Europe (EU and Switzerland) and has been hosted in Europe since 2012. We are GDPR-compliant, registered in the Cloud Security Alliance STAR Level 1 program (self-assessment), operate from ISO 27001 certified datacenters, and support SAML SSO on team plans.
How long does it take to migrate from Maestro Cloud?
Most teams migrate in under an hour. The Maestro .flow files do not change. Replace the CLI command, swap two environment variables, and run.
Does TestingBot offer a free trial for Maestro users?
Yes. TestingBot offers a 30-day free trial for teams coming from Maestro Cloud, no credit card required. Run your existing flow files immediately on real iPhones, Pixels, simulators and emulators.
Can I run Maestro tests from GitHub Actions on TestingBot?
Yes. TestingBot has dedicated CI/CD integration docs with working YAML examples for GitHub Actions, GitLab CI, Jenkins, CircleCI and Bitbucket Pipelines. Parallel sharding works out of the box.
Does TestingBot support tunneling to internal staging backends?
Yes. TestingBot Tunnel exposes your local or staging backends to the cloud devices over a secure connection. Maestro flows can hit internal services without exposing them publicly.
## Run Maestro on real devices for 30 days
No credit card. Your flow files run unchanged. Real iPhones, Android devices, simulators and emulators.
[Start free trial](https://testingbot.com/users/sign_up)
---
URL: https://testingbot.com/devicecloud-alternative
TestingBot vs devicecloud.dev
# The real-device devicecloud.dev alternative
devicecloud.dev runs Maestro on iOS simulators and Android emulators. TestingBot runs Maestro on the same simulators and emulators, plus real iPhones, Pixels and Galaxys with biometrics, push, camera and sensors.
Flat monthly pricing instead of per-test credits. EU data residency. **30-day free trial** , no credit card required.
[Start your free trial](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo)
- **Real iOS & Android:** 200+
- **EU hosted since:** 2012
- **Pricing model:** Flat
Trusted by some of the world's most innovative companies
At a glance
## Six reasons to pick TestingBot over devicecloud.dev
Each one shows up in the first sprint after switching.
- ### Real iPhones and Pixels
Real biometrics, real APNs and FCM push, real camera and sensors. devicecloud.dev cannot reproduce any of these because its cloud is simulators and emulators only.
- ### Flat monthly pricing
Published tiers by concurrent sessions. No per-test credit meter, no surprise top-ups when a long-running suite eats the credit pool.
- ### Tunnel to internal backends
TestingBot Tunnel exposes staging and pre-prod backends to the cloud devices over a secure connection. devicecloud.dev has no documented equivalent.
- ### EU hosted, GDPR ready
Test data, video and logs stay in Europe (EU and Switzerland). GDPR, STAR L1, ISO 27001 datacenters and SAML SSO included. devicecloud.dev is operated from the UK and US.
- ### Every framework, one place
Maestro alongside Appium, XCUITest, Espresso, Playwright and Selenium. devicecloud.dev is Maestro-only.
- ### Drop-in CLI swap
Replace dcd cloud with testingbot maestro. Same flow files, same YAML, same arguments. Migration is a single shell line.
Compare
## TestingBot vs devicecloud.dev, feature by feature
Capabilities and pricing side by side, as of October 2026.
TestingBot vs devicecloud.dev, feature and pricing comparison.| Capability | Real devices TestingBot | devicecloud.dev |
| --- | --- | --- |
| Entry price | Published tiers See [pricing](https://testingbot.com/pricing) | $25 / month Pro plan + $25 credit |
| Max plan | Flat by concurrency | $200 / month 20 + 20 parallel, sims only |
| Real iPhones and iPads | | – |
| Real Android phones and tablets | | – |
| iOS simulators and Android emulators | | |
| Biometrics, camera, push, sensors | Real device support | – |
| Pricing model | Flat monthly | Monthly + credits |
| Local tunnel to internal backends | TestingBot Tunnel | – |
| Multi-framework support | Appium, XCUITest, Espresso, Playwright, Selenium | Maestro only |
| EU data residency | | – |
| Compliance | GDPR, STAR L1, ISO 27001 datacenters, SAML SSO | Not published |
| CI/CD integrations | GitHub, GitLab, Jenkins, CircleCI, Bitbucket | GitHub Actions |
| Free trial | 30 days, no card | $20 starter credit |
Why real devices
## What emulators and simulators cannot verify
Maestro is the same framework on both clouds. What changes is the surface the .flow file runs against. Emulators and simulators cover most layout, navigation and form bugs, but they cannot reproduce these failure modes:
- ● **Biometrics:** Face ID, Touch ID, Android BiometricPrompt. Simulators can fake a result, but they cannot reproduce the OS-level interstitial, the timing or the fallback paths.
- ● **Push notifications:** Real APNs and FCM delivery, foreground vs background handling, deep links from a real payload. Simulators cannot register with the real push services.
- ● **Camera, microphone, sensors:** QR scanning, document capture, gyroscope, accelerometer, real GPS. Emulators report fixed or synthetic values.
- ● **Real network behaviour:** Carrier handover, captive portals, IPv6, real WebView versions across OEMs. Emulators run a vanilla AOSP WebView.
- ● **OEM Android skins:** Samsung One UI, Xiaomi HyperOS, Pixel stock all render and gesture differently. Emulators ship vanilla AOSP only.
Payment flows, onboarding with KYC, login with biometrics and any feature behind a push notification typically fail or pass differently on real hardware. Running a Maestro suite that includes those flows on emulators only means those tests are not actually being exercised.
Migrate in under an hour
## Switch from devicecloud.dev in four steps
Your flow files do not change. The migration is a CLI swap and a credentials update.
1. 1
### Create a free account
Sign up at testingbot.com and grab your TB\_KEY and TB\_SECRET from the account dashboard.
2. 2
### Install the CLI
`npm install -g @testingbot/cli`. Works alongside your existing Maestro and dcd CLI installs.
3. 3
### Swap the run command
Replace `dcd cloud` with `testingbot maestro`. Add `--real-device --device "Pixel 8"` for physical hardware.
4. 4
### Run your suite
Trigger a smoke test, watch the live video, fetch results via the [REST API](https://testingbot.com/support/app-automate/maestro/api) or dashboard.
Coverage
## 200+ real iPhones, Pixels and Galaxys
A grid of physical iPhones, iPads, Android phones and tablets sits behind every `testingbot maestro --real-device` command. Pin a specific model or use a regex like `".*Galaxy.*"` to allocate any available matching device.
Simulators and emulators are included on the same account for fast pre-merge smoke runs. Drop `--real-device` and the same flow runs in seconds on a clean simulator, just like on devicecloud.dev.
[See supported devices](https://testingbot.com/support/app-automate/devices)
- **Real iOS devices:** iPhone 6 → 15 Pro
- **Real Android devices:** Pixel, Galaxy, Xiaomi
- **iOS simulators:** iOS 12 → 17
- **Android emulators:** API 24 → 34
parallel-shards.sh
```
$ testingbot maestro app.apk ./flows \
--real-device \
--device "Pixel 8" \
--shard-split 4
shard 1 of 4 → Pixel 8 ✓ 12 flows · 3m 24s
shard 2 of 4 → Pixel 8 ✓ 12 flows · 3m 18s
shard 3 of 4 → Pixel 8 ✓ 12 flows · 3m 31s
shard 4 of 4 → Pixel 8 ✓ 12 flows · 3m 20s
Total: 48 flows · 3m 31s wall clock
```
Speed
## Parallel Maestro on real devices, not just emulators
Shard flows with `--shard-split` or `--shard-all` across concurrent real devices, simulators or emulators. The TestingBot CLI uses the same shard semantics as the Maestro CLI.
Add concurrency by upgrading the plan, not by buying ad-hoc credit packs. devicecloud.dev's per-test credit model penalises long suites and short flows alike.
[See pricing](https://testingbot.com/pricing)
Why switch
## Why teams move from devicecloud.dev to TestingBot
### They need real hardware
Payment, KYC, biometrics, push, camera. Simulators report synthetic results. Real iPhones and Pixels surface the bugs that ship to App Store and Play Store users.
### Predictable pricing
Flat monthly tiers that scale by concurrency, not per-test credits. Finance can budget. Engineering can scale without raising a top-up ticket.
### Procurement asks for EU
GDPR, CSA STAR and ISO 27001 datacenters are table stakes for European fintech, health and govtech buyers. devicecloud.dev is operated from the UK and US.
### More than just Maestro
Legacy Appium and XCUITest suites coexist with new Maestro flows. One vendor, one billing relationship, one SSO.
### Tunnel to internal backends
Maestro suites that hit staging or pre-prod backends behind the firewall use TestingBot Tunnel without exposing services publicly.
### Mature CI/CD
Working YAML examples for GitHub Actions, GitLab CI, Jenkins, CircleCI and Bitbucket Pipelines, plus a documented REST API.
FAQ
## Frequently asked questions
What is the difference between TestingBot and devicecloud.dev?
TestingBot runs Maestro flows on real iPhones, iPads, Android phones and tablets, plus iOS simulators and Android emulators, in a managed EU and US cloud, with flat monthly pricing. devicecloud.dev runs Maestro flows on iOS simulators and Android emulators only, and charges per test credit on top of a monthly plan.
Does devicecloud.dev support real devices?
No. devicecloud.dev's cloud is iOS simulators and Android emulators only. TestingBot supports both simulators/emulators and a fleet of real physical iPhones, iPads and Android phones.
Can I keep my Maestro flow files as they are?
Yes. Your existing Maestro YAML .flow files run unchanged on TestingBot. Replace dcd cloud or maestro cloud with the testingbot maestro CLI command and your existing flows execute against TestingBot's grid.
How does pricing compare?
TestingBot publishes flat monthly plans based on concurrent sessions. There is no per-test credit metering. devicecloud.dev's Pro plan starts at $25/month with $25 of credit and 5 parallel iOS plus 5 Android. The Max plan is $200/month with 20 parallel iOS plus 20 Android. Heavy users on devicecloud.dev typically burn through credits and add packs.
Is TestingBot hosted in the EU?
Yes. TestingBot has been hosted in Europe since 2012 and is GDPR-compliant, registered in the Cloud Security Alliance STAR Level 1 program (self-assessment), operates from ISO 27001 certified datacenters, and supports SAML SSO. devicecloud.dev runs from the UK and US.
How do I migrate from devicecloud.dev to TestingBot?
Install the TestingBot CLI (npm install -g @testingbot/cli), set your TB\_KEY and TB\_SECRET environment variables, then replace dcd cloud with testingbot maestro in your run script. No flow file changes required.
What about CI/CD?
TestingBot has dedicated CI/CD integration docs with examples for GitHub Actions, GitLab CI, Jenkins, CircleCI and Bitbucket Pipelines. Parallel sharding works out of the box on real devices, simulators and emulators.
Does TestingBot offer a free trial?
Yes. Maestro users get a 30-day free trial on TestingBot, no credit card required. Run your existing flow files immediately on real iPhones, Pixels, simulators and emulators.
Why does running on real devices matter for Maestro?
Emulators and simulators cannot reproduce biometrics (Face ID, Touch ID, BiometricPrompt), real APNs and FCM push delivery, camera, microphone, real sensor data, OEM Android skins (One UI, HyperOS), or real carrier and network behaviour. Payment flows, KYC and biometric login frequently pass on emulators and fail on real hardware.
## Try Maestro on real devices for 30 days
No credit card. Your flow files run unchanged. Real iPhones, Pixels, simulators and emulators in one parallel suite.
[Start free trial](https://testingbot.com/users/sign_up)
---
URL: https://testingbot.com/features/automation/xcuitest
XCUITest cloud
# Run XCUITest on real iPhones & iPads
Run your XCUITest builds on real iPhones and iPads from iOS 12 through iOS 27 with a single CLI command. Massive parallelism, video & log artifacts and a 99.9% uptime SLA.
Choosing a framework? See [Appium vs XCUITest](https://testingbot.com/appium-vs-xcuitest), or the [full automation framework lineup](https://testingbot.com/features).
[Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo)
- **Real Apple devices:** 100+
- **iOS versions:** 12 → 27
- **Uptime SLA:** 99.9%
Trusted by some of the world's most innovative companies
30-second answer
## Why run XCUITest on TestingBot?
Real iPhones and iPads
Run on a fleet of real Apple devices, not the iOS Simulator. Latest iPhone the day Apple ships it, plus a deep legacy fleet for backwards-compat coverage.
Replace your in-house Apple lab
Skip the cabinet of dusty iPhones, the swollen batteries and the MDM headache. TestingBot has run iOS tests on real devices for 5+ years.
Native Swift & Objective-C
Use the same XCUITest framework Apple ships with Xcode. Same APIs, same XCTest assertions, real native performance.
What is XCUITest?
## XCUITest in 60 seconds
### XCUITest
Apple · built into Xcode · Swift / Objective-C
XCUITest is Apple's official UI testing framework for iOS, iPadOS and tvOS. It is built into Xcode, runs natively on the device through XCTest, and lets you script taps, swipes, scrolls, predicates and accessibility queries against your app's real UI.
Because XCUITest runs on the device itself, tests have full access to native gestures, accessibility identifiers and Apple's official element hierarchy. There is no cross-process bridge, XCUITest is the most accurate way to drive a real iPhone.
- Native Apple framework, built into Xcode
- Swift and Objective-C bindings
- Real iPhones, iPads and tvOS. No simulator drift
- Predicates, accessibility identifiers, native gestures
- XCTest assertions, JUnit XML output for CI
One-line CLI
## Run XCUITest on real devices with one command
Install the TestingBot CLI, point it at your .ipa and your test bundle, pick a device and you're done. Uploads, capabilities and result polling are all handled for you.
install the CLI step 1
```bash
# Install the TestingBot CLI
$ npm install -g testingbot-cli
# Authenticate (once) — set your KEY and SECRET
$ export TESTINGBOT_KEY=your-key
$ export TESTINGBOT_SECRET=your-secret
# Build your XCUITest bundle with Xcode (one-off)
$ xcodebuild build-for-testing \
-scheme MyApp \
-destination 'generic/platform=iOS'
```
run XCUITest step 2
```bash
# Run XCUITest on a real iPhone — one command
$ testingbot xcuitest MyApp.ipa MyAppUITests-Runner.zip \
--device "iPhone 16" \
--version "27" \
--real-device
# → live results in your TestingBot dashboard
# → JUnit XML, video and logs via REST API
```
See the full [XCUITest on TestingBot documentation](https://testingbot.com/support/app-automate/xcuitest) for advanced flags, parallel runs and CI integration.
Buy vs build
## TestingBot vs an in-house Apple device lab
Maintaining your own iPhones and iPads is expensive. Devices break, batteries swell, iOS updates ship every few weeks. Here is how a hosted Apple device cloud compares.
| Capability | TestingBot | In-house Apple lab |
| --- | --- | --- |
| Device coverage | Real iPhones and iPads from iOS 12 to 27 | Limited to what you can buy and rack |
| Time to first test | Minutes, upload your .ipa and trigger via API | Weeks of provisioning, MDM, certificates, tethering |
| Parallel scaling | Up to 100 concurrent XCUITest sessions | Capped by the number of devices you own |
| Video & logs | Video, device logs, screenshots and JUnit XML | Roll your own recording & log aggregation |
| Reliability | 99.9% uptime SLA, 24/7 monitoring | You replace dead batteries and broken screens |
| Security | Devices wiped between every session | Manual cleanup, leftover state risks |
| Support | 24/7 human support and dedicated CSM | Stack Overflow and your own logs |
Capabilities
## Everything XCUITest needs in the cloud
Beyond running tests, TestingBot exposes the full set of XCUITest capabilities you would set in Xcode, plus cloud-only features like geolocation and network throttling.
- ### [Build & upload](https://testingbot.com/support/app-automate/xcuitest/build-testsuite)
Compile your .ipa and .xctest with Xcode, then push to TestingBot Storage.
- ### [Test reports](https://testingbot.com/support/app-automate/xcuitest/test-reports)
Video, device logs, screenshots and JUnit XML for every run.
- ### [Screen orientation](https://testingbot.com/support/app-automate/xcuitest/change-screen-orientation)
Force portrait or landscape per test, mid-test rotations included.
- ### [Network throttling](https://testingbot.com/support/app-automate/xcuitest/simulate-network-conditions)
Simulate 3G, 4G, lossy WiFi and offline conditions per session.
- ### [Localization & geo](https://testingbot.com/support/app-automate/xcuitest/set-localization-options)
Set device locale, language and GPS coordinates for App Store and back-end checks.
Real Apple devices
## iPhone and iPad fleet on demand
A fleet of real Apple devices spanning iOS 12 through iOS 27, refreshed every release. Latest flagship iPhones and iPads available the day Apple ships them.
[View all devices](https://testingbot.com/support/app-automate/devices)
iPhone 13 Pro
iPhone 17 Pro
iPhone 17 Pro Max
iPhone 16 Pro
iPad (9th generation)
iPhone Air
Test reports
## Video, screenshots and JUnit XML for every run
Every XCUITest run ships with a video screencast, device logs, viewport screenshots and a JUnit XML report compatible with every major CI parser.
Mark sessions as passed or failed from your tests, fetch artifacts via our [REST API](https://testingbot.com/support/api), and ship results into Jira, Slack or your CI dashboard.
- Full HD video for every test run
- Device logs, app logs and crash reports
- Viewport screenshots on demand
- JUnit XML for Jenkins, GitHub Actions, GitLab CI
Continuous integration
## Plug XCUITest into your CI/CD in minutes
TestingBot integrates with every major CI/CD tool. Trigger XCUITest runs on every build and stream device videos and JUnit reports back into your build dashboards.
### [Jenkins](https://testingbot.com/support/integrations/ci-cd/jenkins)
### [Bamboo](https://testingbot.com/support/integrations/ci-cd/bamboo)
### [TeamCity](https://testingbot.com/support/integrations/ci-cd/teamcity)
### [CircleCI](https://testingbot.com/support/integrations/ci-cd/circleci)
### [GitLab CI](https://testingbot.com/support/integrations/ci-cd/gitlab)
### [GitHub](https://testingbot.com/support/integrations/ci-cd/github-actions)
Spoof device GPS
20+ regions
United States
United Kingdom
Germany
France
Spain
Belgium
Australia
China
Japan
Geolocation testing
## Test your iOS app from 20+ regions
Spoof the device GPS to anywhere in the world and verify **localisation** , **geo-targeting** and **App Store store-front rules** from any country your customers live in.
Secure devices
## Real devices. Wiped after every session.
Each XCUITest session locks one real iPhone or iPad for your test only. The moment your test ends, the device is wiped, all installed apps are deleted, caches are cleared, and the device is returned to the pool ready for the next customer.
GDPR-compliant, with SAML SSO and audit logs available for enterprise teams.
- One real device per session, never shared
- Wiped between every session
- Encrypted .ipa uploads with per-app retention controls
[Visit TestingBot Trust Center](https://trust.testingbot.com/)
1. 1
Your XCUITest run locks one real device
A pristine iPhone or iPad is reserved for your session only.
2. 2
Tests run natively on real Apple hardware
Real sensors, real accelerometer, real Touch ID and Face ID flows.
3. 3
Device wiped on release
Apps removed, caches cleared, accounts wiped. Returned clean to the pool.
Secure Tunnel
## Test apps that talk to internal backends
Use our [secure tunnel](https://testingbot.com/support/tunnel) so XCUITest runs can reach private APIs, staging environments and services running on localhost.
Encrypted end-to-end, no inbound firewall rules required, ready out of the box for popular CI providers.
[Read more](https://testingbot.com/support/tunnel)
FAQ
## XCUITest on TestingBot — frequently asked questions
What is XCUITest and why run it on TestingBot?
XCUITest is Apple's native UI testing framework for iOS, iPadOS and tvOS. TestingBot runs your XCUITest builds on a fleet of real iPhones and iPads, so you do not have to maintain devices, certificates and MDM in-house. Upload your `.ipa` and `.xctest` through our REST API and trigger runs in seconds.
Which iOS versions are supported?
TestingBot supports real iOS devices from iOS 12 through iOS 27, refreshed every Apple release. We carry both the latest flagship iPhones and a deep legacy fleet for backwards-compat coverage.
Do I need a Mac to use XCUITest on TestingBot?
You need Xcode to build your .ipa and .xctest bundle, but the actual test run happens on TestingBot. Many teams build in CI on a Mac runner, then push the artifacts to TestingBot for the device run.
Can I run XCUITest in parallel?
Yes. Plans on TestingBot include up to 100 parallel XCUITest sessions on real devices. Each session locks a dedicated iPhone or iPad, so tests do not interfere with one another.
Do you provide physical devices?
Yes, we provide physical iPhones and iPads, not simulators. Each test run locks a real device for your session only, with native performance and access to real sensors, Touch ID and Face ID. After your session ends, the device is wiped and returned clean to the pool.
How do I see results?
Live in the TestingBot dashboard with HD video, screenshots and device logs. Programmatically via the [REST API](https://testingbot.com/support/api). Plus a JUnit XML report compatible with every major CI parser.
Can XCUITest reach my staging backend?
Install our [secure tunnel](https://testingbot.com/support/tunnel) and your XCUITest runs can reach staging APIs, internal services and localhost without any inbound firewall rules.
Is the device cloud GDPR / SOC 2 compliant?
TestingBot is not SOC 2 or ISO 27001 certified. We are GDPR-compliant with EU-only data residency, registered at CSA STAR Level 1 (a published self-assessment), and our control set is modelled on ISO 27001 Annex A. Our datacenter operators hold ISO 27001. Devices are wiped between sessions, app uploads are encrypted, and SAML SSO and audit logs are available on enterprise plans.
## Ready to scale your XCUITest runs?
Read our [XCUITest documentation](https://testingbot.com/support/app-automate/xcuitest) and run your first test on a real iPhone in under 5 minutes.
[Start a free trial](https://testingbot.com/users/sign_up)
Resources you may like
[](https://testingbot.com/resources/articles/automate-native-ios-apps-xcuitest)
### [Automate native iOS Apps with XCUITest](https://testingbot.com/resources/articles/automate-native-ios-apps-xcuitest)
How to automate Apple's own iOS apps with XCUITest, with code for driving Settings, Messages and Photos, and how to extend it to other apps.
[Read more: Automate native iOS Apps with XCUITest](https://testingbot.com/resources/articles/automate-native-ios-apps-xcuitest)
[](https://testingbot.com/blog/ios-18-testing)
### [Test your website and mobile apps on iOS 18.](https://testingbot.com/blog/ios-18-testing)
Ready to start testing on iOS 18? Get started today with TestingBot: run automated, visual and manual tests on iOS 18.
[Read more: Test your website and mobile apps on iOS 18.](https://testingbot.com/blog/ios-18-testing)
---
URL: https://testingbot.com/features/automation/appium
Appium 1, 2 & 3 ready
# Appium Cloud Automation
Run your Appium tests on real iOS and Android devices. Native, hybrid and mobile web apps. Massive parallelism, video & log artifacts and a 99.9% uptime SLA.
Comparing frameworks? See [Appium vs Espresso](https://testingbot.com/appium-vs-espresso) and [Appium vs XCUITest](https://testingbot.com/appium-vs-xcuitest).
[Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo)
- **Real devices:** 200+
- **Appium versions:** 1, 2 & 3
- **Uptime SLA:** 99.9%
_Appium tests running on real iOS, Android, Samsung and Pixel devices in the TestingBot cloud_
Trusted by some of the world's most innovative companies
5-line setup
## Run your existing Appium scripts on real devices
Point your Appium driver at the TestingBot hub, supply your app and capabilities, and your test runs on a real iPhone or Android phone in seconds.
hub.testingbot.com Copy
JavaScript Python Java Ruby C# PHP
```javascript
// Run an Appium test on a real iPhone in the TestingBot cloud
const { remote } = require('webdriverio');
const driver = await remote({
hostname: 'hub.testingbot.com',
port: 443,
protocol: 'https',
capabilities: {
'platformName': 'iOS',
'appium:deviceName': 'iPhone 15',
'appium:platformVersion': '18',
'appium:app': 'tb://YOUR_APP_ID',
'tb:options': { name: 'Checkout flow', realDevice: true }
}
});
await driver.deleteSession();
```
```python
# Run an Appium test on a real Android device in the TestingBot cloud
from appium import webdriver
from appium.options.android import UiAutomator2Options
options = UiAutomator2Options()
options.platform_version = '14'
options.device_name = 'Galaxy S24'
options.app = 'tb://YOUR_APP_ID'
options.set_capability('tb:options', {'name': 'Checkout flow', realDevice: True})
driver = webdriver.Remote(
'https://hub.testingbot.com/wd/hub',
options=options
)
driver.quit()
```
```java
// Run an Appium test on a real iPhone in the TestingBot cloud
XCUITestOptions options = new XCUITestOptions()
.setPlatformVersion("18")
.setDeviceName("iPhone 15")
.setApp("tb://YOUR_APP_ID");
options.setCapability("tb:options", Map.of("realDevice", true, "name", "Checkout flow"));
IOSDriver driver = new IOSDriver(
new URL("https://KEY:SECRET@hub.testingbot.com/wd/hub"),
options
);
driver.quit();
```
```ruby
# Run an Appium test on a real Android device in the TestingBot cloud
require 'appium_lib'
caps = {
'platformName' => 'Android',
'appium:deviceName' => 'Pixel 9',
'appium:platformVersion' => '15',
'appium:app' => 'tb://YOUR_APP_ID',
'tb:options' => { name: 'Checkout flow', realDevice: true }
}
driver = Appium::Driver.new(
{ caps: caps, appium_lib: { server_url: 'https://hub.testingbot.com/wd/hub' } }, true
).start_driver
```
```csharp
// Run an Appium test on a real iPhone in the TestingBot cloud
var options = new AppiumOptions();
options.PlatformName = "iOS";
options.AddAdditionalAppiumOption("deviceName", "iPhone 15");
options.AddAdditionalAppiumOption("platformVersion", "18");
options.AddAdditionalAppiumOption("app", "tb://YOUR_APP_ID");
options.AddAdditionalAppiumOption("tb:options", new Dictionary { { "realDevice", true }, { "name", "Checkout flow" } });
var driver = new IOSDriver(
new Uri("https://KEY:SECRET@hub.testingbot.com/wd/hub"),
options
);
driver.Quit();
```
```php
// Run an Appium test on a real Android device in the TestingBot cloud
$capabilities = [
'platformName' => 'Android',
'appium:deviceName' => 'Galaxy S24',
'appium:platformVersion' => '14',
'appium:app' => 'tb://YOUR_APP_ID',
'tb:options' => ['realDevice' => true, 'name' => 'Checkout flow'],
];
$driver = RemoteWebDriver::create(
'https://KEY:SECRET@hub.testingbot.com/wd/hub',
$capabilities
);
$driver->quit();
```
Connected · [View documentation](https://testingbot.com/support/app-automate/appium/nodejs) [View documentation](https://testingbot.com/support/app-automate/appium/python) [View documentation](https://testingbot.com/support/app-automate/appium/java) [View documentation](https://testingbot.com/support/app-automate/appium/ruby) [View documentation](https://testingbot.com/support/app-automate/appium/csharp) [View documentation](https://testingbot.com/support/app-automate/appium/php)
Same WebDriver protocol you already use, with a single endpoint switch. See the docs for [Appium on TestingBot](https://testingbot.com/support/app-automate/appium) for full examples.
Real device cloud
## Replace your in-house device lab
Skip buying, hosting and maintaining iOS and Android devices. TestingBot's mobile cloud is on-call 24/7 with hundreds of real, physical devices ready to run your Appium tests.
Every Appium session ships with a video screencast, Appium server logs, device logs, screenshots and network HAR. Pull artifacts from our [REST API](https://testingbot.com/support/api) and post them straight into Jira, Slack or your CI dashboard.
- Real iPhones, iPads, Galaxy, Pixel & OnePlus
- Native, hybrid and mobile web apps
- Pristine devices, factory-reset between sessions
- HD video, logs and HAR for every test
- Upload IPA, APK and TestFlight builds
Languages & frameworks
## Appium integrations for every stack
Appium uses the W3C WebDriver protocol, so your existing test code drops in with a single hub URL change. We support the official Appium client in every major language.
- ### [Java](https://testingbot.com/support/app-automate/appium/java)
- [TestNG](https://testingbot.com/support/app-automate/appium/java/testng)
- [JUnit](https://testingbot.com/support/app-automate/appium/java)
- ### [Python](https://testingbot.com/support/app-automate/appium/python)
- [PyTest](https://testingbot.com/support/app-automate/appium/python)
- [Robot Framework](https://testingbot.com/support/app-automate/appium/robotframework)
- ### [JavaScript](https://testingbot.com/support/app-automate/appium/nodejs)
- [WebDriverIO](https://testingbot.com/support/app-automate/appium/nodejs/webdriverio)
- [CodeceptJS](https://testingbot.com/support/app-automate/appium/nodejs/codeceptjs)
- ### [Ruby](https://testingbot.com/support/app-automate/appium/ruby)
- [RSpec](https://testingbot.com/support/app-automate/appium/ruby)
- [Cucumber](https://testingbot.com/support/app-automate/appium/ruby)
- ### [C#](https://testingbot.com/support/app-automate/appium/csharp)
- [NUnit](https://testingbot.com/support/app-automate/appium/csharp/nunit)
- [SpecFlow](https://testingbot.com/support/app-automate/appium/csharp/specflow)
- ### [PHP](https://testingbot.com/support/app-automate/appium/php)
- [PHPUnit](https://testingbot.com/support/app-automate/appium/php)
- [Codeception](https://testingbot.com/support/app-automate/appium/php)
New to Appium? Read the [Appium quickstart](https://testingbot.com/support/app-automate/appium) or jump straight into a language-specific guide below.
Massive parallelism
## Cut mobile suite time from hours to minutes
Fan out your Appium suite across hundreds of real devices in parallel. Same test code, a fraction of the wall-clock time, no flakiness from emulator drift.
More on how concurrency is sized and priced: [parallel testing](https://testingbot.com/features/parallel-testing).
- Up to 100 concurrent Appium sessions
- Sessions start in seconds, no device queue
- Auto-retry, and your runner's device parallelism works unchanged
[Start free](https://testingbot.com/users/sign_up)
Spoof device GPS
20+ regions
United States
United Kingdom
Germany
France
Spain
Belgium
Australia
China
Japan
Geolocation testing
## Test your app from 20+ regions
Spoof the device GPS to anywhere in the world and verify **localisation** , **geo-targeting** and **store-front rules** in iOS App Store, Google Play and your own backend.
Secure devices
## Real devices. Wiped after every session.
Each Appium session locks one real device for your test only. The moment your test ends, the device is factory-reset, all installed apps are deleted, caches are cleared, and the device is returned to the pool ready for the next customer.
GDPR-compliant, with SAML SSO and audit logs for teams.
- One real device per session, never shared
- Factory-reset between every session
- Encrypted app uploads with per-app retention controls
[Visit TestingBot Trust Center](https://trust.testingbot.com/)
1. 1
Your test locks one real device
A pristine iPhone or Android phone is reserved for your session only.
2. 2
Appium drives real hardware
Real sensors, real GPS, real biometrics. Native, hybrid or mobile web.
3. 3
Device factory-reset on release
Apps removed, caches cleared, accounts wiped. Returned clean to the pool.
Continuous integration
## Plug Appium into your CI/CD in minutes
TestingBot integrates with every major CI/CD tool. Trigger Appium suites on every commit and stream device videos and logs back into your build dashboards.
### [Jenkins](https://testingbot.com/support/integrations/ci-cd/jenkins)
### [Bamboo](https://testingbot.com/support/integrations/ci-cd/bamboo)
### [TeamCity](https://testingbot.com/support/integrations/ci-cd/teamcity)
### [CircleCI](https://testingbot.com/support/integrations/ci-cd/circleci)
### [GitLab CI](https://testingbot.com/support/integrations/ci-cd/gitlab)
### [GitHub](https://testingbot.com/support/integrations/ci-cd/github-actions)
Device cloud
## Real iPhones, iPads, Galaxy & Pixel devices
Test on the iPhones, iPads, Galaxy, Pixel, OnePlus and Xiaomi phones your customers actually use. Latest flagship devices on day one, plus a broad legacy fleet for backwards-compat coverage.
[View all devices](https://testingbot.com/support/app-automate/devices)
iPhone 13 Pro
Pixel 8
Pixel 11
iPhone 17 Pro
iPhone 17 Pro Max
iPhone 16 Pro
Secure Tunnel
## Test apps that talk to internal backends
Use our [secure tunnel](https://testingbot.com/support/tunnel) to let your Appium tests reach private APIs, staging environments and services running on localhost.
Encrypted end-to-end, no inbound firewall rules required, and ready out of the box for popular CI providers.
[Read more](https://testingbot.com/support/tunnel)
Buy vs build
## TestingBot vs an in-house device lab
Building and maintaining your own iOS and Android device lab is expensive. Devices break, batteries swell, OS updates ship constantly. Here is how a hosted device cloud compares.
| Capability | TestingBot | In-house device lab |
| --- | --- | --- |
| Device coverage | Real iPhones, iPads, Galaxy, Pixel, OnePlus, Xiaomi, refreshed every release | Limited to what you can buy and rack yourself |
| Time to first test | Minutes, change one URL in your Appium driver | Weeks of provisioning, MDM and Appium server setup |
| Parallel scaling | Up to 100 concurrent sessions, on demand | Capped by the number of devices you own |
| Video & logs | Video, device logs, Appium logs and HAR included | Roll your own recording & log aggregation |
| Reliability | 99.9% uptime SLA, 24/7 monitoring | You replace dead batteries and broken screens |
| Security | Devices factory-reset between every session | Manual cleanup, leftover state risks |
| Support | 24/7 human support and dedicated CSM | Stack Overflow and your own logs |
FAQ
## Appium on TestingBot: frequently asked questions
What is Appium and why run it on TestingBot?
Appium is the open-source automation framework for native, hybrid and mobile web apps. TestingBot hosts a fleet of real iPhones, iPads, Galaxy and Pixel devices that you can drive with your existing Appium scripts. Point your driver at `https://hub.testingbot.com/wd/hub` and your test runs on a real device in seconds.
Which Appium versions are supported?
TestingBot supports Appium 1, Appium 2 and Appium 3, including Appium 2/3 plugins and the latest XCUITest and UIAutomator2 drivers. You can pin a specific Appium server version per-test through capabilities, so you can keep legacy suites running on Appium 1 while migrating new tests to Appium 3.
Can I test native, hybrid and mobile web apps?
Yes. TestingBot supports all three Appium contexts. Native iOS apps via XCUITest, native Android apps via UIAutomator2 or Espresso, hybrid apps in WebView, and mobile web in Safari and Chrome on real devices.
How do I upload my IPA or APK?
Upload your build to [TestingBot Storage](https://testingbot.com/support/app-automate/help/upload) via the REST API or our CLI, then reference it from your Appium capabilities. Builds can be retained per-app for fast re-runs.
Can I run Appium tests in parallel?
Yes. Plans on TestingBot include up to 100 parallel Appium sessions on real devices. Each session locks a dedicated device, so tests do not interfere with one another.
Are the devices real or simulators?
TestingBot's mobile cloud uses real, physical iOS and Android devices. We also offer iOS Simulators and Android Emulators for early-stage tests, but real devices are the default and the recommended target for accuracy.
Can I test apps that need a private backend?
Install our [secure tunnel](https://testingbot.com/support/tunnel) and your Appium tests can reach staging APIs, internal services and localhost. No inbound firewall rules needed.
Is the device cloud SOC 2 / GDPR compliant?
TestingBot is not SOC 2 or ISO 27001 certified. We are GDPR-compliant with EU-only data residency, registered at CSA STAR Level 1 (a published self-assessment), and our control set is modelled on ISO 27001 Annex A. Our datacenter operators hold ISO 27001. Devices are factory-reset between sessions, app uploads are encrypted and SAML SSO and audit logs are available on enterprise plans.
## More reasons to love TestingBot
### 24/7 support
Got questions? Ask in the 24/7 in-app customer [chat support](https://testingbot.com#) or [send us an email](https://testingbot.com/contact/new). We are here to help you!
TestingBot Support
Hey 👋 How can we help?
I need support
### Trusted by over 14,000 teams
6M+
Tests
120k+
Users
400+
Enterprises
106
Countries
[Request a demo](https://testingbot.com/demo)
### Developer Documentation
Everything you need to integrate TestingBot into your workflow. Browse guides, API references and SDKs for web and mobile testing.
- [Appium Quickstart](https://testingbot.com/support/app-automate/appium)
- [App uploads (IPA / APK)](https://testingbot.com/support/app-automate/help/upload)
- [Available devices](https://testingbot.com/support/app-automate/devices)
- [CI/CD integration](https://testingbot.com/support/integrations/ci-cd)
[Explore Documentation](https://testingbot.com/support)
### Enterprise-grade Security
STAR Level 1
Certified
GDPR
Compliant
SAML SSO
Single Sign-On
99.9%
Uptime SLA
[Visit TestingBot Trust Center](https://trust.testingbot.com/)
## Ready to scale your Appium testing?
Read our [Appium Cloud Testing](https://testingbot.com/support/app-automate/appium) documentation and run your first test in under 5 minutes.
[Start a free trial](https://testingbot.com/users/sign_up)
Resources you may like
[](https://testingbot.com/resources/articles/touch-actions-appium)
### [Using Touch Actions with Appium](https://testingbot.com/resources/articles/touch-actions-appium)
How to simulate taps, swipes and pinches on real devices with Appium Touch Actions, with examples, best practices and common pitfalls.
[Read more: Using Touch Actions with Appium](https://testingbot.com/resources/articles/touch-actions-appium)
[](https://testingbot.com/resources/articles/appium-2-migration)
### [Appium 2 and Appium 3 Migration Guide](https://testingbot.com/resources/articles/appium-2-migration)
What changed between Appium 1, 2 and 3: drivers, capabilities, the removed touch endpoints, and how to upgrade an existing suite.
[Read more: Appium 2 and Appium 3 Migration Guide](https://testingbot.com/resources/articles/appium-2-migration)
[](https://testingbot.com/resources/articles/dark-mode-testing)
### [Dark Mode Testing with Appium](https://testingbot.com/resources/articles/dark-mode-testing)
How to switch a native app between light and dark mode from an Appium test on iOS and Android, and the layout bugs dark mode tends to expose.
[Read more: Dark Mode Testing with Appium](https://testingbot.com/resources/articles/dark-mode-testing)
---
URL: https://testingbot.com/test-in-internet-explorer
# Internet Explorer Testing
Easily test on Internet Explorer Online! Instant access to any IE Browser versions (7-11). Test your website on real IE browsers. No IE Emulators or Simulators.
Signup & Start Testing for Free!
[Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com)
Trusted by some of the world's most innovative companies
Interactive Testing
## Interactive Testing On Internet Explorer Browsers
### IE 11, IE 10, IE 9, IE 8, IE 7
Run tests on any version of internet explorer, on real Windows desktops.
### Responsive Testing
Each test on TestingBot comes with a video screencast of the device, logs and other generated artifacts.
### No more VMs
No more VMs to be maintained or software to be installed. TestingBot has you covered.
### +6100 combinations
Instant access to any browser on Mac, Linux, Windows and Mobile.
Real Browsers
## IE Emulators vs Real Internet Explorer Browsers
Internet Explorer emulators do not always show correct results. Test on real IE browsers, running on a real Windows operating system.
No Emulators, Simulators or Virtual Machines, try TestingBot and get instant access to a real IE browser.
[Start free](https://testingbot.com/users/sign_up)
Global Testing
## Geolocation Testing on IE
Run Selenium tests from real exit nodes around the world. Verify **geo-targeting** , **geo-blocking** and **localisation** from Australia, Belgium, the United Kingdom, the United States, Germany, France, Spain, China and more.
Visual Testing
## Automated IE Screenshots
Take automated screenshots of your website on various IE versions. Rapidly test your webpages for visual differences across multiple IE browsers.
Secure Tunnel
## Test internal & staging apps in Internet Explorer
Use the secure [TestingBot Tunnel](https://testingbot.com/support/tunnel) to quickly and easily test your staging website on any IE browser version.
[Read more](https://testingbot.com/support/tunnel)
FAQ
## Frequently Asked Questions
### Why do I need to test my website on Internet Explorer?
Internet Explorer (IE) has a market share of about 5% in the global browser market. Some organizations or websites are designed to only allow IE, while blocking other browsers for security reasons.
IE is the default browser on most of the (older) Windows editions, which means a large portion of Windows users will use this browser.
### What is an Internet Explorer (IE) emulator?
An Internet Explorer emulator acts as if it is an Internet Explorer browser. It tries to help developers and testers to test their websites on Internet Explorer.
Because this is an emulator, it is not accurate enough to use this for testing.
TestingBot recommends using its IE browsers running on real desktops, no emulator.
### Why should I not test with an IE emulator?
IE Emulators only mimic an Internet Explorer browser screen and User-Agent. It runs slower than a real IE browser and is not as accurate.
You cannot accurately test the CSS support of IE with an emulator. If you are using macOS or Linux, your choices are to install a Windows VM or use a cloud service such as TestingBot.
### How does TestingBot help with internet browser testing?
TestingBot provides a grid of real browsers, including IE7 up to IE11, which you can use for automated and manual testing.
Run a manual IE test on Windows 8, 8.1 or 7. Or perform automated tests on IE7 up to IE11.
### Can I use macOS or Linux to test on IE?
If your computer is a macOS or Linux device, we recommend using TestingBot's cloud service to access a IE browser instance.
No need to download, install and maintain a Windows VM. Use TestingBot's interactive, browser-based, remote IE service.
### More reasons to love TestingBot
### 24/7 support
Got questions? Ask in the 24/7 in-app customer [chat support](https://testingbot.com#) or [send us an email](https://testingbot.com/contact/new). We are here to help you!
TestingBot Support
Hey 👋 How can we help?
I need support
### Trusted by over 14,000 teams
6M+
Tests
120k+
Users
400+
Enterprises
106
Countries
[Request a demo](https://testingbot.com/demo)
### Developer Documentation
Everything you need to integrate TestingBot into your workflow. Browse guides, API references and SDKs for web and mobile testing.
- [Quickstart tutorials for Selenium](https://testingbot.com/support/web-automate/selenium)
- [Appium Examples](https://testingbot.com/support/app-automate/appium)
- [Integrations](https://testingbot.com/support/integrations)
- [List of browsers & devices](https://testingbot.com/support/web-automate/browsers)
[Explore Documentation](https://testingbot.com/support)
### Enterprise-grade Security
STAR Level 1
Certified
GDPR
Compliant
SAML SSO
Single Sign-On
99.9%
Uptime SLA
[Visit TestingBot Trust Center](https://trust.testingbot.com/)
### Ready to start IE Testing?
Start testing your apps with TestingBot.
[Start a free trial](https://testingbot.com/users/sign_up)
---
URL: https://testingbot.com/test-edge-browser
# Online Cross Browser Testing On Microsoft Edge Browser
Easily test on Microsoft Edge Online! Make sure your website works on the latest Edge browser versions that come with Microsoft Windows. Test your website on real Edge browsers.
Signup & Start Testing for Free!
[Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo)
Trusted by some of the world's most innovative companies
Interactive Testing
## Interactive Testing On Internet Explorer Browsers
### Legacy versions up to the latest version
Run tests on various Microsoft Edge versions, on real Windows and macOS desktops.
### Responsive Testing
Each test on TestingBot comes with a video screencast of the device, logs and other generated artifacts.
### No more VMs
No more VMs to be maintained or software to be installed. TestingBot has you covered.
### +6100 combinations
Instant access to multiple Microsoft Edge browsers, on Windows and macOS.
Real Browsers
## Microsoft Edge Emulators vs Real Microsoft Edge Browsers
Microsoft Edge emulators do not always show correct results. Test on real Microsoft Edge browsers, running on a real Windows operating system.
No Emulators, Simulators or Virtual Machines, try TestingBot and get instant access to a real Microsoft Edge browser.
[Start free](https://testingbot.com/users/sign_up)
Global Testing
## Geolocation Testing on Microsoft Edge
Perform \geolocation testing\ in various regions across the world. Test Geo Targeting, Geo Blocking and Geo Localization in 10 countries, including Australia, Belgium, United Kingdom, United States, Germany, France, Spain, China and more.
Visual Testing
## Automated Microsoft Edge Screenshots
Take automated screenshots of your website on various Microsoft Edge versions. Rapidly test your webpages for visual differences across multiple Microsoft Edge browsers.
Secure Tunnel
## Test internal & staging apps in Edge
Use the secure [TestingBot Tunnel](https://testingbot.com/support/tunnel) to quickly and easily test your staging website on any Microsoft Edge browser version.
[Read more](https://testingbot.com/support/tunnel)
FAQ
## Frequently Asked Questions
### Why do I need to test my website on Microsoft Edge?
Microsoft Edge is a popular browser that comes by default on new Microsoft Windows installations. It has a market percentage of 6% globally.
It's important to make sure your website looks and behaves correctly on this web browser.
### How does TestingBot help with Microsoft Edge testing?
TestingBot provides an online cloud of Microsoft Edge browsers, running on Windows and macOS machines.
You can run manual tests on Edge browsers, or use WebDriver automation to automatically test your website.
### Can I use Microsoft Edge on macOS?
TestingBot provides macOS machines equipped with multiple versions of Microsoft Edge.
### More reasons to love TestingBot
### 24/7 support
Got questions? Ask in the 24/7 in-app customer [chat support](https://testingbot.com#) or [send us an email](https://testingbot.com/contact/new). We are here to help you!
TestingBot Support
Hey 👋 How can we help?
I need support
### Trusted by over 14,000 teams
6M+
Tests
120k+
Users
400+
Enterprises
106
Countries
[Request a demo](https://testingbot.com/demo)
### Developer Documentation
Everything you need to integrate TestingBot into your workflow. Browse guides, API references and SDKs for web and mobile testing.
- [Quickstart tutorials for Selenium](https://testingbot.com/support/web-automate/selenium)
- [Appium Examples](https://testingbot.com/support/app-automate/appium)
- [Integrations](https://testingbot.com/support/integrations)
- [List of browsers & devices](https://testingbot.com/support/web-automate/browsers)
[Explore Documentation](https://testingbot.com/support)
### Enterprise-grade Security
STAR Level 1
Certified
GDPR
Compliant
SAML SSO
Single Sign-On
99.9%
Uptime SLA
[Visit TestingBot Trust Center](https://trust.testingbot.com/)
### Ready to start Microsoft Edge Testing?
Start testing your apps with TestingBot.
[Start a free trial](https://testingbot.com/users/sign_up)
---
URL: https://testingbot.com/test-on-safari
Safari cloud
# Test on real Safari, online in the cloud
Run your manual, automated and visual tests on real Safari, from Safari 11 through Safari 27 on macOS, plus Mobile Safari on real iPhones and iPads from iOS 13 to 27. Nobody emulates Safari convincingly, so desktop Safari here is always the real browser on real Apple hardware; for Mobile Safari you can also start on a hosted iOS simulator when you want a fast check.
[Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo)
Free plan, no credit card: 2 live sessions on Safari 27 (macOS Golden Gate). Every Safari and macOS version below is on the paid plans and the 7-day full-access trial.
- **Safari versions:** 11 → 27
- **macOS + iOS devices:** Real
- **Uptime SLA:** 99.9%
Trusted by some of the world's most innovative companies
30-second answer
## Why test Safari on TestingBot?
Real Safari, real Apple hardware
TestingBot streams real Safari running on real macOS and real iPhones / iPads, so there is no emulated WebKit and no "looks-like-Safari" surprises. Hosted iOS simulators are available too when you want a quicker Mobile Safari check.
Every Safari since Safari 11
Test on every Safari version that still ships, from Safari 11 on High Sierra through Safari 27 on Golden Gate. Match what your customers actually use.
Manual, automated, visual
Drive Safari live from your browser, automate it via Selenium, Playwright or Appium, or capture screenshots for visual regression, same grid, same login.
What is Safari?
## Safari in 60 seconds
### Safari
Apple · WebKit · macOS, iOS, iPadOS
Safari is Apple's web browser, built on the WebKit rendering engine. It is the default browser on every macOS, iOS and iPadOS device, and it is the only engine permitted on iOS, so any iPhone or iPad user, on any browser, is effectively running Safari.
That makes Safari coverage non-optional for any consumer-facing site. WebKit's quirks around CSS, ITP (Intelligent Tracking Prevention), date/time inputs and PWAs catch teams off guard if they only test in Chromium during development.
- Built on WebKit, maintained by Apple
- Default browser on macOS, iOS and iPadOS
- ITP, ATS and the strictest cookie policy on the web
- Only browser engine allowed on iOS
- Safari Technology Preview & Safari Beta available
What you can do
## Everything Safari testing needs
Live debugging, automated CI runs, visual regression, extension testing, geolocation — all on the same fleet of real Apple devices.
- ### Live & manual testing
Drive Safari interactively from your browser. Native Web Inspector, network panel, console — same as if you were on a real Mac.
- ### Automated testing
Run Selenium, Playwright or Appium suites against Safari in parallel. Native Safaridriver on desktop, Appium on real iPhones and iPads.
- ### Visual & screenshot testing
Take pixel-perfect screenshots across every Safari version, then diff them automatically with our visual testing service.
- ### Extension testing
Load unsigned Safari App Extensions, Web Extensions (Xcode) and Safari Web Extensions 18.4+ in manual sessions and CI.
- ### Geolocation & ITP
Test from 20+ regions and validate Safari's Intelligent Tracking Prevention behaviour on real exit nodes.
- ### Tech Preview & Beta
Run today's site against tomorrow's Safari. Both Safari Technology Preview and Safari Beta are available alongside stable releases.
Safari versions
## Test On Different Safari Browsers Online
Real Safari shipping with real macOS releases — and Mobile Safari on real iPhones and iPads.
### macOS Desktop
- macOS Golden Gate Safari 27
- macOS Tahoe Safari 26
- macOS Sequoia Safari 18
- macOS Sonoma Safari 18
- macOS Ventura Safari 16
- macOS Monterey Safari 15
- macOS Big Sur Safari 14
- macOS Catalina Safari 13
- macOS Mojave Safari 12
- macOS High Sierra Safari 11
### Mobile Safari
- iOS 27
- iOS 26
- iOS 18
- iOS 17
- iOS 16
- iOS 15
- iOS 14
- iOS 13
- iOS 12
iOS simulator vs real Apple hardware
## When an iOS simulator is enough, and when you need real Apple hardware
If you searched for a Safari emulator, what exists in practice is Apple's iOS Simulator, and TestingBot hosts those for quick Mobile Safari checks. Nothing emulates desktop Safari convincingly, and a simulator's WebKit drifts from the one Apple ships, so release checks belong on real Apple hardware. Here is where the two differ.
| Capability | TestingBot · real Safari | WebKit emulator / simulator |
| --- | --- | --- |
| Browser engine | Real Safari, real WebKit shipped by Apple | Open-source WebKit, drifts from shipped Safari |
| Operating system | Real macOS High Sierra through macOS Golden Gate | Linux container or your dev machine |
| Mobile Safari | Real iPhones and iPads from iOS 13 to 27 | Mobile-emulated user agent only |
| ITP & cookie policy | Real Apple ITP behaviour | Inconsistent or absent |
| Safari extensions | Load unsigned Safari Web Extensions | Not supported |
| Version coverage | Safari 11 → 27, plus Tech Preview & Beta | Latest WebKit only |
Live exit nodes
10+ countries
United States
United Kingdom
Germany
France
Spain
Belgium
Australia
China
Japan
Geolocation testing
## Geolocation Testing on Safari
Test your website with Safari from different regions in the world.
Secure Tunnel
## Test internal & staging apps in Safari
Use the secure [TestingBot Tunnel](https://testingbot.com/support/tunnel) to quickly and easily test your staging website on any Safari browser version on macOS and iOS.
[Read more](https://testingbot.com/support/tunnel)
Mobile Safari · Appium
## Automated Mobile Safari Testing with Appium
Use [Appium](https://testingbot.com/features/automation/appium) to connect to our grid of physical iOS devices and run automated mobile Safari tests against your website.
[Read more about Appium mobile testing](https://testingbot.com/features/automation/appium)
 
Same Appium APIs, real Mobile Safari on real iPhones.
Test from Windows
## Test Safari on Windows, no Mac required
On Windows and need to test something in Safari? You don't have to buy a Mac, set up a virtual machine or chase down a borrowed laptop. TestingBot streams a real macOS desktop straight to your browser, so you can drive any Safari version: from Safari 11 on High Sierra through Safari 27 on Golden Gate, directly from Windows 11, 10, or any older PC. Looking for a Safari emulator for Windows? TestingBot is the better alternative, with real Safari running on real macOS instead of a Windows-based simulator that drifts from what Apple ships.
Open the Web Inspector, switch tabs, validate Intelligent Tracking Prevention, run extensions, every developer tool the real Safari ships with, available from Edge, Chrome or Firefox on your Windows machine.
[Start free](https://testingbot.com/users/sign_up)
Windows
Real macOS
No VM to set up
No Mac to buy
Real Safari dev tools
FAQ
## Frequently Asked Questions
Which Safari versions are supported?
TestingBot supports every Safari from Safari 11 (macOS High Sierra) through Safari 27 (macOS Golden Gate), plus Safari Technology Preview and Safari Beta. On mobile, Mobile Safari is available on real iPhones and iPads from iOS 13 through iOS 27.
Can I use Safari on Windows?
Apple discontinued Safari for Windows in 2012, so there is no modern Safari version that installs natively on a Windows PC. The supported way to test on Safari from Windows today is to drive a real macOS desktop in the cloud. With TestingBot you can stream a real Mac running any Safari version from Safari 11 through Safari 27 directly to Windows 11, 10 or older, with no virtual machine and no Mac purchase. If you were searching for a Safari emulator for Windows, the closest honest equivalent is real Safari on real macOS, streamed to your PC, rather than a WebKit simulation that drifts from what Apple ships.
How can I test Safari without a Mac?
TestingBot streams real macOS desktops to any Windows or Linux browser, so you can test on Safari without owning or maintaining a Mac. You get the actual Safari developer tools, real WebKit rendering and full version coverage from Safari 11 to Safari 27, a cloud replacement for both a Safari emulator and an in-house Mac fleet.
Why do I need to test my website on Safari?
Safari has a market share of about 4% in the global browser market. This browser is built and maintained by Apple and supports all the latest browser technologies. Safari is the default browser on all macOS operating systems, which means a large portion of macOS users will use this browser for their day-to-day browsing. Test Safari's private browsing capabilities, open multiple tabs or test the cross site tracking prevention.
How can I automate Mobile Safari?
We recommend using Appium, which has built-in support to communicate with iOS Safari. Simply run your Appium scripts against the TestingBot device grid. TestingBot will instruct the iOS device to open Safari. Your Appium script will then instrument the mobile Safari browser in an automated way.
What is Safaridriver?
Safaridriver is a utility, shipped by default in every macOS operating system, which allows Selenium to automate Safari through a HTTP based API. To use Safaridriver, you need to make sure it is enabled by running this command in a terminal window: `safaridriver --enable`
How can I test on Mobile Safari?
TestingBot provides access to both iOS simulators and physical iOS devices. Use the developer tools to inspect webpages running on Mobile Safari, straight from TestingBot's Manual Testing service.
Can I test Safari on iPhone and iPad?
TestingBot provides access to physical iOS devices, located in our datacenter. You can run your automated or manual tests against iOS Mobile Safari on our iPhones and iPads.
How to test on older Safari versions?
TestingBot provides access to Safari versions from Safari 11 up to Safari 27.
Why is TestingBot the preferred choice for Safari Testing?
TestingBot provides real Safari browsers, ranging from Safari 11 up to Safari 27. You can use the developer tools that come with Safari to debug webpages, inspect elements and run performance tests. You can test the intelligent tracking prevention, test Safari extensions, open multiple tabs and perform all other actions on a remote Safari instance.
## Ready to start Safari Testing?
Get started with Safari testing today. Run your tests on real Safari browsers and real Apple devices in the TestingBot cloud.
[Start a free trial](https://testingbot.com/users/sign_up)
Resources you may like
[](https://testingbot.com/blog/safari-tech-preview-beta)
### [Test on Safari Tech Preview and Safari beta](https://testingbot.com/blog/safari-tech-preview-beta)
Run Automated and Manual tests on Safari Technology Preview and Safari Beta with TestingBot, on macOS Monterey and macOS Ventura.
[Read more: Test on Safari Tech Preview and Safari beta](https://testingbot.com/blog/safari-tech-preview-beta)
[](https://testingbot.com/blog/ios-18-testing)
### [Test your website and mobile apps on iOS 18.](https://testingbot.com/blog/ios-18-testing)
Ready to start testing on iOS 18? Get started today with TestingBot: run automated, visual and manual tests on iOS 18.
[Read more: Test your website and mobile apps on iOS 18.](https://testingbot.com/blog/ios-18-testing)
[](https://testingbot.com/blog/2016/06/14/safari-technology-preview-automated-testing-with-webdriver)
### [Safari Technology Preview: Automated Testing with WebDriver](https://testingbot.com/blog/2016/06/14/safari-technology-preview-automated-testing-with-webdriver)
[Read more: Safari Technology Preview: Automated Testing with WebDriver](https://testingbot.com/blog/2016/06/14/safari-technology-preview-automated-testing-with-webdriver)
---
URL: https://testingbot.com/test-on-chrome
Chrome cloud
# Test on real Chrome, online in the cloud
An online Chrome browser for manual, automated, responsive and visual testing. Desktop Chrome runs as real, unmodified Google Chrome on real operating systems; for mobile Chrome you can pick a real Android phone or a hosted Android emulator. From Chrome 35 to the latest, plus Chrome Beta, Dev and Canary.
[Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo)
- **Chrome versions:** 35 → latest
- **Platforms:** Win · macOS · Linux · Android
- **Uptime SLA:** 99.9%
Trusted by some of the world's most innovative companies
30-second answer
## Why test Chrome on TestingBot?
Real desktop Chrome, plus Android emulators
On desktop, TestingBot streams unmodified Google Chrome straight from Google on real Windows, macOS and Linux. No headless mode and no Chromium drift. For mobile Chrome, choose a real Android phone when you need hardware fidelity, or a hosted Android emulator when you want a fast check.
Stable, Beta, Dev, Canary
Test on Chrome Stable, Chrome Beta, Chrome Dev and Chrome Canary side by side. Catch regressions before Google rolls them out to your customers.
Manual, automated, responsive, visual
Drive Chrome live from your browser, resize on the fly for responsive testing, automate via Selenium, Playwright or Puppeteer, or capture screenshots for visual regression. Same grid, same login.
What you can do
## Everything Chrome testing needs
Live debugging, automated CI runs, responsive testing, visual regression, extension testing, geolocation, ChromeOS, performance profiling. All on the same fleet of real Chrome browsers.
- ### Live & manual testing
Drive Chrome interactively from your browser. Native DevTools, console, network panel, same as if you were on your own machine.
- ### Automated testing
Run Selenium, Playwright, Puppeteer or Cypress against Chrome in parallel. Native Chromedriver auto-matched to every browser version.
- ### Responsive testing
Resize the browser on the fly during a manual session, or set any viewport from your automation script. Validate breakpoints, mobile menus and fluid layouts on real Chrome.
- ### Visual & screenshot testing
Take pixel-perfect screenshots across every Chrome version, then diff them automatically with the visual testing service.
- ### Extension testing
Side-load unpacked Chrome extensions and CRX bundles in manual sessions and automated runs. Manifest v2 and v3 supported.
- ### Performance & Lighthouse
Capture HAR files, Lighthouse reports and DevTools performance traces. Throttle CPU and network to validate slow-3G behaviour.
- ### Camera, microphone & WebGL
Test getUserMedia with dummy camera and microphone streams, plus WebGL for 3D and canvas-heavy apps, hardware-accelerated on Apple silicon Macs.
- ### ChromeOS testing
Run a ChromeOS emulator hosted on an Android emulator to verify Chromebook behaviour, ChromeOS-specific Chrome quirks and progressive web apps. A practical Chrome OS emulator when you do not have a Chromebook on your desk.
Chrome coverage
## Test On Different Chrome Versions Online
Real Chrome on real Windows, macOS and Linux desktops, plus Chrome on real Android phones.
### Channels & versions
- Chrome Stable latest
- Chrome Beta next
- Chrome Dev future
- Chrome Canary daily
- Older majors 35 → latest
- Auto-updated weekly Google
### Platforms
- Windows 11
- Windows 10
- macOS Tahoe
- macOS Sequoia
- macOS Sonoma
- macOS Ventura
- Linux
- Real Android phones
- ChromeOS (Android emulator)
Chrome emulator vs real Chrome
## When a Chrome emulator is enough, and when you need real Chrome
An emulated or headless Chrome is fine for a fast pass on layout and obvious breakage, and TestingBot hosts Android and ChromeOS emulators for exactly that. Real Chrome on a real operating system is what you want when GPU behaviour, extensions, media playback or the exact shipped Chrome binary matter. Here is where the two differ.
| Capability | TestingBot · real Chrome | Chrome emulator / headless |
| --- | --- | --- |
| Browser binary | Real Google Chrome, signed by Google | Headless Chrome, Chromium build or browser-based emulator |
| Operating system | Real Windows, macOS and Linux | One Linux container or your dev machine |
| Mobile Chrome | Real Chrome on real Android phones | Mobile-emulated user agent only |
| Version coverage | Chrome 35 → latest, plus Beta, Dev and Canary | Whatever Chrome happens to be installed |
| Camera, mic & WebGL | Dummy media streams, WebGL on every platform, GPU-accelerated on macOS | Often disabled or stubbed |
| Video, logs & HAR | HD video, browser logs, HAR and screenshots | Roll your own recording and capture |
Live exit nodes
10+ countries
United States
United Kingdom
Germany
France
Spain
Belgium
Australia
China
Japan
Geolocation testing
## Geolocation Testing on Chrome
Run Chrome from real exit nodes around the world to verify geo-targeting, geo-blocking and localisation from the United States, United Kingdom, Germany, France, Spain, Belgium, Australia, China, Japan and more.
Secure Tunnel
## Test internal & staging Chrome apps securely
Use our [secure tunnel](https://testingbot.com/support/tunnel) to run Chrome tests against firewalled staging sites, internal dashboards or apps still on localhost.
Encrypted end-to-end, no inbound firewall rules required, and built-in support for popular CI providers.
[Read more](https://testingbot.com/support/tunnel)
Mobile Chrome
## Real Chrome on real Android devices
Drive Chrome on real Pixel, Galaxy, OnePlus and Xiaomi devices using Selenium WebDriver, Playwright or Appium. Real hardware, real sensors, accurate results.
[View all devices](https://testingbot.com/support/app-automate/devices)
 
Same WebDriver APIs, real Chrome on real Android phones.
Camera, Microphone & WebGL
## Camera, Microphone and WebGL Testing on Chrome
Test getUserMedia flows with a dummy camera and microphone in real Chrome. No plug-ins, no extra permissions to wire up. Run WebGL tests in the cloud, hardware-accelerated on macOS and software-rendered on Windows and Linux.
### Dummy camera
Drive Chrome with a fake-camera flag so getUserMedia returns a deterministic video stream. Perfect for testing video calls, KYC and webcam-test pages without real hardware.
### Dummy microphone
Use a synthetic audio source for getUserMedia, ideal for chrome microphone test flows, voice recording, audio chat and accessibility scenarios. No real mic required.
### WebGL testing
Run WebGL tests in real Chrome. On macOS (Apple silicon), Chrome renders on the Mac's GPU, so 3D, canvas-heavy SPAs and games are hardware-accelerated. On Windows and Linux, WebGL is software-rendered, which is fine for functional checks but not for measuring graphics performance.
Test from anywhere
## Test Chrome from any OS, even old PCs
Don't have a clean Chrome install? Stuck on a corporate VM, an old PC or a Mac that won't update? TestingBot streams a real desktop running Chrome on Windows 11, 10, macOS or Linux directly to any browser, so you can test on any Chrome version from any device. No installs, no admin rights required.
The streamed Chrome session is the real thing, with full DevTools, profile switching, extension support and Lighthouse audits. You drive it from any browser on your local machine, without installing Chrome or running it locally.
[Start free](https://testingbot.com/users/sign_up)
No install required
No admin rights
Real Chrome DevTools
FAQ
## Frequently Asked Questions
Which Chrome versions are supported?
TestingBot supports every Chrome from Chrome 35 through the current Stable, refreshed within hours of each Google release. Chrome Beta, Chrome Dev and Chrome Canary are available alongside Stable, on Windows, macOS, Linux and real Android devices.
Why do I need to test my website on Chrome?
Chrome is the most popular web browser on the planet, with over 60% market share globally. You want to make sure that your website looks and functions correctly on this browser. The browser is built and maintained by Google and is used all around the world, on Windows, macOS, Linux, ChromeOS and Android. Test cookie behaviour, ad-blockers, the third-party cookie phase-out and Chrome's progressive Privacy Sandbox features.
What is the difference between Chrome, Chromium, Chrome Beta and Chrome Canary?
Chrome (Stable) is the browser everyone uses for day-to-day browsing. It is the same binary TestingBot provides for testing. Chromium uses the same browser engine but ships without Google's branding and proprietary components, and it is open-source. Chrome Beta is the next major Chrome release, roughly 4 to 6 weeks ahead of Stable. Chrome Dev sits one step further out, and Chrome Canary is the bleeding-edge daily build. TestingBot lets you run all four channels side by side so you can spot regressions before Google promotes them to Stable.
What is Chromedriver?
Chromedriver is a program that automates the Chrome browser through the W3C WebDriver protocol. Selenium, Puppeteer (over the Chrome DevTools Protocol) and Playwright all use it to drive Chrome. Chromedriver versions are tightly coupled to Chrome versions, which means you need to make sure the Chromedriver version matches the browser version. At TestingBot this is handled automatically. Pick a Chrome version and the matching Chromedriver is provisioned for you.
Can I test Chrome on real Android devices?
Yes. TestingBot ships real Pixel, Galaxy, OnePlus and Xiaomi devices running real Chrome for Android. Drive them with Selenium WebDriver or Appium and use the actual mobile rendering engine, mobile DevTools and real touch input.
Does TestingBot offer a Chrome emulator, or real Chrome?
Both, and the distinction matters. Desktop Chrome on TestingBot is real Chrome, downloaded straight from Google and running on real Windows, macOS or Linux, not a headless container. For mobile Chrome we offer real Android phones and hosted Android emulators, plus a ChromeOS emulator for Chromebook behaviour. Sessions run inside dedicated single-use virtual machines on Windows, macOS or Linux, with real Chrome DevTools and the same Chrome binary your customers use. If you have been searching for an "online Chrome emulator" or a "Chrome browser online" tool to validate a site, TestingBot gives you the real thing instead.
Can I test Chrome extensions?
Yes. Side-load unpacked extensions and CRX bundles in manual sessions for click-through testing, or load them automatically in Selenium, Playwright and Puppeteer runs. Manifest v2 and v3 are both supported.
Can I run Lighthouse on TestingBot?
Yes. Run Lighthouse audits inside any Chrome session, capture the JSON report as a build artifact, and trend Core Web Vitals over time. Throttle CPU and network from the test options to simulate slow connections.
## Ready to start Chrome Testing?
Start testing your website on real Chrome in the cloud. Every version, every platform, every release channel.
[Start a free trial](https://testingbot.com/users/sign_up)
---
URL: https://testingbot.com/test-on-iphone
Real iPhone cloud
# iPhone testing online, on simulators or real iPhones
Run manual, automated, visual and responsive tests on iPhone in the TestingBot cloud. Start on an iOS simulator when you want a fast first signal, then run the same tests on real iPhones before you ship. From iPhone 6s through iPhone 17, with real Mobile Safari, real iOS apps and real touch input. Nothing to install, no Mac and no jailbroken devices.
[Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo)
- **iPhone models:** 6s to 17
- **iOS versions:** 12 to 27
- **Uptime SLA:** 99.9%
_A real iPhone running Mobile Safari with animated tap gestures_
Trusted by some of the world's most innovative companies
30-second answer
## Why test on real iPhones with TestingBot?
iOS simulators and real iPhones
Launch a hosted iOS simulator in seconds for quick checks, or connect to a physical, unaltered iPhone in our datacenter when you need real Apple silicon, real Mobile Safari and real touch and motion sensors. Both run from the same browser session, with no Mac and no Xcode to install.
Every iPhone since the 6s
Pick from iPhone 6s, SE, 7, 8, X, XR, XS, 11, 12, 13, 14, 15, 16 and 17 across iOS 12 through iOS 27. Match what your customers actually carry in their pockets.
Manual, automated, app & web
Drive an iPhone live from your browser, run Appium suites against the device cloud, capture screenshots for visual regression, or upload your IPA and exercise the real iOS app.
What you can do
## Everything iPhone testing needs
Live debugging, automated Appium runs, mobile-web and native-app testing, gestures, network throttling, geolocation. All on a fleet of real iPhones in our datacenter.
- ### Live & manual testing
Take an iPhone in your browser. Tap, swipe, type, rotate and inspect, with full DevTools through Mobile Safari. Same as if you were holding the phone in your hand.
- ### Appium automation
Run Appium 1, 2 and 3 suites against real iPhones in parallel. WebDriverAgent, XCUITest and full XCTest support, no Mac required on your side.
- ### Native iOS app testing
Upload your IPA, install it in seconds, and exercise it on real iPhones. App permissions, biometrics, keyboards, push and deep links all behave like real hardware.
- ### Real Mobile Safari
Test your website on Mobile Safari, the only browser engine Apple permits on iOS, on real Apple hardware. The Safari your iPhone customers actually use.
- ### Visual & screenshot testing
Capture pixel-perfect screenshots across every iPhone model and iOS version, then diff them automatically with the visual testing service.
- ### Tap, swipe, pinch and rotate
Drive every multi-touch gesture exactly as a customer would, including pinch zoom, long-press, scroll, three-finger swipes and device rotation.
- ### Network throttling
Test on WiFi, 5G, 4G, 3G, Edge and offline conditions. Catch slow-network bugs before App Store reviewers do.
- ### Responsive testing
Switch between portrait and landscape, swap between iPhone Pro Max and iPhone SE in one click, and validate breakpoints on real Safari.
Device coverage
## Test On Different iPhones Online
Real iPhones from iPhone 6s through the latest iPhone 17, plus iPad. Real Apple silicon, real Mobile Safari, real touch input.
### iPhone models
- [iPhone 17 Pro Max](https://testingbot.com/test-on-iphone/apple-iphone-17-pro-max)
- [iPhone 17 Pro](https://testingbot.com/test-on-iphone/apple-iphone-17-pro)
- [iPhone 16 Pro Max](https://testingbot.com/test-on-iphone/apple-iphone-16-pro-max)
- [iPhone 16](https://testingbot.com/test-on-iphone/apple-iphone-16)
- [iPhone 15 Pro](https://testingbot.com/test-on-iphone/apple-iphone-15-pro)
- [iPhone 15](https://testingbot.com/test-on-iphone/apple-iphone-15)
- [iPhone 14 Pro](https://testingbot.com/test-on-iphone/apple-iphone-14-pro)
- [iPhone 14](https://testingbot.com/test-on-iphone/apple-iphone-14)
- [iPhone 13 Pro Max](https://testingbot.com/test-on-iphone/apple-iphone-13-pro-max)
- [iPhone 13 Pro](https://testingbot.com/test-on-iphone/apple-iphone-13-pro)
- [iPhone 13](https://testingbot.com/test-on-iphone/apple-iphone-13)
- [iPhone 13 mini](https://testingbot.com/test-on-iphone/apple-iphone-13-mini)
- [iPhone 12 Pro Max](https://testingbot.com/test-on-iphone/apple-iphone-12-pro-max)
- [iPhone 12](https://testingbot.com/test-on-iphone/apple-iphone-12)
- [iPhone 12 mini](https://testingbot.com/test-on-iphone/apple-iphone-12-mini)
- [iPhone 11 Pro Max](https://testingbot.com/test-on-iphone/apple-iphone-11-pro-max)
- [iPhone 11 Pro](https://testingbot.com/test-on-iphone/apple-iphone-11-pro)
- [iPhone 11](https://testingbot.com/test-on-iphone/apple-iphone-11)
- [iPhone XR](https://testingbot.com/test-on-iphone/apple-iphone-xr)
- [iPhone XS Max](https://testingbot.com/test-on-iphone/apple-iphone-xs-max)
- [iPhone X](https://testingbot.com/test-on-iphone/apple-iphone-x)
- [iPhone SE (2022)](https://testingbot.com/test-on-iphone/apple-iphone-se-2022)
- [iPhone SE (2020)](https://testingbot.com/test-on-iphone/apple-iphone-se-2020)
- [iPhone 8 Plus](https://testingbot.com/test-on-iphone/apple-iphone-8-plus)
- [iPhone 8](https://testingbot.com/test-on-iphone/apple-iphone-8)
- [iPhone 7](https://testingbot.com/test-on-iphone/apple-iphone-7)
- [iPhone 6s Plus](https://testingbot.com/test-on-iphone/apple-iphone-6s-plus)
- [iPhone 6s](https://testingbot.com/test-on-iphone/apple-iphone-6s)
[View all iPhones and iPads](https://testingbot.com/support/app-automate/devices)
### iOS versions
- iOS 27
- iOS 26
- iOS 18
- iOS 17
- iOS 16
- iOS 15
- iOS 14
- iOS 13
- iOS 12
Latest iOS releases land on the cloud within hours of Apple shipping them. Older majors stay around for legacy regression coverage.
New · iOS 27.1 simulator
### iPhone Duo, before it ships
Apple has not released the iPhone Duo yet, but you can already test on it as an iOS 27.1 simulator. See both the cover screen and the inner screen, fold and unfold with one click, and debug with Safari Web Inspector. It runs as a simulator, not a real device.
[Start a live test on iPhone Duo](https://testingbot.com/members/manual/new)
iOS simulator vs real iPhone
## When an iOS simulator is enough, and when you need a real iPhone
TestingBot gives you both. An iOS simulator is the right tool for a fast pass on layout, navigation and obvious breakage, and it is cheap to run on every commit. A real iPhone is what you want before release, because a simulator cannot reproduce Apple silicon performance, the shipped WebKit, biometrics or real touch. Here is where the two differ.
| Capability | TestingBot, real iPhones | iOS simulator |
| --- | --- | --- |
| Hardware | Real Apple silicon, real iPhone in our datacenter | x86 or ARM Mac, no real Apple radios or sensors |
| Mobile Safari | Real WebKit shipped on the device by Apple | Simulator WebKit, drifts from shipped Safari |
| Touch & gestures | Real multi-touch, real haptics, real rotation | Mouse-faked taps, no haptics |
| Native apps | Real iOS, real frameworks, real ClassKit and StoreKit | Some Apple frameworks missing or stubbed |
| Device coverage | iPhone 6s through iPhone 17, iOS 12 to 27 | Hosted iOS simulator images, no Mac or Xcode install |
| Video, logs and HAR | HD video, device logs, HAR and screenshots | Roll your own recording and capture |
Native iOS apps
## Native iOS app testing
Upload your IPA and TestingBot installs it on a real iPhone in seconds. Drive the app live or via Appium, capture full HD video, syslog and crash reports, and exercise biometrics, push notifications, deep links and the iOS keyboard exactly as your customers will.
Every iPhone above is physical hardware racked in our European datacenter, not a simulator. The [real device testing](https://testingbot.com/real-device-testing) overview lists the full fleet, explains how each handset is wiped and re-imaged between sessions, and shows which plans include it.
[Read more](https://testingbot.com/features/automation/appium)
upload-and-test.js
// 1) Upload your IPA to TestingBot storageconst TestingBot = require('testingbot-api'); const tb = new TestingBot({ api\_key: 'YOUR\_KEY', api\_secret: 'YOUR\_SECRET' }); tb.uploadFile('./MyApp.ipa', (err, res) =\> { // res.app\_url → "tb://..."// 2) Run your Appium test on a real iPhone driver.init({ platformName: 'iOS', 'appium:deviceName': 'iPhone 17', 'appium:platformVersion': '27', 'appium:realDevice': true, 'appium:app': res.app\_url }); });
Live exit nodes
10+ countries
United States
United Kingdom
Germany
France
Spain
Belgium
Australia
China
Japan
Geolocation testing
## Geolocation Testing on iPhones
Run iPhone tests from real exit nodes around the world to verify geo-targeting, geo-blocking and localisation from the United States, United Kingdom, Germany, France, Spain, Belgium, Australia, China, Japan and more.
Secure Tunnel
## Test internal & staging sites on real iPhones
Use our [secure tunnel](https://testingbot.com/support/tunnel) to run iPhone tests against firewalled staging sites, internal dashboards or apps still on localhost.
Encrypted end-to-end, no inbound firewall rules required, ready out of the box for popular CI providers.
[Read more](https://testingbot.com/support/tunnel)
Secure devices
## Real iPhones. Wiped after every session.
Each session locks a single iPhone for your test only. The moment your test ends, the device is wiped, all installed apps are deleted, caches are cleared, and the iPhone is returned to the pool ready for the next customer.
GDPR-compliant, with SAML SSO and audit logs available for enterprise teams.
- One real iPhone per session, never shared
- Wiped between every session
- Encrypted IPA uploads with per-app retention controls
1 customer per session
Datacenter hosted
Wiped after each test
CSA STAR Level 1
GDPR compliant
SAML SSO enterprise
Test without a Mac
## Test on iPhones from any OS, no Mac required
On Windows or Linux and need to validate something on an iPhone? You don't have to buy a Mac, set up Xcode, or chase down a borrowed iPad. TestingBot streams a real iPhone from our datacenter directly to any browser, so you can drive a physical iPhone from Windows 11, 10, Linux or any older PC.
The streamed session is the real device, with Mobile Safari, the iOS keyboard, biometrics and notifications. You drive it from any browser on your local machine, without installing Xcode or owning Apple hardware.
[Start free](https://testingbot.com/users/sign_up)
No Mac to buy
No Xcode to install
Real iPhone in your browser
FAQ
## Frequently Asked Questions
Which iPhones and iOS versions are supported?
TestingBot supports physical iPhones from iPhone 6s through iPhone 17, including the Pro and Pro Max line, plus the iPhone SE family. iOS versions range from iOS 12 through iOS 27. New flagship phones are added on or close to the day Apple ships them.
How do I test a mobile app on iPhone?
Testing your iOS app on a variety of iOS devices is important before publishing to the App Store. TestingBot lets you upload your IPA and install it on a number of different iPhones in our datacenter. Choose between various iOS versions and screen sizes to make sure your mobile app works correctly on every iPhone your customers might own.
Why is testing your website on an iPhone necessary?
iPhone has roughly a 28% global mobile market share, much higher in the United States, the United Kingdom, Japan and Australia. Your website needs to look and behave correctly on Mobile Safari for those users. Mobile Safari is the only browser engine Apple permits on iOS, so testing on Chrome, Firefox or any other browser does not represent the experience your iPhone customers actually have.
Should I use an iOS simulator or a real iPhone?
Use both, at different points. An iOS simulator is fast to launch and cheap to run on every commit, and it is perfectly good for catching layout breakage, broken navigation and obvious regressions. TestingBot hosts iOS simulators so you can do that without a Mac or an Xcode install. Before you ship, move to a real iPhone. A simulator does not match real device performance, several Apple frameworks (StoreKit, ClassKit, biometrics, push) are missing or stubbed, and the WebKit it ships drifts from the WebKit on a retail iPhone. The practical split is simulators for speed during development, real devices for release confidence.
Can I test Mobile Safari on a real iPhone?
Yes. TestingBot provides physical, unaltered iPhones, straight from the store and never jailbroken. Run your manual or automated tests against real Mobile Safari on a physical iPhone in our datacenter, with full Web Inspector, network panel and console.
Can I test on iPhone without owning a Mac?
Yes. TestingBot streams real iPhones to any Windows, macOS or Linux browser. You get the real device, real Mobile Safari and real iOS, without owning a Mac, an iPhone or a copy of Xcode.
Can I run Appium against your iPhone cloud?
Yes. TestingBot supports Appium 1, 2 and 3 against real iPhones, with full XCUITest and WebDriverAgent. Point your existing Appium suite at our hub and pick a device through capabilities.
Can I test on a foldable iPhone?
Yes. The iPhone Duo is available on TestingBot as an iOS 27.1 simulator for live and automated testing, before Apple releases the device. Switch between the cover screen and the inner screen during a live test to check how your layout adapts. For automated tests, set deviceName to "iPhone Duo" and version to "27.1".
## Ready to test on real iPhones?
Get started with iPhone testing today. Run your tests on hosted iOS simulators or on real iPhones in the TestingBot cloud, with nothing to install and no jailbroken devices.
[Start a free trial](https://testingbot.com/users/sign_up)
---
URL: https://testingbot.com/integrations
36+ Integrations
# One platform, every tool in your stack
Plug TestingBot into the CI, project management and testing tools your team already uses. From Jenkins and GitHub Actions to Jira and Slack, 36 integrations are ready to wire up in minutes.
[Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo)
- **Integrations:** 36+
- **Categories:** 8
- **Time to first run:** \<10 min
Trusted by some of the world's most innovative companies
Most popular
## Top integrations teams set up on day one
A short list our customers wire up on day one.
- [ ](https://testingbot.com/support/web-automate/selenium)
### [Selenium](https://testingbot.com/support/web-automate/selenium)
Run your Selenium tests on a cloud of 6100+ browsers.
[Try with Selenium](https://testingbot.com/support/web-automate/selenium)
- [ ](https://testingbot.com/support/app-automate/appium)
### [Appium](https://testingbot.com/support/app-automate/appium)
Native, hybrid and web app testing on real iOS and Android.
[Try with Appium](https://testingbot.com/support/app-automate/appium)
- [ ](https://testingbot.com/support/integrations/ci-cd/jenkins)
### [Jenkins](https://testingbot.com/support/integrations/ci-cd/jenkins)
Drop-in pipeline plugin with tunnel and parallel support.
[Try with Jenkins](https://testingbot.com/support/integrations/ci-cd/jenkins)
- [ ](https://testingbot.com/support/integrations/ci-cd/github-actions)
### [GitHub Actions](https://testingbot.com/support/integrations/ci-cd/github-actions)
Launch TestingBot runs from any workflow with one action.
[Try with GitHub Actions](https://testingbot.com/support/integrations/ci-cd/github-actions)
- [ ](https://testingbot.com/support/integrations/slack)
### [Slack](https://testingbot.com/support/integrations/slack)
Get notified in any channel when a test finishes or fails.
[Try with Slack](https://testingbot.com/support/integrations/slack)
- [ ](https://testingbot.com/support/integrations/jira)
### [Jira](https://testingbot.com/support/integrations/jira)
Create issues with test meta-data, screenshots and video.
[Try with Jira](https://testingbot.com/support/integrations/jira)
- [ ](https://testingbot.com/support/integrations/ci-cd/gitlab)
### [GitLab](https://testingbot.com/support/integrations/ci-cd/gitlab)
GitLab CI integration with built-in test reporting.
[Try with GitLab](https://testingbot.com/support/integrations/ci-cd/gitlab)
- [ ](https://testingbot.com/support/integrations/ci-cd/circleci)
### [CircleCI](https://testingbot.com/support/integrations/ci-cd/circleci)
CircleCI orb for one-line setup in your config.yml.
[Try with CircleCI](https://testingbot.com/support/integrations/ci-cd/circleci)
### Categories
[Automation Framework 10](https://testingbot.com#automation) [Project Management 9](https://testingbot.com#project) [CI/CD 10](https://testingbot.com#ci) [Record and Playback 7](https://testingbot.com#record) [App Distribution 2](https://testingbot.com#distribution) [Browser Extension 1](https://testingbot.com#extension) [Marketplace 4](https://testingbot.com#marketplace) [Other 1](https://testingbot.com#other)
[Request integration](https://testingbot.com/contact/new)
## Automation Framework
Run tests with popular testing frameworks
### [Selenium](https://testingbot.com/support/web-automate/selenium)
Run tests on our grid with 6100+ browsers
### [Appium](https://testingbot.com/support/app-automate/appium)
Test native, hybrid and webapps on real devices
### [Cypress](https://testingbot.com/support/web-automate/cypress)
Run Cypress tests on macOS and Windows
### [Puppeteer](https://testingbot.com/support/web-automate/puppeteer)
Testing with Puppeteer in the cloud
### [Playwright](https://testingbot.com/support/web-automate/playwright)
Testing with Playwright in the cloud
### [Espresso](https://testingbot.com/support/app-automate/espresso)
Mobile app testing on Android
### [XCUITest](https://testingbot.com/support/app-automate/xcuitest)
Automated testing on iOS
### [Maestro](https://testingbot.com/support/app-automate/maestro)
Testing on iOS and Android
### [Vibium](https://testingbot.com/support/web-automate/vibium)
AI-agent browser automation in the cloud
### [Mobilewright](https://testingbot.com/support/app-automate/mobilewright)
Playwright-style mobile tests on iOS and Android
## Project Management
Connect with your favorite project tools
### [Slack](https://testingbot.com/support/integrations/slack)
Get notified when tests finish or fail
### [WebHooks](https://testingbot.com/support/integrations/webhooks)
Receive an HTTP callback on test events
### [incident.io](https://testingbot.com/support/integrations/webhooks/integrations/incident-io)
Fire alerts and page on-call for failed tests
### [PagerDuty](https://testingbot.com/support/integrations/webhooks/integrations/pagerduty)
Open incidents for failed tests
### [OpsGenie](https://testingbot.com/support/integrations/webhooks/integrations/opsgenie)
Create alerts for failed tests
### [Jira](https://testingbot.com/support/integrations/jira)
Create issues with test meta-data
### [GitHub](https://testingbot.com/support/integrations/github-issue-tracker)
Create issues with test meta-data
### [Azure DevOps](https://testingbot.com/support/integrations/azure-devops-issue-tracker)
Create work items with test meta-data
### [Trello](https://testingbot.com/support/integrations/trello-issue-tracker)
Create cards from test failures
### [Asana](https://testingbot.com/support/integrations/asana-issue-tracker)
Create tasks from test failures
### [Linear](https://testingbot.com/support/integrations/linear-issue-tracker)
Create issues from test failures
### [Bugsnag](https://testingbot.com/support/integrations/bugsnag)
Create issues with screenshots
### [ZebRunner](https://testingbot.com/support/integrations/zebrunner)
Smart analysis of test results
## CI/CD
Integrate with your continuous integration pipeline
### [TeamCity](https://testingbot.com/support/integrations/ci-cd/teamcity)
Plugin for JetBrains TeamCity
### [Jenkins](https://testingbot.com/support/integrations/ci-cd/jenkins)
Pipeline + tunnel support
### [Bamboo](https://testingbot.com/support/integrations/ci-cd/bamboo)
Atlassian Bamboo plugin
### [Azure DevOps](https://testingbot.com/support/integrations/ci-cd/azure)
Integrate with VSTS
### [CircleCI](https://testingbot.com/support/integrations/ci-cd/circleci)
Integrate with CircleCI
### [Bitbucket](https://testingbot.com/support/integrations/ci-cd/bitbucket)
Bitbucket Pipelines integration
### [GitLab](https://testingbot.com/support/integrations/ci-cd/gitlab)
Integrate with GitLab CI
### [Travis CI](https://testingbot.com/support/integrations/ci-cd/travis-ci)
Integrate with Travis CI
### [GitHub Actions](https://testingbot.com/support/integrations/ci-cd/github-actions)
Launch tests from workflows
### [Bitrise](https://testingbot.com/support/integrations/ci-cd/bitrise)
Mobile app CI/CD integration
### [Buildkite](https://testingbot.com/support/integrations/ci-cd/buildkite)
Tunnel, statuses and annotations
### [Gradle](https://testingbot.com/support/integrations/ci-cd/gradle)
Espresso tests from your build
### [AWS CodePipeline](https://testingbot.com/support/integrations/ci-cd/aws-codepipeline)
CodeBuild buildspec with tunnel
## Record and Playback
Create tests without writing code
### [Selenium IDE](https://testingbot.com/support/web-automate/codeless-automation/add-test)
Record with browser extension
### [Katalon Studio](https://testingbot.com/support/web-automate/selenium/katalon-studio)
Record and run on TestingBot
### [Oxygen IDE](https://testingbot.com/support/web-automate/selenium/oxygen)
Record and playback tests
### [Ranorex Studio](https://testingbot.com/support/integrations/ranorex)
Record and run tests
### [Cerberus](https://testingbot.com/support/integrations/cerberus)
Run Cerberus tests
### [Tricentis Tosca](https://testingbot.com/support/integrations/tricentis-tosca)
Run Tosca tests
### [QMetry](https://testingbot.com/support/integrations/qmetry)
Codeless test automation
## App Distribution
Test apps from distribution platforms
### [App Center](https://testingbot.com/support/integrations/appcenter)
Test apps from Microsoft App Center
### [Firebase](https://testingbot.com/support/integrations/firebase-app-distribution)
Test releases from Firebase App Distribution
### [Fastlane](https://testingbot.com/support/integrations/fastlane)
Upload app builds from your Fastlane lanes
## Browser Extension
Test instantly from your browser
### [TestingBot Extension](https://testingbot.com/support/web-live/extension)
Test any webpage instantly
## Marketplace
Available in popular marketplaces
### [Shopify](https://apps.shopify.com/testingbot-1)
Test your Shopify store
### [Semrush](https://www.semrush.com/apps/ai-website-testing/)
Test on 100+ browser combinations
### [Wix](https://www.wix.com/app-market/web-solution/website-testing)
Test your Wix website
### [AWS Marketplace](https://aws.amazon.com/marketplace/seller-profile?id=seller-k6mxju5cqvrp4)
License through AWS
## Other
Additional tools and integrations
### [Appium Inspector](https://testingbot.com/support/integrations/appium-desktop)
Visualize and inspect mobile app elements
How it works
## From signup to first integration in under 10 minutes
1. 1
### Grab your API key
Sign up for a free account and copy your key and secret from the dashboard.
2. 2
### Pick an integration
Browse the catalog below and follow the per-tool setup guide. Most are a single config block or one CLI flag.
3. 3
### Run your first test
Trigger a build, push a commit, or click "Run" in your tool. Logs, video and screenshots stream straight back into TestingBot.
Don't see your tool?
## Build a custom integration with our REST API
Every integration on this page is built on the same public REST API and webhooks you can use. Drop us a note if you want us to ship a first-class plugin for your tool.
[Read API docs](https://testingbot.com/support/api) [Request an integration](https://testingbot.com/contact/new)
FAQ
## Common questions about integrations
Is there a public API I can build my own integration against?
Yes. Every integration on this page is built on the same REST API and webhook events that you can call. The full reference, including authentication, rate limits and webhook payloads, lives in the TestingBot support docs.
Can you build a custom integration for our internal tool?
We do build first-class integrations on request when there's broad demand. For internal tools, the public REST API plus webhooks is usually the fastest path, and most internal CI or dashboard integrations take an afternoon. Contact us with your use case and we'll point you at the right endpoints.
Do the integrations cost extra?
No. Every integration listed on this page is included in every paid plan and the free trial. There are no per-seat or per-integration fees.
How do integrations authenticate with TestingBot?
Most integrations use your TestingBot API key and secret, which you can find on the dashboard. CI plugins and the Slack/Jira/Bugsnag connectors authenticate via OAuth or signed webhooks where the target tool supports it.
Will the CI/CD plugins work with self-hosted Jenkins, GitLab or Bitbucket?
Yes. The plugins talk to our cloud over HTTPS and don't require an inbound connection. For tests that need to reach a private staging environment, install TestingBot Tunnel on the same network as your build agent.
How do I request a new integration?
Open the "Request integration" link on this page or contact support. We prioritise integrations by customer demand, so when we ship a new one, customers who asked for it are notified first.
---
URL: https://testingbot.com/test-on-ipad
Real iPad cloud
# iPad testing online, on simulators or real iPads
Run manual, automated, visual and responsive tests on iPad in the TestingBot cloud. Start on an iPadOS simulator when you want a fast first signal, then run the same tests on real iPads before you ship. iPad Pro, iPad Air, iPad mini and iPad 6th through 11th generation, with real Mobile Safari, real iPadOS apps and real multi-touch input. Nothing to install, no Mac and no jailbroken devices.
[Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo)
- **iPad models:** 6th gen → 11
- **iPadOS versions:** 12 to 27
- **Uptime SLA:** 99.9%
_A real iPad running Mobile Safari with animated multi-touch gestures_
Trusted by some of the world's most innovative companies
30-second answer
## Why test on real iPads with TestingBot?
iPadOS simulators and real iPads
Launch a hosted iPadOS simulator in seconds for quick checks, or connect to a physical, unaltered iPad in our datacenter when you need real Apple silicon, real Mobile Safari, real multi-touch and real iPadOS. Both run from the same browser session, with no Mac and no Xcode to install.
Every iPad worth testing on
Pick from iPad Pro 12.9" M2, iPad Pro 11" M4, iPad Air, iPad mini and iPad 6th through 11th generation, across iPadOS 12 through iPadOS 27. Match what your customers actually use.
Manual, automated, app & web
Drive an iPad live from your browser, run Appium suites against the device cloud, capture screenshots for visual regression across portrait and landscape, or upload your IPA and exercise the real iPadOS app.
What you can do
## Everything iPad testing needs
Live debugging, automated Appium runs, mobile-web and native-app testing, multi-touch gestures, split view, network throttling and geolocation. All on a fleet of real iPads in our datacenter.
- ### Live & manual testing
Take an iPad in your browser. Tap, swipe, pinch, rotate and inspect, with full DevTools through Mobile Safari. Same as if you were holding the iPad in your hands.
- ### Appium automation
Run Appium 1, 2 and 3 suites against real iPads in parallel. WebDriverAgent, XCUITest and full XCTest support, no Mac required on your side.
- ### Native iPadOS app testing
Upload your IPA, install it in seconds, and exercise it on real iPads. App permissions, keyboards, push notifications and deep links all behave like real hardware.
- ### Real Mobile Safari
Test your website on Mobile Safari, the only browser engine Apple permits on iPadOS, on real Apple hardware. The Safari your iPad customers actually use, including the "Request Desktop Site" path.
- ### Visual & screenshot testing
Capture pixel-perfect screenshots across every iPad model and iPadOS version, in portrait and landscape, then diff them automatically with our visual testing service.
- ### Tap, swipe, pinch & rotate
Drive every multi-touch gesture exactly as a customer would, including pinch zoom, long-press, three- and four-finger swipes, split-view drag and device rotation.
- ### Network throttling
Test on WiFi, 5G, 4G, 3G, Edge and offline conditions. Catch slow-network bugs on cellular iPads before App Store reviewers do.
- ### Portrait & landscape
Rotate the iPad in one click, switch between iPad mini 8.3" and iPad Pro 12.9", and validate every breakpoint on real Mobile Safari.
Device coverage
## Test On Different iPads Online
iPad Pro, iPad Air, iPad mini and iPad 6th through 11th generation. Real Apple silicon, real Mobile Safari, real multi-touch input.
### iPad models
- [iPad Pro 12.9" M2](https://testingbot.com/test-on-ipad/apple-ipad-pro-12-9-m2)
- [iPad Pro 11" M4](https://testingbot.com/test-on-ipad/apple-ipad-pro-11-m4)
- [iPad Air 13" M2](https://testingbot.com/test-on-ipad/apple-ipad-air-13-m2)
- [iPad Air 11" M2](https://testingbot.com/test-on-ipad/apple-ipad-air-11-m2)
- [iPad 11th gen](https://testingbot.com/test-on-ipad/apple-ipad-11)
- [iPad 10th gen](https://testingbot.com/test-on-ipad/apple-ipad-10)
- [iPad 9th gen](https://testingbot.com/test-on-ipad/apple-ipad-9)
- [iPad 8th gen](https://testingbot.com/test-on-ipad/apple-ipad-8)
- [iPad 7th gen](https://testingbot.com/test-on-ipad/apple-ipad-7)
- [iPad 6th gen](https://testingbot.com/test-on-ipad/apple-ipad-6)
- [iPad mini 7](https://testingbot.com/test-on-ipad/apple-ipad-mini-7)
- [iPad mini 6](https://testingbot.com/test-on-ipad/apple-ipad-mini-6)
[View all iPads and iPhones](https://testingbot.com/support/app-automate/devices)
### iPadOS versions
- iPadOS 27
- iPadOS 26
- iPadOS 18
- iPadOS 17
- iPadOS 16
- iPadOS 15
- iPadOS 14
- iPadOS 13
- iPadOS 12
Latest iPadOS releases land on the cloud within hours of Apple shipping them. Older majors stay around for legacy regression coverage.
iPadOS simulator vs real iPad
## When an iPad simulator is enough, and when you need a real iPad
TestingBot gives you both. An iPadOS simulator is the right tool for a fast pass on layout, navigation and obvious breakage, and it is cheap to run on every commit. A real iPad is what you want before release, because a simulator cannot reproduce Apple silicon performance, the shipped WebKit, biometrics or real multi-touch. Here is where the two differ.
| Capability | TestingBot, real iPads | iPadOS simulator |
| --- | --- | --- |
| Hardware | Real Apple silicon, real iPad in our datacenter | x86 or ARM Mac, no real Apple radios or sensors |
| Mobile Safari | Real WebKit shipped on the device by Apple | Simulator WebKit, drifts from shipped Safari |
| Multi-touch & gestures | Real multi-touch, split view, drag-and-drop, rotation | Mouse-faked taps, no real multi-touch |
| Native apps | Real iPadOS, real StoreKit, ClassKit and HealthKit | Some Apple frameworks missing or stubbed |
| Device coverage | iPad Pro, Air, mini and iPad 6th to 11th generation | Hosted iPadOS simulator images, no Mac or Xcode install |
| Video, logs and HAR | HD video, device logs, HAR and screenshots | Roll your own recording and capture |
Native iPadOS apps
## Native iPadOS app testing
Upload your IPA and TestingBot installs it on a real iPad in seconds. Drive the app live or via Appium, capture full HD video, syslog and crash reports, and exercise push notifications, deep links and the iPadOS keyboard exactly as your customers will.
[Read more](https://testingbot.com/features/automation/appium)
upload-and-test.js
// 1) Upload your IPA to TestingBot storageconst TestingBot = require('testingbot-api'); const tb = new TestingBot({ api\_key: 'YOUR\_KEY', api\_secret: 'YOUR\_SECRET' }); tb.uploadFile('./MyApp.ipa', (err, res) =\> { // res.app\_url → "tb://..."// 2) Run your Appium test on a real iPad driver.init({ platformName: 'iOS', 'appium:deviceName': 'iPad 9th generation', 'appium:platformVersion': '27', 'appium:realDevice': true, 'appium:app': res.app\_url }); });
Live exit nodes
10+ countries
United States
United Kingdom
Germany
France
Spain
Belgium
Australia
China
Japan
Geolocation testing
## Geolocation Testing on iPads
Run iPad tests from real exit nodes around the world to verify geo-targeting, geo-blocking and localisation from the United States, United Kingdom, Germany, France, Spain, Belgium, Australia, China, Japan and more.
Secure Tunnel
## Test internal & staging sites on real iPads
Use our [secure tunnel](https://testingbot.com/support/tunnel) to run iPad tests against firewalled staging sites, internal dashboards or apps still on localhost.
Encrypted end-to-end, no inbound firewall rules required, ready out of the box for popular CI providers.
[Read more](https://testingbot.com/support/tunnel)
Secure devices
## Real iPads. Wiped after every session.
Each session locks a single iPad for your test only. The moment your test ends, the device is wiped, all installed apps are deleted, caches are cleared, and the iPad is returned to the pool ready for the next customer.
GDPR-compliant, with SAML SSO and audit logs available for enterprise teams.
- One real iPad per session, never shared
- Wiped between every session
- Encrypted IPA uploads with per-app retention controls
1 customer per session
Datacenter hosted
Wiped after each test
CSA STAR Level 1
GDPR compliant
SAML SSO enterprise
Test without a Mac
## Test on iPads from any OS, no Mac required
On Windows or Linux and need to validate something on an iPad? You don't have to buy a Mac, set up Xcode, or chase down a borrowed iPad. TestingBot streams a real iPad from our datacenter directly to any browser, so you can drive a physical iPad Pro, iPad Air, iPad mini or any older generation from Windows 11, 10, Linux or any PC.
The streamed session is the real device, with Mobile Safari, the iPadOS keyboard, gestures and notifications. You drive it from any browser on your local machine, without installing Xcode or owning Apple hardware.
[Start free](https://testingbot.com/users/sign_up)
No Mac to buy
No Xcode to install
Real iPad in your browser
FAQ
## Frequently Asked Questions
Which iPads and iPadOS versions are supported?
TestingBot supports physical iPads from iPad 6th generation through the latest iPad Pro and iPad Air models, including the iPad mini line. iPadOS versions range from iPadOS 12 through iPadOS 27. New iPads are added on or close to the day Apple ships them.
How do I test a mobile app on iPad?
Testing your iPadOS app on a variety of iPad devices is important before publishing to the App Store. TestingBot allows you to upload and test your app on a number of different iPads. Choose between various iPadOS versions and screen sizes, to make sure your mobile app works on every iPad your customers might own.
Why is testing your website on an iPad necessary?
iPads have a meaningful share of global tablet usage, and on tablets your customers use Mobile Safari. It is important to make sure your website looks and behaves correctly on iPad. Safari is the default browser on iPad, so you should test your website on Mobile Safari to make sure iPad users have a good experience. Test in portrait and landscape mode to make sure your website and apps work correctly on tablets.
Should I use an iPadOS simulator or a real iPad?
Use both, at different points. An iPadOS simulator is fast to launch and cheap to run on every commit, and it is good for catching layout breakage, broken navigation and obvious regressions. TestingBot hosts iPadOS simulators so you can do that without a Mac or an Xcode install. Before you ship, move to a real iPad. A simulator does not mimic exact device performance, several Apple frameworks (StoreKit, ClassKit, biometrics, push) are missing or stubbed, and because it cannot reproduce real hardware, battery, GPU and sensor measurements are not accurate. The practical split is simulators for speed during development, real devices for release confidence.
Can I test Safari on iPad?
TestingBot provides access to unaltered iPads. These come straight from the store and are not jailbroken. Run your tests on real Mobile Safari, on physical iPad devices in our datacenter.
Can I test on iPad without owning a Mac or iPad?
Yes. TestingBot streams real iPads to any Windows, macOS or Linux browser. You get the real device, real Mobile Safari and real iPadOS, without owning a Mac, an iPad or a copy of Xcode.
Can I run Appium against your iPad cloud?
Yes. TestingBot supports Appium 1, 2 and 3 against real iPads, with full XCUITest and WebDriverAgent. Point your existing Appium suite at our hub and pick an iPad model and iPadOS version through capabilities.
## Ready to test on real iPads?
Get started with iPad testing today. Run your tests on hosted iPadOS simulators or on real iPads in the TestingBot cloud, with nothing to install and no jailbroken devices.
[Start a free trial](https://testingbot.com/users/sign_up)
---
URL: https://testingbot.com/test-on-galaxy
Real Samsung Galaxy cloud
# Samsung Galaxy testing online, on emulators or real phones
Run manual, automated, visual and responsive tests on Samsung Galaxy in the TestingBot cloud. Start on an Android emulator when you want a fast first signal, then run the same tests on real Galaxy hardware before you ship. From Galaxy S8 through the new Galaxy S26, plus Galaxy A and Galaxy Tab, with real Chrome on Android, real apps and real touch input. Nothing to install, no rooted devices.
[Start free](https://testingbot.com/users/sign_up) [Request a demo](https://testingbot.com/demo)
- **Galaxy models:** S8 to S26
- **Android versions:** 9 to 16
- **Uptime SLA:** 99.9%
_A real Samsung Galaxy running Chrome for Android with animated tap gestures_
Trusted by some of the world's most innovative companies
30-second answer
## Why test on real Samsung Galaxy with TestingBot?
Android emulators and real Samsung devices
Run on a hosted Android emulator in seconds for quick checks, or connect to a physical, unmodified Samsung Galaxy phone or tablet in our datacenter when you need real Samsung hardware, real Chrome on Android and real touch and sensors. Both run from the same browser session, with no local Android SDK to install.
Every Galaxy worth testing on
Pick from Galaxy S26, S25, S24, S23, S21, S20, S10 and S8, plus Galaxy A55, A12 and Galaxy Tab A, across Android 9 through Android 16 and Samsung One UI. Match what your customers actually carry.
Manual, automated, app & web
Drive a Galaxy live from your browser, run Appium, Espresso or UiAutomator2 suites against the device cloud, capture screenshots for visual regression, or upload your APK and exercise the real Android app.
What you can do
## Everything Samsung Galaxy testing needs
Live debugging, automated Appium and Espresso runs, mobile-web and native-app testing, gestures, network throttling, geolocation. All on a fleet of real Samsung Galaxy devices in our datacenter.
- ### Live & manual testing
Take a real Galaxy in your browser. Tap, swipe, type, rotate and inspect, with full Chrome DevTools through the Android browser. Same as if you were holding the phone in your hand.
- ### Appium automation
Run Appium 1, 2 and 3 suites against real Galaxy phones and tablets in parallel. UiAutomator2 driver, full XPath and accessibility locators, no signing certificates to wrangle.
- ### Espresso & UiAutomator2
Upload your APK plus instrumentation APK and TestingBot runs Espresso or UiAutomator2 on a real Galaxy from any CI runner, no local Android SDK required.
- ### Native Android app testing
Upload your APK or AAB, install it in seconds, and exercise it on real Galaxy devices. Permissions, biometrics, keyboards, push notifications and deep links all behave like real hardware.
- ### Real Chrome on Android
Test your website on Chrome for Android, the browser most of your mobile users actually use, on a real Samsung Galaxy with real Android WebView behavior.
- ### Visual & screenshot testing
Capture pixel-perfect screenshots across every Galaxy model and Android version, then diff them automatically with our visual testing service.
- ### Tap, swipe, pinch & rotate
Drive every multi-touch gesture exactly as a customer would, including pinch zoom, long-press, scroll, three-finger swipes and device rotation.
- ### Network throttling
Test on WiFi, 5G, 4G, 3G, Edge and offline conditions. Catch slow-network bugs before Play Store reviewers do.
- ### Phones & tablets
Switch between portrait and landscape, swap between a flagship Galaxy S26 and a Galaxy Tab A in one click, and validate every breakpoint on real Chrome for Android.
Device coverage
## Test On Galaxy Devices Online
Galaxy S26, S25, S24, S23, S21, S20, S10, S8, plus Galaxy A55, A12 and Galaxy Tab A. Real Samsung hardware, real Chrome on Android, real touch input.
### Galaxy models
- Galaxy S26
- Galaxy S25
- Galaxy S24
- Galaxy S23
- Galaxy S21
- Galaxy S20
- Galaxy S10
- Galaxy S8
- Galaxy A55
- Galaxy A12
- Galaxy Tab A
[View all Galaxy devices](https://testingbot.com/support/app-automate/devices)
### Android versions
- Android 16
- Android 15
- Android 14
- Android 13
- Android 11
- Android 10
- Android 9
Latest Android releases land on the cloud within hours of Samsung shipping the One UI update. Older majors stay around for legacy regression coverage.
Android emulator vs real Galaxy
## When an Android emulator is enough, and when you need a real Galaxy
TestingBot gives you both. An Android emulator is the right tool for a fast first pass on layout, navigation and obvious breakage, and it is cheap to run on every commit. A real Galaxy is what you want before release, because an emulator cannot reproduce Samsung hardware, One UI behaviour, real radios or real touch. Here is where the two differ.
| Capability | Real Galaxy devices | Android emulator |
| --- | --- | --- |
| Hardware | Real Samsung Galaxy phone in our datacenter | x86 host CPU, no real Samsung radios or sensors |
| Chrome on Android | Real Chrome and Android WebView shipped on the device | Emulator Chrome, drifts from shipped Android Chrome |
| Touch & gestures | Real multi-touch, real rotation, real haptics | Mouse-faked taps, no haptics |
| Native apps | Real Android, real Play Services, real Samsung One UI | Generic AOSP, missing or stubbed Samsung-specific behaviour |
| Device coverage | Galaxy S8 through Galaxy S26, plus Galaxy A and Galaxy Tab A | Hosted Android system images, no local SDK or download |
| Video, logs and HAR | HD video, logcat, HAR and screenshots | Roll your own recording and capture |
Native Android apps
## Native Android app testing
Upload your APK or AAB and TestingBot installs it on a real Galaxy in seconds. Drive the app live or via Appium, Espresso or UiAutomator2, capture full HD video, logcat and crash reports, and exercise biometrics, push notifications, deep links and the Samsung keyboard exactly as your customers will.
Every Galaxy above is physical hardware racked in our European datacenter, not an emulator. The [real device testing](https://testingbot.com/real-device-testing) overview lists the full fleet, explains how each handset is wiped and re-imaged between sessions, and shows which plans include it.
[Read more](https://testingbot.com/features/automation/appium)
upload-and-test.js
// 1) Upload your APK to TestingBot storageconst TestingBot = require('testingbot-api'); const tb = new TestingBot({ api\_key: 'YOUR\_KEY', api\_secret: 'YOUR\_SECRET' }); tb.uploadFile('./MyApp.apk', (err, res) =\> { // res.app\_url → "tb://..."// 2) Run your Appium test on a real Galaxy driver.init({ platformName: 'Android', 'appium:deviceName': 'Samsung Galaxy S26', 'appium:platformVersion': '16', 'appium:realDevice': true, 'appium:app': res.app\_url }); });
Live exit nodes
10+ countries
United States
United Kingdom
Germany
France
Spain
Belgium
Australia
China
Japan
Geolocation testing
## Geolocation Testing on Galaxy
Run Galaxy tests from real exit nodes around the world to verify geo-targeting, geo-blocking and localisation from the United States, United Kingdom, Germany, France, Spain, Belgium, Australia, China, Japan and more.
Secure Tunnel
## Test internal & staging sites on real Galaxy devices
Use our [secure tunnel](https://testingbot.com/support/tunnel) to run Galaxy tests against firewalled staging sites, internal dashboards or apps still on localhost.
Encrypted end-to-end, no inbound firewall rules required, ready out of the box for popular CI providers.
[Read more](https://testingbot.com/support/tunnel)
Secure devices
## Real Galaxy devices. Wiped after every session.
Each session locks a single Samsung Galaxy for your test only. The moment your test ends, the device is wiped, all installed apps are deleted, caches are cleared, and the device is returned to the pool ready for the next customer.
GDPR-compliant, with SAML SSO and audit logs available for enterprise teams.
- One real Samsung Galaxy per session, never shared
- Wiped between every session
- Encrypted APK uploads with per-app retention controls
1 customer per session
Datacenter hosted
Wiped after each test
CSA STAR Level 1
GDPR compliant
SAML SSO enterprise
Test from any OS
## Test on Galaxy from any OS, no Android device required
On Windows, macOS or Linux and need to validate something on a Samsung Galaxy? You don't have to buy the hardware, set up Android Studio, or chase down a borrowed device. TestingBot streams a real Galaxy from our datacenter directly to any browser.
The streamed session is the real device, with Chrome for Android, the Samsung keyboard, biometrics and notifications. You drive it from any browser on your local machine, without installing Android Studio or owning a Samsung phone.
Not tied to a specific handset? Run a generic [Android emulator online](https://testingbot.com/android-emulator-online) instead.
[Start free](https://testingbot.com/users/sign_up)
No device to buy
No SDK to install
Real Galaxy in your browser
FAQ
## Frequently Asked Questions
Which Samsung Galaxy models and Android versions are supported?
TestingBot supports physical Samsung Galaxy phones from Galaxy S8 through the new Galaxy S26, plus Galaxy A55, A12 and Galaxy Tab A. Android versions range from Android 9 through Android 16, and we keep Samsung One UI majors current. New flagship Galaxy phones are added on or close to the day Samsung ships them.
Why should I test on Samsung Galaxy devices?
Samsung Galaxy devices have roughly 28% global mobile market share, and meaningfully higher in markets like Brazil, India and South Korea. Your Android coverage is incomplete without them. TestingBot lets you test on the most popular Galaxy phones and tablets, with real Samsung One UI behaviour and the same Chrome for Android your users are running.
Why use TestingBot to test on Samsung Galaxy?
TestingBot provides physical Samsung devices, ranging from Galaxy S8 through Galaxy S26, plus Galaxy A55, A12 and Galaxy Tab A, hosted in our EU datacenter. You instantly interact with a remote, physical device, including full Chrome DevTools, Android logcat and a downloadable video of the session.
Should I use a Samsung Galaxy emulator or a real device?
Use both, at different points. An Android emulator is fast, cheap to run on every commit and perfectly good for catching layout breakage, broken navigation and obvious regressions. TestingBot hosts Android emulators so you can do that without installing the Android SDK locally. Before you ship, move to a real Galaxy. An emulator runs a generic AOSP system image on host CPU, so it misses biometrics, Samsung One UI behaviour, push notifications, real-network conditions and the exact Chrome for Android build that Samsung ships. Google also recommends testing on real Android devices before submitting to the Play Store. The practical split is emulators for speed during development, real devices for release confidence.
Can I test Chrome on Android with Samsung Galaxy?
Yes. Every Samsung Galaxy device in the TestingBot cloud ships with Chrome for Android and the Samsung Internet browser, both unmodified and up-to-date. Devices are not rooted and come with Google Play Services and Chrome for Android, just like a retail device. Run Mobile-Web tests on real Chrome, not on a desktop browser pretending to be one.
Can I run Appium and Espresso against your Galaxy cloud?
Yes. TestingBot supports Appium 1, 2 and 3 with the UiAutomator2 driver, plus native Espresso runs. Point your existing Android suite at our hub and pick a Galaxy model and Android version through capabilities.
Can I test on Galaxy without owning the device?
Yes. TestingBot streams real Galaxy phones to any Windows, macOS or Linux browser. You get the real device, real Chrome for Android and real Samsung One UI, without buying the hardware or installing Android Studio.
## Ready to test on real Samsung Galaxy?
Get started with Galaxy testing today. Run your tests on hosted Android emulators or on real Samsung devices in the TestingBot cloud, with nothing to install and no rooted devices.
[Start a free trial](https://testingbot.com/users/sign_up)
---
URL: https://testingbot.com/android-emulator-online
Android emulator in your browser
# Android emulator online, no SDK required
Run a virtual Android phone on our servers and drive it from your browser. Upload an APK or open any URL in Chrome on Android, with no SDK to install and no local hardware. Real Android devices sit beside them for release validation.
[Start free](https://testingbot.com/users/sign_up) [See plans](https://testingbot.com/pricing)
100 free minutes to start. No credit card, no Android SDK, nothing to install.
- **Emulator configurations:** 57
- **Device models:** 35
- **Android versions:** 4.4–17
android-emulator.js
// Start a browser session on an Android emulator driver.init({ platformName: 'Android', 'appium:deviceName': 'Pixel 8', 'appium:platformVersion': '17', 'appium:realDevice': false, 'appium:browserName': 'Chrome' }); // Same test, real hardware: realDevice → true
One capability switches an emulator run onto physical hardware. Drive it with [Appium](https://testingbot.com/support/app-automate/appium), or open a session and tap through it by hand.
## What is an Android emulator online?
An online Android emulator is a virtual Android device that runs on a remote server and streams to your web browser, so you can install an APK or open a site in Chrome on Android without an Android SDK, a system image or local hardware. It behaves like a phone in software: the operating system, the browser and your app are real, the hardware underneath is not.
In a browser session
## What you can do
### Install an APK
Upload a build to TestingBot Storage and install it on an emulator session, by hand or from a test.
### Open any site in Chrome on Android
Load a public URL, or reach a staging environment behind your firewall through TestingBot Tunnel.
### Capture screenshots and video
Every session records video and logs, so a failure comes with evidence rather than a stack trace alone.
### Automate it
Drive the same session with Appium or Espresso instead of tapping through it by hand.
Framework guides: [Appium](https://testingbot.com/support/app-automate/appium) and [Espresso](https://testingbot.com/support/app-automate/espresso).
Scale and CI
## Run emulators in parallel from your pipeline
One emulator in a browser tab is useful for a quick look. A suite needs many at once. Emulator sessions draw on the same concurrency as every other session, so the same plan that runs your browser tests runs these, and [running them in parallel](https://testingbot.com/features/parallel-testing) is what turns a long suite into a short one.
Wire it into [Jenkins](https://testingbot.com/support/integrations/ci-cd/jenkins), [GitHub Actions](https://testingbot.com/support/integrations/ci-cd/github-actions), [CircleCI](https://testingbot.com/support/integrations/ci-cd/circleci), [Bitrise](https://testingbot.com/support/integrations/ci-cd/bitrise) or [GitLab](https://testingbot.com/support/integrations/ci-cd/gitlab) and emulator runs happen on every push. The [full CI/CD integration list](https://testingbot.com/support/integrations/ci-cd) covers the rest.
Where emulators stop
## Emulators for speed, real devices for release confidence
An emulator is a software model of a phone running on server hardware. It boots in seconds, costs less per minute and scales further, which is exactly what you want on every commit. What it cannot reproduce is physical hardware: the real GPU, the camera, biometric sensors, battery behaviour and genuine network conditions.
So we do not claim emulators replace hardware. TestingBot hosts 57 emulator configurations and 35 real Android devices across 22 models, and most teams use both: emulators in CI on every push, [real Android devices](https://testingbot.com/real-device-testing) on the release branch.
| Use it for | Emulator | Real device |
| --- | --- | --- |
| Functional flows and regression in CI | Yes | Yes |
| Responsive web testing in Chrome | Yes | Yes |
| High-volume parallel runs on every commit | Yes | Works, but slower |
| GPU rendering, camera and sensors | Use hardware | Yes |
| Biometrics, push notifications, battery | Use hardware | Yes |
| Final release validation | Use hardware | Yes |
Coverage
## Android versions available
Emulator images span Android 4.4 to 17, so you can reproduce a bug on an old release and verify the fix on the newest one in the same suite.
Android 4.4 Android 5 Android 6 Android 7.1 Android 8 Android 9 Android 10 Android 11 Android 12 Android 13 Android 14 Android 15 Android 16 Android 17
Looking for a specific handset rather than a generic Android image? [Samsung Galaxy testing](https://testingbot.com/test-on-galaxy) covers the Samsung Galaxy range on physical hardware.
## Getting started
1. 1
Create a TestingBot account. The free plan needs no credit card.
2. 2
Pick an Android version and device model, and start a session in your browser.
3. 3
Upload your APK, or type a URL to open it in Chrome on Android.
4. 4
When you are ready to automate, point Appium at the same capabilities.
## Frequently asked questions
- **Can I run an Android emulator online without installing anything?:** Yes. TestingBot runs the emulator in our datacenter and streams it to your browser, so there is no Android SDK, no system image download and no local hardware requirement. You open a session, interact with it in the browser, and it is wiped when you finish.
- **Can I run an APK on an online Android emulator?:** Yes. Upload your .apk to TestingBot Storage and install it on an emulator session, manually or from an automated test. You can also drive it with Appium or Espresso rather than tapping through it by hand.
- **What is the difference between an Android emulator and a real Android device?:** An emulator is a software model of a phone running on server hardware. It boots faster, costs less and scales further, which makes it a good fit for functional checks and CI. A real device is physical hardware with a real GPU, real sensors, a real camera and real battery behaviour. Use emulators for speed and real devices for release confidence.
- **Which Android versions can I emulate?:** TestingBot hosts emulator images spanning Android 4.4 to 17, across 35 device models, so you can reproduce a bug on an old release and verify the fix on the newest one in the same suite.
- **Can I test a website rather than an app?:** Yes. Every emulator session runs Chrome on Android, so you can load any public URL, or reach a staging site behind your firewall through TestingBot Tunnel. Automated web tests run with Selenium, Playwright or Appium against the same session.
- **Is an online Android emulator good enough for release testing?:** For most functional coverage, yes. For anything touching the GPU, camera, biometrics, push notifications, real network conditions or battery, test on physical hardware before you ship. Most teams run emulators on every commit and real devices on the release branch.
---
URL: https://testingbot.com/support
Developer platform
# Documentation
TestingBot is a cloud testing platform with 6100+ real browsers and physical iOS and Android devices. Use this guide to install your test framework, run your first test, then scale to CI.
## Explore by testing type
### [Automated Web Testing](https://testingbot.com/support/web-automate/selenium/nodejs)
Get started with automated testing of websites.
- [Selenium](https://testingbot.com/support/web-automate/selenium)
- [Cypress](https://testingbot.com/support/web-automate/cypress)
- [Puppeteer](https://testingbot.com/support/web-automate/puppeteer)
- [Playwright](https://testingbot.com/support/web-automate/playwright)
[Browse all 6100+ remote browsers →](https://testingbot.com/support/web-automate/browsers)
### [Automated App Testing](https://testingbot.com/support/app-automate)
Run automated tests against native mobile apps.
- [Appium](https://testingbot.com/support/app-automate/appium)
- [Espresso](https://testingbot.com/support/app-automate/espresso)
- [XCUITest](https://testingbot.com/support/app-automate/xcuitest)
- [Maestro](https://testingbot.com/support/app-automate/maestro)
[Browse all physical iOS and Android devices →](https://testingbot.com/support/app-automate/devices)
### [AI Testing](https://testingbot.com/support/ai)
Use AI to test. Connect remote browsers with LLM frameworks.
- [Codeless tests with AI](https://testingbot.com/support/ai/codeless)
- [Generate tests with LLM](https://testingbot.com/support/ai/chat)
- [Test Recorder](https://testingbot.com/support/web-automate/codeless-automation/add-test)
- [Functions](https://testingbot.com/support/functions)
[AI Integrations →](https://testingbot.com/support/ai/integrations)
### [Manual Web & App Testing](https://testingbot.com/support/web-live)
Control a remote browser or device interactively.
- [Manual Web Testing](https://testingbot.com/support/web-live)
- [Manual App Testing](https://testingbot.com/support/app-live)
[About live testing →](https://testingbot.com/features/manual-browser-testing)
### [Visual Website Testing](https://testingbot.com/support/visual-testing)
Capture and compare screenshots across browsers.
- [Automated Visual Testing](https://testingbot.com/support/visual-testing/automated)
- [Codeless Visual Testing](https://testingbot.com/support/visual-testing/codeless)
[About visual testing →](https://testingbot.com/features/screenshot-testing)
### [Accessibility Web Testing](https://testingbot.com/support/accessibility)
Test your URLs against WCAG accessibility rules.
- [Selenium + Accessibility](https://testingbot.com/support/accessibility/web/selenium)
- [Scheduled Testing](https://testingbot.com/support/accessibility/web/scheduled)
[About accessibility testing →](https://testingbot.com/features/accessibility-testing)
## What's new
### [Latest from the blog](https://testingbot.com/resources/articles)
Tutorials, deep dives and release notes for TestingBot.
- [Parallel Testing: What Actually Scales](https://testingbot.com/resources/articles/parallel-testing) Sep 1, 2026
- [Automated Regression Testing in CI](https://testingbot.com/resources/articles/automated-regression-testing) Aug 31, 2026
- [Mobile App Test Automation at Scale](https://testingbot.com/resources/articles/mobile-app-test-automation) Aug 31, 2026
- [AI in Software Testing: What Works](https://testingbot.com/resources/articles/ai-in-software-testing) Aug 31, 2026
[See all product updates](https://testingbot.com/product-updates)
### Platform info
[All systems operational](https://status.testingbot.com)
Live service status View →
[Trust Center](https://trust.testingbot.com) Compliance
- GDPR
- CSA STAR
- EU hosted
Resources
- [Changelog Latest releases](https://testingbot.com/product-updates)
- [REST API Endpoint reference](https://testingbot.com/support/api)
- [Configuration Capabilities reference](https://testingbot.com/support/other/configuration)
- [FAQ Common questions](https://testingbot.com/support/faq)
- [llms.txt Sitemap for AI agents](https://testingbot.com/llms.txt)
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/faq
# Frequently asked questions
Answers to the most common questions about running Selenium, Appium and cloud testing on TestingBot. For framework-specific guides see the [documentation index](https://testingbot.com/support); for billing and account questions contact [info@testingbot.com](mailto:info@testingbot.com).
## Questions
How does TestingBot's cloud test grid work?
TestingBot runs an open-source Selenium and Appium grid that has been tuned for speed and reliability. Every test runs on a freshly-provisioned virtual machine (or real iOS/Android device). When the test ends, the VM is destroyed so there is no state carried between runs. You connect to the grid by changing your WebDriver endpoint to https://hub.testingbot.com/wd/hub and passing your API key and secret as capabilities — no other changes to your test code are required.
How are costs calculated and how much will I be charged?
TestingBot accounts include a pool of free minutes you can use to evaluate the service. After that, billing is based on test minutes and the number of parallel tests in your plan. We charge from the moment the test session starts until it ends — queue time and failures caused on our side are not billed. See [pricing](https://testingbot.com/pricing) for current plans, and [contact us](https://testingbot.com/contact/new) for volume or enterprise pricing.
What happens if my test does not seem to run on your system?
If a test does not start or behaves unexpectedly, check the dashboard at https://testingbot.com/members for an error code on the session. Common causes are an invalid browser/OS combination, missing or expired API credentials, or a tunnel that did not connect. Errors that originate on TestingBot's infrastructure are logged on our side and are not billed. Contact info@testingbot.com or open Slack with the session id and we will investigate.
Why don't I see screenshots after my test finished?
Screenshots and video are uploaded to our storage after the test completes and can take up to a few minutes to appear in the dashboard depending on the length of the session. If screenshots are still missing after 5 minutes, confirm that you have not disabled them via the screenrecorder or screenshots capabilities, and check that your session ended with driver.quit() (or the equivalent) rather than being killed mid-flight.
My test fails with "could not find browser/operating system" — what's wrong?
A CapabilityNotPresentOnTheGridException (or a similar "browser not found" error) means the browser + version + platform combination you requested is not currently offered. Check the live list of [supported browser combinations](https://testingbot.com/support/web-automate/browsers) and adjust your capabilities. Use latest, latest-1, latest-2 etc. for the major-version aliases if you do not need to pin to an exact version.
Can I run my tests in parallel on TestingBot?
Yes. Every plan includes a number of concurrent test sessions — you can run that many tests at the same time. Each session is fully isolated on its own VM or device, so parallel tests never interfere with each other. Most CI test runners (pytest-xdist, Jest, Mocha-parallel, Maven Surefire) and Selenium Grid clients launch parallel sessions natively.
How long are test artifacts (logs, screenshots, video) kept?
Test session metadata, screenshots, video recordings and logs are retained for 30 days. You can also delete a session manually from the dashboard or via the REST API at any time, and immediate deletion after each test can be configured. Test data is stored in our EU data centres; see the [Trust Center](https://trust.testingbot.com) for compliance details.
Does TestingBot integrate with CI/CD systems like GitHub Actions, Jenkins or CircleCI?
Yes. TestingBot integrates with all major CI/CD systems — GitHub Actions, Jenkins, CircleCI, GitLab CI, Bitbucket Pipelines, Travis CI, TeamCity and Azure DevOps among others. You point your existing Selenium or Appium test runner at the TestingBot endpoint and add your key and secret as environment variables. See [integration guides](https://testingbot.com/support/integrations) for CI templates and recipes.
How secure is TestingBot? Do you support SSO and tunnels?
TestingBot is GDPR-compliant and registered in the CSA STAR registry (Level 1 self-assessment); our CAIQ, Technical & Organizational Measures and Bitsight rating are published in the [Trust Center](https://trust.testingbot.com). The infrastructure is hosted in the EU. SAML 2.0 single sign-on (Okta, Azure AD, Google Workspace) is available on Enterprise plans. To test pre-production sites, use the [TestingBot Tunnel](https://testingbot.com/support/tunnel), an encrypted connection between your network and the test VM.
What's the difference between Live (manual) and Automated testing on TestingBot?
Automated testing runs Selenium, Appium, Cypress, Playwright, Puppeteer or other framework scripts against the grid programmatically. Live testing gives you an interactive remote browser or device window — you point and click yourself, exactly like using a local browser. Most teams use both: automation for regression suites and live testing for exploratory testing and reproducing bugs reported by users. Pricing for each is separate (see [pricing](https://testingbot.com/pricing)).
## Need more help?
- Browse the full [documentation](https://testingbot.com/support) for framework-specific guides.
- Join the [TestingBot Slack workspace](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw).
- Email [info@testingbot.com](mailto:info@testingbot.com) with your session id for fastest help.
- Check the [status page](https://status.testingbot.com) for known incidents.
---
URL: https://testingbot.com/support/analytics
# Test Analytics
Test Analytics turns every automated test you run on TestingBot into answers: is the suite healthy, which tests are failing or flaky, on which browsers and devices, why they fail and which build broke them.
It works for every automated framework you run on TestingBot: Selenium WebDriver, Playwright, Puppeteer, Cypress, Appium, Espresso, XCUITest and Maestro. There is nothing to install: open [Analytics](https://testingbot.com/members/tests) in the sidebar of the member dashboard.
_The Test Analytics dashboard: the range in one sentence, the numbers against the range before, and every day in the range._
## The four views
Test Analytics is a set of linked pages. You start wide and drill down by clicking: a day in the chart, a test in a table, a run in a list.
| View | Answers | Where to find it |
| --- | --- | --- |
| [**Test Analytics dashboard**](https://testingbot.com/support/analytics/guide) | How healthy is the whole suite? Which tests need attention, which errors fail the most runs, which environment fails more than it should? | [Analytics](https://testingbot.com/members/tests) in the sidebar |
| [**A day of runs**](https://testingbot.com/support/analytics/day) | What changed on this day compared with the day before, what broke, what got fixed? | Click a day in the _Runs per day_ chart |
| [**All runs of a test**](https://testingbot.com/support/analytics/insights) | Is this one test stable, flaky, failing or broken by a build? On which environment, since when and with which error? | Click a test in _Needs attention_, or _All runs_ on a test's detail page |
| [**Build analytics**](https://testingbot.com/support/analytics/builds) | Can this build merge? What did it break compared with the build before, and how well did it use parallel sessions? | [Automated Builds](https://testingbot.com/members/builds), or a build in _Recent builds_ |
How each label is decided (flaky, failing, broken, newly failing, slower, flips, p50 and p95) is explained on [Metrics and definitions](https://testingbot.com/support/analytics/metrics).
## Get the most out of Test Analytics
Test Analytics reads what your tests send to TestingBot. Four capabilities make the difference between a list of sessions and real analytics:
| Send | Why it matters |
| --- | --- |
| [A test name](https://testingbot.com/support/web-automate/selenium/test-options#name) (`name`) | Runs are grouped into a test by their **exact name**. Without a stable name, every run is a test of its own and nothing can be called flaky or failing. Use the same name every time the same test runs, and a different name for each test. |
| [The result](https://testingbot.com/support/api/tests#updatetest) (passed or failed) | TestingBot cannot know whether your assertions passed. Runs without a result count as _unknown_: they are not in the pass rate. Most frameworks can report the result automatically; see for example [marking Playwright tests](https://testingbot.com/support/web-automate/playwright/mark-test-status). |
| [A build](https://testingbot.com/support/web-automate/selenium/test-options#build) (`build`) | Groups the runs of one CI run. Builds unlock _Recent builds_, new failures per build, the [build page](https://testingbot.com/support/analytics/builds), and the _Broken_ verdict ("build X failed every run"). |
| [Groups](https://testingbot.com/support/web-automate/selenium/test-options#grouping) (`groups`) | Tags your runs, for example per project, team or suite. Every tag becomes a value of the _Tag_ filter. |
Some suites put the build number in each test name, such as `nightly 20261008.1 - Login`. The build page takes the build's name off the front of the test name when it compares builds, but Test Analytics groups runs by their full name. Keep the build number in the `build` capability instead, so all runs of a test share one name.
## What is counted
- **Automated runs only.** Manual Live sessions are not part of Test Analytics.
- **Your whole account.** Runs started by your [team members](https://testingbot.com/support/team/sub-accounts) are included. Use the _Started by_ filter to look at one person's runs.
- **Finished runs, by start time.** A run belongs to the day it started on.
- **UTC.** Days and ranges are in UTC, whatever your time zone. The range is shown on the right of the filter bar, for example `Sep 15 – Oct 14, 2026 · UTC`.
- **Durations of passed runs only.** A failed run stops early or hangs until a timeout, so the p50 and p95 durations are measured on passed runs.
- **Against the range before.** Every number with a change next to it is compared with the range of the same length just before the one you are looking at.
## A typical triage
_Hover over the chart to compare days, then click a day to see what broke on it._
1. Open [Analytics](https://testingbot.com/members/tests). The line at the top says whether the range is _Healthy_ or _Needs attention_, and names the test to fix first.
2. In [Needs attention](https://testingbot.com/support/analytics/guide#attention), pick the _Failing_ or _Newly failing_ tab. _Why it fails_ shows each test's latest error.
3. Click a test to open [All runs](https://testingbot.com/support/analytics/insights). The verdict says whether the test is broken by a build, failing on one browser, or flaky. _Why it failed_ groups its failures by error.
4. Click a run in the chart or the table. It opens in the test panel next to the list, with its video, steps, logs and the failed step.
5. Fixed it? The next day's page lists the test under [Fixed today](https://testingbot.com/support/analytics/day#fixed).
## Sharing, exports and API
- **Every view is a link.** The range, filters and tabs are part of the URL. Use _Copy link_ to share exactly what you are looking at with a team member.
- **CSV export.** Each view has an _Export CSV_ button. The columns are listed on each page of this documentation.
- **Insights API.** Build your own dashboards on the same data with the [Insights API](https://testingbot.com/support/api/insights): daily trends, a summary, top failure messages and per-test flakiness.
- **Slack.** The [Slack integration](https://testingbot.com/support/integrations/slack) answers Test Analytics questions from inside Slack.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/analytics/insights
# All runs of a test
The All runs page shows every run of one test, so you can decide in seconds whether it is **stable** , **flaky** , **failing** or **broken by a build** , on which browsers and devices, since when, and with which error. It replaces the scatter plot of the former Test Insights page.
_All runs of a flaky test: the verdict names the browser and the error, the chart shows every run._
A test is every run with **exactly the same name**. Name your tests with the [`name` capability](https://testingbot.com/support/web-automate/selenium/test-options#name), or [through the API](https://testingbot.com/support/api/tests#updatetest), and use the same name every time the test runs. Tests without a name (`unnamed test…`) have no history to show.
There are several ways to open it:
- On a test's detail page, the _History_ strip in the header shows its last 20 runs as bars, oldest left. Click **All runs** next to it.
- On the [Test Analytics dashboard](https://testingbot.com/support/analytics/guide#attention), click a test in _Needs attention_, or _Runs →_ under an error in _Top errors_.
- On a [day page](https://testingbot.com/support/analytics/day), click _All runs_ next to a run or a fixed test.
- On a [build page](https://testingbot.com/support/analytics/builds), click _Runs →_ next to a test under _Since \_, or _All runs →_ next to a session.
## Range and filters
| Control | Options |
| --- | --- |
| **Range** | `24h`, `7d`, `30d` (the default), `90d` or `Custom`: a _From_ and _To_ date, up to 366 days. |
| **Environment** | Only the runs on one browser or device. The list holds the environments this test ran on in the range. |
| **Build** | Only the runs of one build. The list holds the builds this test ran in in the range. |
| **Result** | _Passed_ or _Failed_ runs only. |
A range holds up to the newest 1,000 runs of the test. When it has more, the page says so under the verdict: pick a shorter range to see all of them.
## The verdict
The card at the top reads the runs for you and sums them up in one sentence, with a label. You do not have to study the chart to triage a test.
_A flaky test: where it flakes, how often, and the error it flakes with._
_A test broken by a build: the build and the browsers it fails on._
| Label | When | Example |
| --- | --- | --- |
| **Broken** | The latest build failed every one of its runs (at least 2), and an earlier build passed. | _Broken since build 1.42.0 on every browser: all 6 of its runs failed._ |
| **Failing** | The last 5 runs or more all failed, or an environment failed every one of its runs (at least 3). | _Failing on Safari 18: all 7 of its runs failed, since Oct 2._ |
| **Flaky** | On an environment with 5 or more passed or failed runs, between 10% and 90% failed, and the result changed at least twice. | _Flaky on Firefox 143: 4 of 19 runs failed._ |
| **Stable** | None of the above. Occasional failures are called one-offs. | _Stable: 58 of 59 runs passed; the failure was a one-off._ |
| **Unknown** | Every run ended without a result. | _No result: all 12 runs ended without passing or failing._ |
| **No runs** | The test did not finish a run in this range. | |
Once the errors are read (a moment after the page shows), the sentence also names the error when all failures share one, for example _…, all on `element not interactable` at step 11._ When an earlier build broke the test and a later build fixed it, the verdict says that too: _Separately, build 1.40.2 broke it on Chrome 141: all 3 of its runs failed, until 1.40.3 fixed it._
_Show the failures_ opens the _Failed_ tab of [Runs](https://testingbot.com#runs). All rules and thresholds are on [Metrics and definitions](https://testingbot.com/support/analytics/metrics).
## The numbers
Each number is compared with the range of the same length just before this one.
| Number | What it is |
| --- | --- |
| **Pass rate** | Passed out of passed and failed runs, with the change in points. |
| **Runs** | The test's runs in the range, and how many failed or have no result. |
| **Flips** | How often the result changed between consecutive runs on the same environment, for the environment where it changed the most (named next to the number). A stable or a constantly failing test has few flips; a flaky test has many. |
| **Duration p50 / p95** | The median and 95th percentile duration of the passed runs, with the change of the median in seconds. |
| **Last failure** | How long ago the test last failed, and where (environment and build). Click it to open that run. |
## Duration per run
A chart with a mark for every run, placed at the time it started (across) and its duration (up):
_The verdict names the error once the failures are read. Hover over a run to see where and why it failed._
- **Green dot** : passed. **Red diamond** : failed. **Grey dot** : no result.
- **Blue line** : the median duration of the passed runs, moving along the range, so you see when the test got slower.
- **Blue band** : from p50 to p95 of the passed runs. A passed run above the band is slower than 95% of the others. With too few passed runs for a moving median, a dashed line shows the median of all passed runs instead.
- **Dashed vertical lines** : where each build first ran, labelled with the build name.
- **Shaded area** : a build that failed every run, up to the build that fixed it (or to the end of the range if it is still broken).
- A run far slower than all others, such as one that hung until a timeout, is drawn on the top edge so it does not flatten the rest of the chart.
The legend only lists what the chart draws. Hover over a mark to see its result and environment, when it ran, its build and duration, and for a failed run its error. **Click a mark** to open the run in the test panel next to the page.
_A broken test: the window of the build that broke it is shaded, up to now._
## By environment and per day
**By environment** has a row per browser or device the test ran on:
_The flaky test only fails on Safari 26, always with the same error._
- **Pass rate** on that environment.
- **Last 20** : its last 20 runs there, oldest on the left: green passed, red failed, grey unknown. A flaky test shows red bars scattered between green ones; a failing one shows red at the end.
- **p50** : the median duration of its passed runs there.
- **Flips** : how often its result changed there. More than 6 is highlighted.
**Per day** shows the test's passed (green), failed (red) and unknown (grey) runs for each day of the range. Hover over a day for its numbers.
## Why it failed
The test's failures, grouped by their error, most runs first. The error of a run is the one its detail page shows: the last step in error and its message. Parts of a message that change from run to run (the Chrome version Chrome adds, element and session ids, durations) are ignored, so the same failure is one group.
Each group shows:
- the error message and how many runs failed with it,
- the browsers or devices it happened on,
- `new in ` when it first happened in a build after the start of the range: the build that introduced it,
- the step it failed at, for example `step 11`,
- the first and last day it happened, and a bar per day with its failures,
- _Latest run →_, which opens its latest run in the test panel.
The errors are read from the steps of each run, so this card fills a moment after the page shows. It groups the newest 40 failures; the _Failed_ tab of [Runs](https://testingbot.com#runs) lists all of them. A failure without a step in error (the test marked itself as failed, for example after a visual or accessibility check) is counted under the list, but not grouped.
## Runs
Every run of the test in the range, newest first:
_Runs, newest first. The run you came from is highlighted._
| Column | Shows |
| --- | --- |
| **Started** | When the run started (UTC) and how long ago. The run you came from is highlighted and marked _the run you came from_. |
| **Result** | Passed, Failed or Unknown. |
| **Environment** | The browser or device, and its operating system. |
| **Build** | The build it ran in, if any. |
| **Duration** | The duration, with a bar: red for a failed run, amber for a passed run slower than p95, grey otherwise. The tick on the bar marks the median. |
| **Failed at** | For a failed run, the step and command it failed at, for example `step 7 · click`. |
The tabs narrow the list and show how many runs each has:
| Tab | Lists |
| --- | --- |
| **All** | Every run. |
| **Failed** | The failed runs. |
| **Slower than p95** | Passed runs that took longer than the p95 duration. |
| **Changed result** | Runs whose result differs from the run before them on the same environment: the moments it started failing or passing again. |
The first 12 runs show; _Load 25 more_ adds the next ones. Click a run to open it in the test panel next to the list, with its video, steps and logs. _Back to the run_ at the top returns to the run you came from.
## Copy link and CSV export
The range, filters and the open tab are part of the URL: **Copy link** copies it to share this exact view.
**Export CSV** downloads the runs in the current range, filters and tab, newest first, named after the test:
| Column | Contains |
| --- | --- |
| `started_at` | When the run started, in UTC (ISO 8601) |
| `result` | `passed`, `failed` or `unknown` |
| `browser` | The browser name and version |
| `os` | The operating system or device |
| `build` | The build name, if any |
| `duration_seconds` | From start to finish, in seconds |
| `session_id` | The session id, as used by the [REST API](https://testingbot.com/support/api/tests) |
| `url` | The link to the run in the member dashboard |
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/analytics/builds
# Build analytics
A build is a group of test runs, usually all the tests of one CI run. Add a [`build` capability](https://testingbot.com/support/web-automate/selenium/test-options#build) to your tests and TestingBot groups every run with the same build name together.
The build page answers the question you have when CI goes red: **can this build merge, and if not, what did it break?** It compares the build with the build before it, test by test and browser by browser, so a test that was already flaky is not blamed on your change.
## The builds list
[Automated Builds](https://testingbot.com/members/builds) in the member dashboard lists your builds, newest first, with their tests, status and start time. Filter the list by build name, status or start date, and click a build to open its page. The newest builds of a range are also on the [Test Analytics dashboard](https://testingbot.com/support/analytics/guide#builds), under _Recent builds_.
While a build is still running (a session is running, or one started or finished in the last 10 minutes), its page shows the live sessions table, which adds sessions as they start. Once the build is quiet, the page shows the full analysis described below.
## The build before
Most of the page compares this build with an earlier one. TestingBot picks that build in this order:
1. The build you choose in **Compare with** , in the top right.
2. With a [GitHub CI link](https://testingbot.com/support/integrations/ci-cd/github-pr-checks) (registered with [`POST /v1/builds/{id}/ci`](https://testingbot.com/support/api/tests#post-builds-id-ci)): the newest earlier build on the same repository and branch.
3. Otherwise: the newest of the 20 earlier builds that ran at least half of this build's tests. A build with the same name pattern is preferred, so `1.42.0` is compared with `1.41.0` and `nightly 20261008.1` with `nightly 20261007.1`, not with a run of another suite.
Runs are compared by test name and environment. When a suite puts the build name at the front of each test name, such as `nightly 20261008.1-Login`, that part is taken off so the same test matches across builds.
The head of the page also shows the build's result, its GitHub pull request, branch and commit when it has a CI link, when it ran, and buttons to the builds either side. _Status badge_ gives you a [badge](https://testingbot.com/support/other/status-badges) for the build's status, and the `⋯` menu edits or deletes the build.
## The verdict
The first card says in one sentence what the build did against the build before:
_The build page head: the CI link, the build before, the verdict and the numbers against main-2513._
| Label | When |
| --- | --- |
| **Broke N tests** | A test failed that passed in the build before. This is the one to look at before merging. |
| **Flaky only** | Every failure is a test that is [flaky](https://testingbot.com/support/analytics/metrics#flaky) in the builds before too: not caused by this change. |
| **Failed** | Tests failed, but none that passed before: they were already failing, or the build before did not run them. |
| **Passed** | Nothing failed. |
| **Running** | Sessions have not reported back yet. |
| **No result** | No session passed or failed. |
| **No sessions** | The build has no sessions. |
Once the errors are read, the sentence names the error when the broken tests share one. _Show the failures_ opens the failed sessions.
## The numbers
| Number | What it is |
| --- | --- |
| **Result** | Passed sessions out of all sessions, with a bar, and what the build before had. |
| **Failed** | Failed sessions, with the change against the build before, split into new, flaky and still failing. |
| **Tests** | How many tests on how many environments, how many were fixed and how many were added since the build before. |
| **Wall time** | From the first session's start to the last session's finish, with the change against the build before (grey when it changed by less than 10%), and when the first failure happened. |
| **Parallel slots** | How many sessions were busy on average (all session time divided by the wall time), out of the most sessions that ran at once, for example `7.1 of 8 busy · 89% used`. It also names the environment that kept the build open at the end, when one session ran alone. |
## Tests × browsers
A grid with a row per test and a column per environment (the 10 with the most sessions). Each cell shows the sessions of that test on that environment: up to 3, so retries are visible. Next to each row, a strip shows the same test in the build before, one bar per column.
Failed tests come first. The first 15 rows show (or every failed test, if more failed); _Show all_ lists the rest, up to 300 tests. Click a cell to open that session in the test panel.
_Tests × browsers and what changed since the build before: three tests broke on every browser; the other failures are not this build's doing._
## Since the build before
The _Since \_ card groups the tests by what changed. Next to the title it says how the build before was picked: the build before on the same branch, the build before with these tests, or the build you picked.
| Group | Tests that |
| --- | --- |
| **Started failing** | Failed here and passed in the build before. |
| **Flaky** | Failed here, but fail now and then in the builds before too: probably not this change. |
| **Fixed** | Failed in the build before and pass on every environment now. |
| **Got slower** | Took 30% or more over their median duration in the builds before, on the same environment. |
| **Still failing** | Failed here and in the build before. |
| **No result in \** | Failed here, and in the build before neither passed nor failed on that environment. |
Each test shows the environments it applies to, and links to its [All runs](https://testingbot.com/support/analytics/insights) page. Each group lists 5 tests; the Sessions table has all of them. Flaky and slower are read from the tests' runs in the 30 builds before this one, for the 40 tests with the most failures and the slowest.
When no earlier build ran at least half of these tests, the card says so; pick a build in _Compare with_ to compare anyway.
## How the build ran
_Hover over a session to see when it ran and why it failed._
A timeline of the build with a row per parallel slot and a bar per session, from its start to its end: light green passed, red failed, blue still running, grey no result. A session takes the first slot that is free when it starts.
- A dashed red line marks the **first failure** , with the time since the build started.
- The session that **finished last** is named at the top.
- Under the chart: the wall time, how much of the time the slots were idle, and how long the last session ran alone, if it did.
A long tail where one session runs alone means the build waits for one slow test: split it, or start it first. Hover over a bar to see the session, its environment, duration and error. Click it to open the session. A build with more than 1,500 sessions draws the first 1,500 and says so.
## Why it failed and slowest tests
**Why it failed** groups the build's failed sessions by their error, most sessions first, the same way as on a test's [All runs](https://testingbot.com/support/analytics/insights#why) page. It loads a moment after the page, because the errors are read from the sessions' steps. An error that only this build has is tagged `new in `; one that also failed in the build before is tagged `failed before too`. _Open a session_ opens one of its sessions.
_The build's errors, and its slowest tests against their median in the builds before._
**Slowest tests** lists the 6 tests with the longest sessions in the build. Each shows its longest session with a bar, a tick at its median duration on the same environment in the builds before, and how much slower it was. 30% or more is highlighted.
A build with visual tests also shows a **Visual comparisons** summary of its screenshot comparisons.
## Sessions
Every session of the build, failed first, with its test, result, environment, duration, the step it failed at, and what it did against the build before. Each row links to the test's _All runs_ page.
_Sessions, failed first, each with what it did against the build before._
The tabs are _All_, _Failed_, _Started failing_, _Flaky_ and _Fixed_; without a build before to compare with, only _All_ and _Failed_. _Filter by test name_ narrows the list. The first 25 sessions show; _Load more_ adds the next 25.
## Build efficiency
The shared build page (the page a [status badge](https://testingbot.com/support/other/status-badges) links to) shows a **build efficiency** percentage. It tells you how well your test runner used parallel sessions to shorten the build.
It is the duration of the longest test in the build, as a percentage of the build's wall time:
- 10 tests of 30 seconds each, run one after the other, take **300 seconds** : an efficiency of 10%.
- The same 10 tests run in parallel take **30 seconds** , as long as the longest test: an efficiency of 100%.
| Efficiency | Degree of parallelization | Description |
| --- | --- | --- |
| 0%-20% | Sequential | Your tests ran one after the other. Consider running them in parallel to shorten your build. |
| 20%-90% | Semi-parallel | Some tests ran in parallel. Add more parallel sessions to run all tests of the build at the same time. |
| 90%-100% | Fully parallel | The build is as fast as its longest test. |
On the build page in the member dashboard, [Parallel slots](https://testingbot.com#kpis) and [How the build ran](https://testingbot.com#timeline) show the same thing in more detail.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/analytics/guide
# Test Analytics dashboard
The Test Analytics dashboard summarises every automated run in your account over a time range: the pass rate and how it changed, the tests that need attention and why they fail, the environments that fail the most, and your recent builds.
Open it with **Analytics** in the sidebar of the [member dashboard](https://testingbot.com/members/tests). The line under the title tells you what the page covers, for example `Every automated run in your account · 42 tests · 9 environments · 31 builds`.
The page reads from top to bottom, from the overall answer to the detail behind it:
1. [The digest](https://testingbot.com#digest): one line that says whether the range is healthy.
2. [The numbers](https://testingbot.com#kpis): runs, pass rate, failed runs, flaky tests, tests and duration, each against the range before.
3. [Runs per day](https://testingbot.com#runs-per-day): a chart of every day in the range.
4. [Needs attention](https://testingbot.com#attention): the tests that are failing, newly failing, flaky or slower.
5. [By environment](https://testingbot.com#environments) and [Top errors](https://testingbot.com#errors): where and why runs fail.
6. [Recent builds](https://testingbot.com#builds): the newest builds in the range.
## Range and filters
The filter bar at the top decides which runs every part of the page is about. A change applies at once: there is no separate search button.
_Custom shows a From and To date in place. The Build filter searches all your builds; tick several and press Apply._
| Control | Options |
| --- | --- |
| **Range** | `24h`, `7d`, `30d` (the default), `90d` or `Custom`. The preset ranges end now. _Custom_ shows a _From_ and _To_ date: pick whole days and press _Apply_. A custom range can span up to 366 days. |
| **Environment** | Opens the same browser and device picker as the test dashboard. Select one or more browsers, devices and operating systems. With one selected, the button shows its name (for example `Chrome 141 · Windows 11`); with more, how many are selected. |
| **Build** | A searchable list of your builds, newest first. Tick one or more and press _Apply_; _Clear_ removes the selection. The list shows the first 100 matches: type part of a build name to find older builds. |
| **Tag** | Only runs with this tag, set with the [`groups` capability](https://testingbot.com/support/web-automate/selenium/test-options#grouping). |
| **Started by** | Only runs started by one member of your team. Shown when your account has [team members](https://testingbot.com/support/team/sub-accounts). |
The range in dates is shown on the right of the filter bar, for example `Sep 15 – Oct 14, 2026 · UTC`. All days are UTC days.
When nothing ran in the selection, the page says _No runs in this selection_. Your earlier runs are still there: widen the range, clear a filter, or use _Reset to the last 30 days_.
You can also filter by clicking: a name in the [By environment](https://testingbot.com#environments) table filters the page to that environment.
## The digest
The first card sums up the range in a sentence or two, with a label on the left:
_The digest and the numbers below it, each against the 30 days before._
| Label | When |
| --- | --- |
| **Needs attention** | The pass rate is under 95%, the pass rate fell by 2 points or more against the range before, or at least one test is [failing](https://testingbot.com/support/analytics/metrics#failing). |
| **Healthy** | None of the above. |
| **No runs** | No automated run finished in the range. |
The sentence can say up to four things:
- The pass rate over the range and how it moved, for example _Pass rate 98.4% over 17,262 runs, down 0.8 pts on the range before._
- How many tests are failing and flaky, and where most of the flaky ones flake, for example _4 tests are failing and 3 are flaky, 2 of them on Safari 26._
- An environment that fails far more than it runs: _Safari 26 is 24% of the runs but 67% of the failures._ This is said when an environment's share of the failed runs is 10 points or more above its share of all runs.
- The test to fix first, with a link to its [All runs](https://testingbot.com/support/analytics/insights) page: _Fix first: Account › order history, which failed 97 of its 411 runs._
## The numbers
Below the digest, six numbers describe the range. Most have a change next to them, measured against the **range of the same length just before this one** : the 30 days before the last 30 days, for example. A green change is an improvement, a red one is worse, and a grey one is neither good nor bad.
| Number | What it is | Change shown |
| --- | --- | --- |
| **Runs** | Finished automated runs in the range. The sparkline shows the runs per day. | In percent |
| **Pass rate** | Passed runs out of the runs that passed or failed. Runs without a result (unknown) are left out. The sparkline shows the pass rate per day. | In percentage points (`pts`) |
| **Failed runs** | Runs that failed. The sparkline shows the failed runs per day. | In runs |
| **Flaky tests** | Tests whose result keeps changing, out of all tests. See [how flaky is decided](https://testingbot.com/support/analytics/metrics#flaky). A `+` after the number means it is a lower bound: more tests failed than could be read. | None |
| **Tests** | Different test names that ran in the range. | In tests |
| **Duration p50 / p95** | The median and 95th percentile duration of the passed runs: half of the passed runs took less than p50, 95% took less than p95. | p50, in seconds |
## Runs per day
A bar for every day in the range, with the runs of that day stacked by their result:
_A day with an outage stands out as a red block and a dip in the pass rate. The hover card gives the day's numbers._
- **Light green** : passed runs.
- **Red** : failed runs.
- **Grey** : runs without a result (unknown).
- **Blue line** : the day's pass rate, read on the percentage scale on the right. That scale starts at the lowest pass rate in the range (rounded down to 10%, and at most 90%), so a drop from 99% to 94% is easy to see.
- **Shaded columns** : weekends, so a quiet Saturday is not mistaken for a problem.
Hover over a day to see its date, the passed, failed and unknown runs, and the pass rate. **Click a day** to open [that day's page](https://testingbot.com/support/analytics/day): what changed against the day before, hour by hour, and every run.
## Needs attention
The tests worth a look, worst first. Every test is the set of runs with the same name, and it gets one state:
_Needs attention: what each test's state is, where it fails and why._
| State | Meaning |
| --- | --- |
| **Failing** | Its last 5 runs failed, an environment failed every one of its runs (3 or more), or its latest build failed every run after an earlier build passed. |
| **Newly failing** | Failing now, and it ran without failing in the range before. When a build broke it, the label names the build: _New in 1.42.0_. |
| **Flaky** | On at least one environment, it failed between 10% and 90% of 5 or more runs, and its result flipped at least twice. |
| **Slower** | The average duration of its passed runs grew by 30% or more against the range before, with at least 5 passed runs in each range. |
These are the same rules as the verdict on a test's [All runs](https://testingbot.com/support/analytics/insights#verdict) page, so a test called flaky here is flaky there too. All thresholds are on [Metrics and definitions](https://testingbot.com/support/analytics/metrics).
### Tabs
_All_ lists every test in the table. The _Failing_, _Newly failing_, _Flaky_ and _Slower_ tabs narrow it to one state and show how many tests each has; a tab with no tests is not shown. The first 6 tests show; _Show all_ lists the rest.
_The errors fill in a moment after the page shows. The tabs narrow the list without reloading the page._
### Columns
| Column | Shows |
| --- | --- |
| **Test** | The test name, how many runs it had, and when it last ran. |
| **State** | Failing, Newly failing (or _New in \_), Flaky or Slower. |
| **Pass rate** | The test's own pass rate in the range. |
| **Last 20** | A bar per run for its last 20 runs, oldest on the left: green passed, red failed, grey unknown. |
| **Flips** | How often the result changed between consecutive runs on the same environment. More than 6 is highlighted. |
| **Where** | The browsers or devices it fails on (up to 3, then `+N`). For a slower test: where it ran. |
| **Why it fails** | The error of its latest failed run, and the step it failed at, for example `step 7 · click`. This is filled in a moment after the page shows, because it is read from the run's steps. For a slower test: its average duration before and now, for example `avg 41s → 58s`. |
Click a test name, or _All runs →_, to open the [All runs](https://testingbot.com/support/analytics/insights) page of that test.
To keep the page fast, Needs attention reads the runs of the 20 tests that failed most recently (and of tests that got slower), up to 1,000 runs each. When more tests than that failed, the flaky count shows a `+` and the digest says _at least_. Narrow the range or filter by build or environment to see the others.
## By environment
A row per browser or device, most runs first. This shows whether failures are spread evenly or come from one place.
_By environment and Top errors side by side: Safari 26 fails far more than its share, and one outage failed three checkout tests._
| Column | Shows |
| --- | --- |
| **Environment** | The browser and version with its operating system logo, or for a mobile run the device, for example `Safari · iPhone 16 (real device)`. Hover for the full name. |
| **Runs** | Its runs, and its share of all runs in percent. |
| **Pass rate** | Its pass rate in the range. |
| **Share of failures** | Its part of all failed runs. Shown in bold red when it is 10 points or more above its share of runs: this environment fails more than its fair share. |
| **p50** | The median duration of its passed runs. |
| **Trend** | Its pass rate per day. |
Click an environment to filter the whole page to it.
A native app test (Espresso, XCUITest, Maestro or Appium with an app) and a browser test on the same device are listed as two environments, because they test different things.
## Top errors
The errors that fail the most runs, across all tests that are failing or flaky. Where _Needs attention_ lists tests, Top errors groups their failures by cause: one broken selector or one unreachable server often fails many tests at once.
A run's error is the one its test detail page shows: the last step in error and its message. Errors are grouped by message, with the parts that change between runs taken out, so these count as one error:
- the `(Session info: chrome=141.0…)` that Chrome adds,
- element and session ids (hexadecimal ids of 16 characters or more),
- durations such as `30000ms` or `1.5s`.
Each error, most runs first, shows:
- the error message, and how many runs failed with it,
- how many tests it failed, and on which browsers or devices (up to 4, then `+N more`),
- the first and last day it happened in the range,
- a bar per day with its failures, so you see whether it started on one day or comes and goes,
- _Runs →_: the failed runs of the test with the most of these failures, on its All runs page.
Top errors loads a moment after the rest of the page, because each error is read from the run's steps. It reads the newest 10 failures of up to 30 tests (those with the most failures), and says so under the list when that is a sample.
A failure without a step in error is not grouped. This happens when the test marked itself as failed (a visual or accessibility check, for example) or failed further back than the steps that are read.
## Recent builds
The 5 newest [builds](https://testingbot.com/support/web-automate/selenium/test-options#build) that ran in the range, newest first:
_Recent builds: main-2514 broke tests that passed in the build before it._
| Column | Shows |
| --- | --- |
| **Build** | The build name and when its first run started. |
| **Runs** | Its runs in the range. |
| **Pass rate** | Its pass rate. |
| **Failed** | Its failed runs. |
| **New failures** | Tests that failed in this build and passed in the build before it, for example `+3`. A dash means there is no earlier build in the range to compare with. A flaky test that happened to fail is counted here too; the [build page](https://testingbot.com/support/analytics/builds#since) tells flaky tests apart. |
| **Wall time** | From the start of its first run to the end of its last. |
Click a build to open its [build page](https://testingbot.com/support/analytics/builds). When no run in the range has a build, the card explains how to set the `build` capability.
## Copy link and CSV export
The range, the filters and the tabs are all part of the page's URL. **Copy link** copies it, so a team member opens exactly the same view. You can bookmark it too.
**Export CSV** downloads the runs per day of the current range and filters, as `test-analytics--.csv`:
| Column | Contains |
| --- | --- |
| `date` | The UTC day, as `YYYY-MM-DD` |
| `runs` | Finished runs that day |
| `passed` | Passed runs |
| `failed` | Failed runs |
| `unknown` | Runs without a result |
| `pass_rate` | Passed out of passed and failed, in percent (empty when none passed or failed) |
For every single run of a day, use the export on [the day's page](https://testingbot.com/support/analytics/day#export). For the same numbers in your own tools, use the [Insights API](https://testingbot.com/support/api/insights).
The numbers are kept for 5 minutes per range and filter, so a reload is fast. A run that finishes in the meantime shows up within 5 minutes.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/analytics/day
# A day of runs
The day page answers the question you ask after a red night in CI: _what changed today?_ It compares one day with the day before, shows the day hour by hour, lists every test that failed and every test that was fixed, and every run of the day.
Open it by clicking a day in the [Runs per day](https://testingbot.com/support/analytics/guide#runs-per-day) chart on the Test Analytics dashboard. The buttons in the top right, labelled with the dates either side (for example `‹ Oct 13` and `Oct 15 ›`), go to the day before and the day after, so you can walk through a week day by day.
The title is the date, for example `Tuesday, Oct 14, 2026`, and the line below it sums the day up: its runs, tests and environments, and the builds that ran. Days are UTC days, from 00:00 to 24:00 UTC.
_A day against the day before: the build that broke a test is named, with the time it first ran._
## Against the day before
The first card compares the day with the day before, with a label on the left:
| Label | When |
| --- | --- |
| **Worse than \** | The pass rate fell by 2 points or more, or a test started failing. |
| **Better than \** | The pass rate rose by 2 points or more, and no test started failing. |
| **Like \** | Neither. |
| **No runs** | No automated run finished that day. |
When tests started failing, the sentence names the build in which they first failed, so you know which change to look at. _Show what broke_ jumps to the [What broke](https://testingbot.com#broke) list.
A whole day is compared with the whole day before, also while today is still going: early in the day, today has fewer runs than yesterday.
## The numbers
| Number | What it is |
| --- | --- |
| **Runs** | Finished runs that day, with the change against the day before. |
| **Pass rate** | Passed out of passed and failed runs, with the change in points, and how many passed and failed. |
| **Newly failing** | Tests that failed today and passed the day before (at least one pass and no failure the day before). |
| **Fixed** | Tests that failed the day before and passed every run today. |
| **Duration p50 / p95** | The median and 95th percentile duration of the day's passed runs, with the change of the median in seconds. |
## The day, hour by hour
A timeline of the day from 00:00 to 24:00 UTC with a row per environment and a mark per run, placed at the time the run started:
_Each cluster is a build. Failed runs are red diamonds; hover one to see the test, the environment and the time._
- **Green dot** : the run passed.
- **Red diamond** : the run failed.
- **Grey dot** : the run has no result.
- **Dashed red line** : the moment a build first ran that day. The 5 newest builds are named at the top, with their start time.
The 5 environments with the most runs get a row each; the others share an _Other_ row. This makes patterns easy to spot: failures that start right after a build line point at that build, and failures that fill one row point at one browser or device.
Hover over a mark to see the test, its environment, the time it started, its duration and its build, and for a failed run where it failed. **Click a mark** to open the run in the test panel next to the page, with its video, steps and logs.
On a busy day the chart draws the newest 2,000 runs and says so above the chart: _The newest 2,000 of 5,312 runs_. All runs are still in the numbers and in the [Runs](https://testingbot.com#runs) list.
## What broke
Every test that failed that day. Tests that **started failing** come first (they passed the day before), then the others, most failures first. A day has too few runs per test to call a test flaky, so this list shows every failure rather than only failing and flaky tests.
_What broke: the test that started failing first, then every other test that failed that day._
The table has the same columns as [Needs attention](https://testingbot.com/support/analytics/guide#attention-columns) on the dashboard: the test, its state, pass rate, last 20 runs, flips, where it fails and why it fails. The state is either _Started failing_ or _Failed 3 of 8_: how many of its runs that day failed. The _Started failing_ and _Failed_ tabs narrow the list. The first 6 tests show; _Show all_ lists the rest.
What broke lists up to 200 tests, those with the most failures. On a day with more failed tests it says so; the _Failed_ tab of [Runs](https://testingbot.com#runs) still has every failed run.
## Compared per environment
The _Compared with \_ table has a row per environment, most runs first: its runs, pass rate, the change of its pass rate against the day before, and its failed runs. A red change shows which browser or device got worse. The first 8 environments show; _Show all_ lists the rest.
_Each environment against the day before, and the tests that were fixed._
## Fixed today
Tests that failed the day before and passed every run today, most runs first, for example _failed 4 on Oct 13 · passed 6 today_. _All runs →_ opens the test's [All runs](https://testingbot.com/support/analytics/insights) page, to confirm the fix holds.
## Runs
Every run of the day, newest first, with its start time, test name, result, environment, build, duration and, for a failed run, the step it failed at.
_Every run of the day. The bar shows each run's duration; the tick marks the day's median._
| Tab | Lists |
| --- | --- |
| **All** | Every run of the day. |
| **Failed** | The failed runs. |
| **Newly failing** | The failed runs of tests that passed the day before. |
| **Slower than p95** | Passed runs that took longer than the day's p95 duration. |
Each tab shows how many runs it has. _Filter by test name_ narrows the list to tests whose name contains what you type. The first 25 runs show; _Load 25 more_ adds the next ones. Runs that finish while you read the page do not shift the list.
Click a run to open it in the test panel next to the list, or _All runs_ to open that test's [All runs](https://testingbot.com/support/analytics/insights) page.
A day page whose URL carries Test Analytics filters (an environment, build, tag or team member) says _Filtered as on Test Analytics_ under the title. _Show every run_ removes the filters.
## CSV export
**Export CSV** downloads every run matching the day's Test Analytics filters, newest first, regardless of the open Runs tab or test-name search, as `test-runs-.csv`. It has a row per run:
| Column | Contains |
| --- | --- |
| `started_at` | When the run started, in UTC (ISO 8601) |
| `test` | The test name |
| `result` | `passed`, `failed` or `unknown` |
| `browser` | The browser name and version |
| `os` | The operating system or device |
| `build` | The build name, if any |
| `duration_seconds` | From start to finish, in seconds |
| `session_id` | The session id, as used by the [REST API](https://testingbot.com/support/api/tests) |
| `url` | The link to the run in the member dashboard |
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/analytics/metrics
# Metrics and definitions
Test Analytics labels tests as flaky, failing or broken with fixed rules, not a model, so the same runs always get the same label, and you can check every label against the runs. This page lists those rules and the limits of each view.
## Runs, tests and results
| Term | Meaning |
| --- | --- |
| **Run** (or session) | One finished automated test session: Selenium, Playwright, Puppeteer, Cypress, Appium, Espresso, XCUITest or Maestro. Manual Live sessions are not counted. |
| **Test** | All runs with exactly the same [name](https://testingbot.com/support/web-automate/selenium/test-options#name). `Login` and `Login (retry)` are two tests. |
| **Passed / Failed** | The result your test reported, through a capability, a framework integration or the [REST API](https://testingbot.com/support/api/tests#updatetest). |
| **Unknown** | A run that ended without a result. It counts as a run, but not as a pass or a failure. |
| **Decided run** | A run that passed or failed. |
## Pass rate
**Passed runs ÷ (passed + failed runs)**, in percent, rounded to one decimal. Unknown runs are left out, so a suite that does not report results does not look like it fails. With no decided runs, the pass rate is shown as `—`.
A change in pass rate is shown in **percentage points** (`pts`): from 95% to 92% is `-3 pts`.
## Ranges and comparisons
- All dates and days are **UTC**. A run belongs to the day it started on.
- The preset ranges (`24h`, `7d`, `30d`, `90d`) end now. A custom range is whole UTC days, up to 366 days.
- Every change (`+12%`, `-3 pts`, `+4s`) compares the range with the **range of the same length just before it**. On the [day page](https://testingbot.com/support/analytics/day), that is the whole day before.
- Green means better, red means worse, grey means neither: more runs is not better or worse in itself.
## Durations, p50 and p95
- A run's **duration** is its wall time: from the moment it started to the moment it finished.
- Duration numbers use **passed runs only**. A failed run stops early or hangs until a timeout; either would distort what a normal duration is.
- **p50** is the median: half of the passed runs took less. With an even number of runs, it is the average of the two middle ones.
- **p95** : 95% of the passed runs took less. A passed run above p95 is listed under _Slower than p95_.
- A **build's wall time** is from the start of its first session to the end of its last.
## Flips
A flip is a change of result between two consecutive decided runs of a test **on the same environment** : passed then failed, or failed then passed.
`pass, pass, fail, pass, fail` on Chrome is 3 flips. A test that fails every time has no flips, and neither does one that always passes. Many flips is the signature of a flaky test. More than 6 flips is highlighted in the tables.
## Test states
A test's state is decided from its runs in the range (the newest 1,000 at most), in this order: the first rule that matches wins.
| State | Rule |
| --- | --- |
| **Broken** | The test's **latest build** failed every one of its runs of this test (at least 2, and none unknown), and an earlier build passed. One failed run is not yet a broken build. A build that broke the test and a later build that fixed it is mentioned in the verdict, but the test is then judged on its other runs. |
| **Failing** | The **last 5 runs** all failed (an unknown result ends the streak), **or** an environment failed every one of its runs (at least 3): a browser or device the test does not work on. |
| **Flaky** | On at least one environment with **5 or more decided runs** , **between 10% and 90%** of them failed, and the result **flipped at least twice**. Runs of a broken build are left out. Fewer runs say too little: pass, fail, pass is one failure, not flakiness. |
| **Stable** | None of the above. Failures that do not meet these rules are one-offs. |
| **Unknown** | Every run ended without a result. |
On the [dashboard](https://testingbot.com/support/analytics/guide#attention), a broken test is listed as _Failing_, and two more states are added:
| State | Rule |
| --- | --- |
| **Newly failing** | Failing (or broken) now, and in the range before it had runs but no failure. Labelled _New in \_ when a build broke it. |
| **Slower** | The average duration of its passed runs grew by **30% or more** against the range before, with at least 5 passed runs in each range. |
Needs attention orders tests worst first: failing, newly failing, flaky (most failures first), slower.
## Changes between days and builds
| Term | Where | Meaning |
| --- | --- | --- |
| **Started failing** / **Newly failing** | [Day page](https://testingbot.com/support/analytics/day) | Failed on this day, and the day before it passed at least once and did not fail. Looser than the dashboard's _Newly failing_: one failure is enough, the test does not have to be failing by the rules above. |
| **Fixed** | Day page | Failed the day before, and passed every run on this day. |
| **New failures** | [Recent builds](https://testingbot.com/support/analytics/guide#builds) | Tests that failed in a build and passed in the build before it (in the order the builds first ran). |
| **Started failing** | [Build page](https://testingbot.com/support/analytics/builds#since) | Failed in this build, and passed on the same environment in the build before. |
| **Flaky** | Build page | Failed in this build, and the test is flaky over its runs in the 30 builds before. |
| **Fixed** | Build page | Failed in the build before, and passes on every environment in this build. |
| **Got slower** | Build page | A session 30% or more over the test's median duration on the same environment in the builds before. |
| **Changed result** | [All runs](https://testingbot.com/support/analytics/insights#runs) | A run whose result differs from the run before it on the same environment. |
## Environments
- An **environment** is a browser and version on an operating system, or a mobile device. Mobile runs are named by their device, with _real device_, _simulator_ or _emulator_ on hover.
- A native app run (Espresso, XCUITest, Maestro, or Appium with an app) and a browser run on the same device are **two environments**.
- **Share of failures** is an environment's part of all failed runs. When it is **10 points or more** above its share of all runs, the environment fails more than its fair share: it is highlighted and named in the digest.
## Error grouping
A failed run's error is the one its test detail page shows: the **last step in error** and its message. Errors are grouped by message, after taking out what changes between runs:
- the `(Session info: chrome=…)` Chrome adds to its messages,
- hexadecimal ids of 16 characters or more, such as element and session ids,
- durations such as `30000ms` or `1.5s`.
A run that the test itself marked as failed (after a visual or accessibility check, for example) has no step in error, so it is counted but not grouped. Errors are read from the run's steps, which is why they appear a moment after the page shows.
## Limits
To keep every page fast on accounts with millions of runs, some parts read a sample. The page always says so when it does.
| Part | Limit |
| --- | --- |
| Custom range | 366 days |
| All runs of a test | The newest 1,000 runs in the range |
| Needs attention | Up to 20 candidate tests: those with the most recent failures, followed by slower-only tests; up to 1,000 runs per test. Counts become lower bounds (`+`, _at least_) when more failed tests exist |
| Top errors | The newest 10 failures of the 30 tests with the most failures; 6 errors shown |
| Why it failed (All runs) | The newest 40 failures |
| Duration percentiles (dashboard) | The newest 50,000 passed runs |
| Recent builds | The 5 newest builds in the range |
| What broke (day page) | 200 tests, those with the most failures |
| The day, hour by hour | The newest 2,000 runs; 5 environment rows plus _Other_ |
| Build page | 5,000 sessions, failures first; the timeline draws the first 1,500 |
| Build history (flaky, slower) | The 30 builds before, for the 40 tests with the most failures and the slowest |
| Freshness | Numbers are kept for 5 minutes per range and filter |
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/billing
# Billing
Everything about paying for TestingBot: how billing works, the payment methods we accept, and how to manage or cancel your subscription.
- [Payment Options](https://testingbot.com/support/billing/payment-options): credit card, PayPal, wire transfer, and AWS Marketplace.
- [Charges](https://testingbot.com/support/billing/charges): how the monthly and annual billing cycle works.
- [Upgrading](https://testingbot.com/support/billing/upgrading): change your plan or top up with Pay As You Go.
- [Invoices](https://testingbot.com/support/billing/invoices): download invoices and set your company details and VAT number.
- [Cancel](https://testingbot.com/support/billing/cancel): stop your subscription at any time.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/billing/payment-options
# Payment Options
You can see the list of our prices on our [pricing page](https://testingbot.com/pricing).
We accept the following payment options:
- **Credit card:** handled securely by Stripe
- **PayPal:** available through Stripe
- **Wire transfer and purchase orders:** for annual and enterprise plans
- **EU local methods:** SEPA Direct Debit and iDEAL, through Stripe
- **AWS Marketplace:** buy through your existing AWS account
TestingBot is PCI compliant. Please view our [SAQ A](https://testingbot.com/pci.pdf).
## Credit Card
Billing occurs as a subscription and is handled by Stripe.com. PayPal and EU local methods such as SEPA Direct Debit and iDEAL are also processed through Stripe.
All payments happen on a HTTPS secured page.
**Your payment data is never stored or logged on our website.**
## AWS Marketplace
TestingBot is available on the AWS Marketplace. This is a convenient option for teams that already buy through AWS and want testing on the same bill.
- Pay through your existing AWS account, with no new vendor to set up
- One consolidated AWS invoice for your finance team
- Faster procurement, since AWS is already an approved vendor
For custom or multi-year terms, [contact us](https://testingbot.com/contact/new) to arrange an offer through the Marketplace.
[View TestingBot on AWS Marketplace](https://aws.amazon.com/marketplace/pp/prodview-yv6ds4mf77fao)
## Wire Transfer and Purchase Orders
For annual and enterprise plans we also accept wire transfer and purchase orders. [Contact us](https://testingbot.com/contact/new) for bank details and to arrange payment.
## Invoices and VAT
Invoices for your subscription are available in your [account billing](https://testingbot.com/members/invoices) section. See the [Invoices](https://testingbot.com/support/billing/invoices) page for how to add your company details and VAT number.
Customers in the EU are charged VAT at checkout. Businesses elsewhere in the EU with a valid VAT ID are reverse charged and pay no VAT. Belgian businesses are charged Belgian VAT.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/billing/charges
# Charges
Billing occurs as a subscription, monthly or annual. Each cycle your account is recharged with testing minutes. How many minutes are added depends on the subscription you choose.
You can see the various subscriptions on our [pricing page](https://testingbot.com/pricing).
## What if I use more than the provided test minutes?
You will no longer be able to run your tests until your minutes refresh at the start of your next billing period.
**We will never automatically re-charge your subscription for extra minutes, no hidden costs.**
## Need more minutes? Pay As You Go
If you need extra minutes before your next billing date, you can buy a one-time [Pay As You Go](https://testingbot.com/pricing) credit pack. The minutes are added to your account right away, with no change to your subscription. To raise your allowance every cycle, [upgrade your plan](https://testingbot.com/support/billing/upgrading) instead.
## When will I be billed?
Monthly subscriptions are billed on the same day each month. If you started on the 11th, you are charged on the 11th of the following month. Annual subscriptions are billed once a year on your renewal date.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/billing/upgrading
# Upgrading subscription
At any time you can [upgrade your current subscription](https://testingbot.com/pricing).
You will not lose your remaining testing minutes for that month.
Minutes will be transferred over immediately when you upgrade your plan.
The number of simultaneous VMs allowed will be automatically increased after your upgrade.
## Downgrading
To move to a smaller plan, select it on the [pricing page](https://testingbot.com/pricing). If you need help changing or reducing your plan, [contact us](https://testingbot.com/contact/new). To stop your subscription entirely, see [Cancel Subscription](https://testingbot.com/support/billing/cancel).
## Monthly or annual billing
Plans are available on a monthly or annual basis. Annual billing is discounted, and you can choose it on the [pricing page](https://testingbot.com/pricing).
## Pay As You Go
If you only need extra minutes occasionally, you can buy a one-time [Pay As You Go](https://testingbot.com/pricing) credit pack instead of upgrading. The minutes are added to your account right away.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/billing/invoices
# Invoices
We provide invoices for all customers with an active subscription.
You can find monthly/yearly invoices (depending on the subscription) in the member area under [Billing](https://testingbot.com/members/invoices).
Each invoice can be downloaded as a PDF from the Invoices table.
If you are a sub-account in a team account, you will not be able to see the invoices. Only parent accounts or standalone accounts can see invoices.
## Changing Billing Details
To change your billing details (Company name, address, VAT number, ...) please go to [Account Settings](https://testingbot.com/members/user/edit) and update the fields on the Profile tab.
When you update your billing details, future invoices will reflect the new information. Re-download the invoice from the Billing page to get the updated copy.
## VAT number
If your company is located in the EU, add your VAT number on the Profile tab of your [Account Settings](https://testingbot.com/members/user/edit), or enter it at checkout.
Customers in the EU are charged VAT at checkout. Businesses elsewhere in the EU with a valid VAT number are reverse charged and pay no VAT, which is reflected on the invoice. Belgian businesses are charged Belgian VAT.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/billing/cancel
# Cancel Subscription
You can stop your subscription at any time.
Go to [Billing](https://testingbot.com/members/invoices) in the member area and click the **Cancel Subscription** button under "Manage your Subscription". Your account remains active until the end of your current billing period. You keep access to all features until then.
Customers who subscribed through [AWS Marketplace](https://aws.amazon.com/marketplace/pp/prodview-yv6ds4mf77fao) should cancel through their AWS Marketplace subscription page.
Sorry to see you go. Please [let us know](https://testingbot.com/contact/new) how we can improve our service.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/parallel-calculator
# Parallel Testing Calculator
Parallel testing allows you to run multiple tests simultaneously across various browsers, devices, and OS combinations. This approach enables more tests to be run at once, significantly decreasing overall testing time. Faster build times mean faster releases and less time spent waiting for builds to complete.
Use this calculator to understand how parallel testing can help you achieve your test coverage and build execution time goals. Simply input your current testing setup and desired build time to calculate the optimal number of parallel tests needed.
| Team | Current State | Desired Future State | Recommendation |
| --- | --- | --- | --- |
| # of Browsers and/or Devices (Current) | # of Parallel Tests | Build Time (hours) | # of Browsers and/or Devices (Desired) | Build Speed |
| --- | --- | --- | --- | --- |
| Team 1 | | | | | Select... Supersonic (15m) Fast (30m) Basic (2h) Custom | - |
| + Add Team Maximum of 10 teams reached. | Total Recommendation: - |
Calculate Save & Share
Use this URL to view & share these results:
Copy
## Understanding the Calculator
- **# of Browsers and/or Devices:** The number of unique desktop browser and/or mobile device combinations to test.
- **# of Parallel Tests:** The number of parallel tests you are currently running.
- **Build Time:** Time taken for your entire build to complete.
---
URL: https://testingbot.com/support/team
# TestingBot Team Management
TestingBot provides team accounts, allowing you to share test results with your colleagues.
- [Account Settings](https://testingbot.com/support/team/account)
- [Team & Sub-Accounts](https://testingbot.com/support/team/sub-accounts)
- [Roles & Permissions](https://testingbot.com/support/team/rbac)
- [Roles & Permissions Reference](https://testingbot.com/support/team/rbac/roles-and-permissions)
- [Custom Roles](https://testingbot.com/support/team/rbac/custom-roles)
- [Member Roles](https://testingbot.com/support/team/rbac/member-roles)
- [Product Access](https://testingbot.com/support/team/rbac/product-access)
- [Service Accounts](https://testingbot.com/support/team/service-accounts)
- [Audit Logs](https://testingbot.com/support/team/audit-logs)
- [2FA](https://testingbot.com/support/team/2fa)
- [Enforce 2FA](https://testingbot.com/support/team/enforce-2fa)
- [SSO](https://testingbot.com/support/team/sso)
- [Microsoft Entra](https://testingbot.com/support/team/sso/microsoft-entra)
- [Okta](https://testingbot.com/support/team/sso/okta)
---
URL: https://testingbot.com/support/team/sub-accounts
# Managing Sub-Accounts
Sub-accounts are accounts that can be linked to your account. They're a great way to share your subscription and test results with your team members. You can invite several people at once, assign each invitation a role, and manage invitations that are still pending.
- Everyone in the same group can see each other's tests.
- Each sub-account has its own login credentials to log into our TestingBot member dashboard, and they have their own key/secret credentials to run tests on our Selenium grid.
A sub-account can create/run/delete tests, see test results and interact with the member dashboard. The things **sub-accounts can't do** :
- Access any subscription related items: payment information, invoices, accounting details, ...
- Add/remove any other sub-accounts (unless they have the Admin role)
## Adding Sub-Accounts
Go to [My Account \> Add a team member](https://testingbot.com/members/sub/new) to invite people to your team.
Enter one or more email addresses in the invite field (press Enter, comma, or space after each one to confirm it), choose a [role](https://testingbot.com#permissions), then click **Send invitations**. You can invite up to 10 people at a time this way. The same role is applied to everyone in the submission.
What happens next depends on the email address:
- **The person already has a TestingBot account:** they receive an invitation to join your team. It stays under [Pending Invitations](https://testingbot.com#pending) until they accept.
- **The email address is new to TestingBot:** they receive an invitation with a link to create a free account using that address. Once they confirm their email address, they are taken to the invitation to accept it. It stays under [Pending Invitations](https://testingbot.com#pending) until then.
### Inviting a large team
To add more people at once, use **Or upload a CSV** and select a `.csv` file with one email address per line. The role you selected is applied to every address in the file.
## Roles and Permissions
Every team member now has **two roles** that work together:
- **IAM role** (Owner, Admin, or User): administrative authority. It decides who can manage the account and grant access, for example managing team members, billing, security, and integrations. The IAM role is derived from the member's organization role and is fixed.
- **RBAC role** (Admin, User, Viewer, or a custom role): product capability. It decides what the member can do in the products, such as running, viewing, or deleting tests. Custom roles are RBAC only and can never grant administrative authority.
For the full model, permission matrix, and how product access resolves, see [Role-based access control](https://testingbot.com/support/team/rbac) and the [Roles and permissions reference](https://testingbot.com/support/team/rbac/roles-and-permissions), which are the source of truth. You can change a member's roles later by editing them on the [Team Members](https://testingbot.com/members/sub) page.
## Pending Invitations
Every invitation appears under **Pending Invitations** on the [Team Members](https://testingbot.com/members/sub) page until they accept. For each invitation you can see the email address, who sent it, and the role it grants.
Admins can manage a pending invitation:
- **Resend:** send the invitation email again.
- **Revoke:** cancel the invitation.
## Modifying Sub-Accounts
To edit or remove a sub-account, go to [My Account \> Team Members](https://testingbot.com/members/sub) and use the actions menu next to a member. From the edit screen you can change their role, read-only access, and concurrency limit.
**Modify access** now sets both an IAM role and an RBAC role for the member. See [Assign member roles](https://testingbot.com/support/team/rbac/member-roles) for details on choosing each one.
### Disconnecting a sub-account
You can also **disconnect a sub-account** from the actions menu.
This disconnects the sub-account from your group.
The account will not be removed from our system, but it will no longer be associated with your account or other sub-accounts.
## Read-Only (Viewer) Accounts
A **Viewer** is a read-only sub-account: it can view tests and results but cannot modify or remove any results from the team account. You can assign the Viewer role when inviting someone, or switch an existing member to read-only by editing them on the Team Members page.
## Exporting Members
Admins can export the team to a CSV file. On the [Team Members](https://testingbot.com/members/sub) page, click **Export Members** and choose what to include:
- **All:** current members and pending invitations.
- **Members:** current team members only.
- **Invitations:** pending invitations only.
The file downloads as `team_members_.csv`.
## Concurrency Limit
The Concurrency Limit indicates how many tests a user can run simultaneously. The more tests you run concurrently, the faster you can process all your tests.
By default, a sub-account will have the same amount of concurrency as the parent account (which depends on the pricing plan that the parent account is subscribed to).
It is possible to change the concurrency limit per user. This way, you can for example divide the amount of concurrency you have across various team members.
For example: you have an Enterprise plan with 30 parallel tests: You can assign 10 parallel tests to team member **A** , 15 to team member **B** and 5 to team member **C**. A user can not run more tests concurrently than the maximum amount assigned. If the user tries to run more tests, the tests will be queued in our system, waiting for a slot to free up.
Two limits apply to every test a team member starts: the member's own concurrency limit, counted over that member's tests alone, and the plan's concurrency, counted over the whole team including the parent account. A member's running tests therefore never hold another member below its own limit, and the team as a whole never runs more tests than the plan allows. If you do not change this setting for any of the sub-accounts, every sub-account keeps the parent account's maximum, and the plan's concurrency is what the team shares. Whenever either limit is reached, the extra tests are queued until a slot frees up.
## Tests Overview for Sub-Accounts
To see a list of tests created by a sub-account, click "Tests run by this user".
The list contains tests run by this sub-account (with their own key/secret).
## Team Activity Log
Team admins are able to see an overview of the actions performed by their team members in the Team Activity Log overview page. On the Team Management page, please click the **Activity Log** tab to open the overview.
In this activity log we will show the following events for your team members:
- A user from your team logged in
- A user from your team logged out
- A user was invited to your team
- An invitation was accepted or declined
- A user was added to your team
- A user was removed from your team
- A team member's user level was upgraded or downgraded
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/team/2fa
# Why Two-Factor Authentication (2FA)
Two Factor Authentication adds an extra layer of security to your TestingBot account. After entering your email and password, you'll also need to supply a one-time verification code.
TestingBot supports two methods for generating that code: a time-based authenticator app (TOTP) or an SMS message to your mobile phone. We recommend an authenticator app — it works without cell coverage and is generally more secure than SMS.
With 2FA enabled, even if your password is compromised, an attacker would still need access to your authenticator app or phone to sign in.
Team admins can require 2FA for everyone on the team. See [Enforce Two-Factor Authentication](https://testingbot.com/support/team/enforce-2fa).
## Enable 2FA
Log into your TestingBot account and go to [Security](https://testingbot.com/members/user/security) in your account settings.
Click the [Enable 2FA](https://testingbot.com/members/2fa/enable) button to begin the setup.
On the setup page, you can choose either method:
- **Authenticator app (recommended):** scan the QR code with any TOTP-compatible app such as Google Authenticator, Authy, 1Password or Microsoft Authenticator, then enter the 6-digit code it generates to verify.
- **SMS:** enter your mobile phone number with country code (for example, a US number starts with `+1`). We'll send a verification code via SMS that you enter to verify.
Once 2FA is enabled, every subsequent login will prompt you for a verification code after your email and password.
## Authenticator App
Any TOTP-compatible authenticator app will work with TestingBot 2FA, including Google Authenticator, Authy, 1Password, and Microsoft Authenticator.
Scan the QR code shown on the [Enable 2FA](https://testingbot.com/members/2fa/enable) page to register TestingBot with your app. The app will then generate a new 6-digit code every 30 seconds, which you enter when signing in.
## Disabling 2FA
Log into your TestingBot account and go to [Security](https://testingbot.com/members/user/security) in your account settings.
Click the **Disable 2FA** button to turn off two-factor authentication for your account.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/team/enforce-2fa
# Enforce Two-Factor Authentication
As a team admin you can require every member of your team to use [two-factor authentication (2FA)](https://testingbot.com/support/team/2fa) when signing in to TestingBot. When enforcement is on, members who have not set up 2FA are guided through setup before they can use the dashboard, so your whole team is protected by a second factor, not just the people who opted in.
Enforcement is configured from the **Authentication** tab on your team management page. Any team admin (the account owner or a member with the Admin role) can turn it on or off. Two-factor authentication is a feature of our paid plans.
## Turn on enforcement
Open the [Authentication](https://testingbot.com/members/authentication) tab under your team management page (Members dashboard, then **Team → Authentication** ), and follow these steps:
- Click **Configure** on the "Enforce two-factor authentication" card.
- Review the effects in the confirmation dialog.
- Click **Confirm**.
Every member who does not yet have 2FA is automatically emailed with instructions to set it up. The card then shows that enforcement is active, along with how many members have completed setup.
## What happens when enforcement is on
- **Everyone must comply.** All human team members, including admins and the account owner, are required to use 2FA. [Service accounts](https://testingbot.com/support/team/service-accounts) are API-only and cannot sign in to the dashboard, so they are excluded.
- **Current members get 7 days.** Members who already belong to the team have a 7-day grace period from the moment enforcement is enabled. After that, they are asked to set up 2FA before they can continue using the dashboard, and are prompted to set it up at their next sign in.
- **New members set up immediately.** Anyone added to the team after enforcement is enabled must set up 2FA before they can use their account.
- **Pending members are flagged.** On the [Team Members](https://testingbot.com/members/sub) list, anyone who has not yet set up 2FA shows a `2FA pending` badge, so you can see compliance at a glance.
Members set up 2FA through the normal flow described in [Two-Factor Authentication](https://testingbot.com/support/team/2fa): an authenticator app (recommended) or SMS.
## Reset a member's 2FA
If a member loses access to their authenticator app or phone, a team admin can reset their 2FA. On the [Team Members](https://testingbot.com/members/sub) list, open the actions menu (`…`) next to the member and choose **Reset 2FA**.
The member's existing 2FA is removed and they are emailed about the change. The next time they sign in they are prompted to set up two-factor authentication again. Their old authenticator entry for TestingBot will no longer work, so they should remove it and add a fresh one during setup.
## Disable enforcement
To stop requiring 2FA across your team, open the [Authentication](https://testingbot.com/members/authentication) tab and click **Disable enforcement**. Members who already enabled 2FA keep it; they are simply no longer forced to use it. You can re-enable enforcement at any time.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/team/rbac
# Role-based access control
Role-based access control (RBAC) lets you control what each member of your team can see and do. TestingBot uses two types of roles. One type governs administrative authority over the account. The other governs which product capabilities a member can use. Keeping these separate lets you grant someone broad product access without making them an account administrator, or the other way around.
Role-based access control is available on the Enterprise plan. This includes the IAM and RBAC roles, custom roles, and per-member access management described on these pages. On other plans, the [Roles & Permissions](https://testingbot.com/members/roles) page shows the built-in roles with an option to upgrade. [See plans](https://testingbot.com/pricing).
This page is an overview. It explains the two role types, the built-in roles, where to manage them, and how TestingBot decides whether a given action is allowed.
## The two role types
Every team member has exactly one role of each type. The two types answer different questions and do not overlap.
- **IAM (Identity & Access Management): administrative authority.** This type decides who can manage the account itself, such as inviting and removing members, changing roles, managing billing and the subscription, and configuring team-wide settings. The IAM roles are **Owner** , **Admin** , and **User**. The Owner has full control of the account. Admins can manage most team and account settings. Users have no administrative authority.
- **RBAC (Role-Based Access Control): product capability.** This type decides what a member can do with the testing products, such as running and managing tests, viewing results, and changing test data. The RBAC roles are **Admin** , **User** , and **Viewer** , plus any **custom roles** you create. An Admin has full product capability. A User can run and manage tests but cannot delete them. A Viewer has read-only access.
Because the two role types are independent, a person can be an IAM **User** with no administrative power while holding an RBAC **Admin** role with full product capability, or any other combination that fits their responsibilities.
## Built-in roles
Each role type ships with a fixed set of built-in roles. Built-in roles cannot be edited or removed. If you need finer-grained product permissions, create a [custom role](https://testingbot.com/support/team/rbac/custom-roles) for the RBAC product capability.
| Type | Role | Description |
| --- | --- | --- |
| IAM | **Owner** | Full control of the account, including billing, the subscription, and all members. There is one Owner per account. |
| IAM | **Admin** | Manages team members, roles, and most account settings. Cannot transfer ownership of the account. |
| IAM | **User** | No administrative authority. Belongs to the team but cannot manage members, roles, or billing. |
| RBAC | **Admin** | Full product capability. Can run, manage, and delete tests and test data across the team. |
| RBAC | **User** | Can run and manage tests. Cannot delete tests or test data. |
| RBAC | **Viewer** | Read-only access. Can view tests and results but cannot run or modify anything. |
## Where to manage roles
Admins open Team Management and select the **Roles & Permissions** tab at [/members/roles](https://testingbot.com/members/roles) to review the built-in roles and to create custom RBAC roles.
Each team member shows both an IAM role and an RBAC role on the [Team Members](https://testingbot.com/members/sub) page, where you can change either role for an individual member.
Custom RBAC roles let you define your own permission sets. See [Custom roles](https://testingbot.com/support/team/rbac/custom-roles) for details.
## How access is decided
When a member attempts an action, TestingBot resolves both roles.
- The **IAM role** is resolved from the member's role in the account (Owner, Admin, or User). This determines whether an administrative action is allowed.
- The **RBAC role** is resolved from the member's tier (Admin, User, or Viewer) or from the custom role assigned to them. This determines which product capabilities are granted.
- Product access also depends on your **plan**. A capability is available only when the RBAC role grants it and the plan includes it. Both conditions must be true, so a permission the plan does not cover is never available regardless of role.
The **Owner** always has full access and is never restricted by RBAC roles.
## Learn more
- [Roles & permissions reference](https://testingbot.com/support/team/rbac/roles-and-permissions): the full permission reference for the built-in roles.
- [Custom roles](https://testingbot.com/support/team/rbac/custom-roles): define your own RBAC permission sets.
- [Member roles](https://testingbot.com/support/team/rbac/member-roles): assign IAM and RBAC roles to individual team members.
- [Product access](https://testingbot.com/support/team/rbac/product-access): how plans and RBAC roles combine to grant capabilities.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/team/rbac/roles-and-permissions
# Roles and permissions reference
This page is the authoritative catalog of every permission in TestingBot. Permissions are split across two role types. RBAC permissions control what you can do inside the testing products. IAM permissions control who can administer the account and grant access to others.
Each permission has a stable key (for example `tests.run`). The tables below show the exact key for every permission and which built-in role grants it. For an introduction to how these two role types fit together, see [Role-based access control](https://testingbot.com/support/team/rbac). To manage assignments, open [Roles & Permissions](https://testingbot.com/members/roles) and [Team Members](https://testingbot.com/members/sub).
Role-based access control is available on the Enterprise plan. [See plans](https://testingbot.com/pricing).
## RBAC permissions
RBAC permissions describe product and entity capabilities: what a member can do in Live Testing, Automated Testing, Visual Testing, and the other products. They are resolved from the RBAC tier (Admin, User, or Viewer) or from an assigned custom role. The account Owner and service accounts always receive full RBAC capability.
A "✓" means the role's bundle grants the permission. An empty cell means it does not.
| Permission | Key | Admin | User | Viewer |
| --- | --- | --- | --- | --- |
| Products |
| --- |
| Live Web Testing | `product.live_web.access` | ✓ | ✓ | ✓ |
| Live App Testing | `product.live_app.access` | ✓ | ✓ | ✓ |
| Automated Testing | `product.automation.access` | ✓ | ✓ | ✓ |
| Visual Testing | `product.visual.access` | ✓ | ✓ | ✓ |
| Accessibility Testing | `product.accessibility.access` | ✓ | ✓ | ✓ |
| AI Testing | `product.ai.access` | ✓ | ✓ | ✓ |
| Tests |
| --- |
| View tests | `tests.view` | ✓ | ✓ | ✓ |
| Run tests | `tests.run` | ✓ | ✓ | |
| Delete tests | `tests.delete` | ✓ | | |
| Integrations |
| --- |
| View integrations | `integrations.view` | ✓ | ✓ | ✓ |
| Reports |
| --- |
| View reports | `reports.view` | ✓ | ✓ | ✓ |
The six `product.*` permissions are necessary but not sufficient: a member can only use a product when both the role grants the product permission and the plan includes that product. In the role editor, products that the plan does not include show a "plan" badge. A member's effective product access is therefore the role permission combined with plan inclusion. For more detail, see [Product access](https://testingbot.com/support/team/rbac/product-access).
## IAM permissions
IAM permissions describe administrative authority: who can manage the account, manage team members, manage billing, and grant access to others. They are derived from the organization role (Owner, Admin, or User) and are fixed, so they cannot be customized.
A "✓" means the role grants the permission. An empty cell means it does not.
| Permission | Key | Owner | Admin | User |
| --- | --- | --- | --- | --- |
| Team |
| --- |
| View team members | `team.members.view` | ✓ | ✓ | |
| Manage team members | `team.members.manage` | ✓ | ✓ | |
| Manage roles | `team.roles.manage` | ✓ | ✓ | |
| Billing |
| --- |
| View billing | `billing.view` | ✓ | ✓ | |
| Manage billing | `billing.manage` | ✓ | ✓ | |
| Account |
| --- |
| View account settings | `account.settings.view` | ✓ | ✓ | |
| Manage account settings | `account.settings.manage` | ✓ | ✓ | |
| Transfer ownership | `account.transfer_ownership` | ✓ | | |
| Delete account | `account.delete` | ✓ | | |
| Manage integrations | `integrations.manage` | ✓ | ✓ | |
| Manage service accounts | `service_accounts.manage` | ✓ | ✓ | |
| Manage security | `security.manage` | ✓ | ✓ | |
Note that `integrations.manage` is an IAM permission (who can configure integrations for the account), while `integrations.view` in the RBAC table above is a separate product capability. For background on these two role types, see [Member roles](https://testingbot.com/support/team/rbac/member-roles) and [Service accounts](https://testingbot.com/support/team/service-accounts).
## Key differences
- A **Viewer** (RBAC) can view, but cannot run or delete tests: both `tests.run` and `tests.delete` are withheld. This is the read-only product role.
- A **User** (RBAC) can do everything a member needs day to day, but cannot delete tests: `tests.delete` is reserved for Admin.
- An **Admin** (IAM) can manage the team, billing, account settings, integrations, service accounts, and security, but cannot transfer ownership or delete the account: `account.transfer_ownership` and `account.delete` are reserved for the Owner.
- A **User** (IAM) has no administrative authority at all. IAM permissions are fixed and derived from the organization role, so they cannot be customized.
- **Custom roles are RBAC-only.** They can grant or withhold any RBAC permission, but can never grant IAM permissions. See [Custom roles](https://testingbot.com/support/team/rbac/custom-roles).
Related reading: [Roles and permissions overview](https://testingbot.com/support/team/rbac/roles-and-permissions), [Sub-accounts](https://testingbot.com/support/team/sub-accounts), [Two-factor authentication](https://testingbot.com/support/team/2fa), [Enforce two-factor authentication](https://testingbot.com/support/team/enforce-2fa), and [Audit logs](https://testingbot.com/support/team/audit-logs).
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/team/rbac/custom-roles
# Manage custom roles
Custom roles let you define granular role-based access control (RBAC) roles for your team. Instead of using only the built-in Admin, User, and Viewer tiers, you can build a role that grants exactly the products, test actions, integration views, and reports each member needs.
Role-based access control, including custom roles, is available on the Enterprise plan. On other plans, the [Roles & Permissions](https://testingbot.com/members/roles) page shows the built-in roles with an option to upgrade. [See plans](https://testingbot.com/pricing).
Custom roles are RBAC-only: they control what a member can do inside the products. They can never grant administrative (IAM) permissions such as managing billing, integrations, or other team members. For the difference between the two role types, see [Role-Based Access Control](https://testingbot.com/support/team/rbac) and [Roles & Permissions](https://testingbot.com/support/team/rbac/roles-and-permissions).
## Create a custom role
Go to [Roles & Permissions](https://testingbot.com/members/roles) and click **New role**. The role editor has the following fields:
- **Role name:** a short, descriptive name shown when you assign the role to a member.
- **Base tier:** the starting tier for the role, one of Admin, Member, or Viewer. This sets the ceiling for the non-product permissions the role can include.
- **Description:** an optional note explaining what the role is for.
- **Permissions:** a grid of permissions to enable, grouped by Products, Tests, Integrations, and Reports.
The permissions grid covers the full RBAC set:
- **Products:** Live Web Testing, Live App Testing, Automated Testing, Visual Testing, Accessibility Testing, and AI Testing. Each product also requires your plan to include that product, shown by a **plan** badge in the editor.
- **Tests:** View tests, Run tests, and Delete tests.
- **Integrations:** View integrations.
- **Reports:** View reports.
For reference, the built-in tiers bundle these permissions as follows:
| Permission | Admin | User | Viewer |
| --- | --- | --- | --- |
| Product access (all 6 products) | ✓ | ✓ | ✓ |
| View tests | ✓ | ✓ | ✓ |
| Run tests | ✓ | ✓ | · |
| Delete tests | ✓ | · | · |
| View integrations | ✓ | ✓ | ✓ |
| View reports | ✓ | ✓ | ✓ |
Admin includes all 11 RBAC permissions. User includes everything except Delete tests. Viewer is read-only, with everything except Run tests and Delete tests.
## Base tier and ceiling
The base tier you select caps which non-product permissions the role can include. A role with a Viewer base tier cannot exceed Viewer, so it cannot grant Run tests or Delete tests no matter what you select in the grid. Choose a base tier that matches the highest level of access the role should reach.
Product permissions are not capped by the base tier. Instead, each product permission is gated by your plan: the member gets access to a product only when your plan includes that product **and** the role grants `product..access`. A member's effective product access is therefore the plan and the role combined.
## Rules and limits
To keep custom roles safe, the editor enforces a few rules:
- You can only grant permissions you hold yourself. Permissions you do not have are greyed out in the grid.
- Custom roles cannot grant administrative (IAM) permissions. They are RBAC-only and never include things like managing integrations, billing, or team members.
- Owner-only permissions cannot be granted by a non-owner admin.
- Only the account owner can create Admin-tier roles.
The account owner and service accounts always have full RBAC capability and are not limited by these roles.
## Assign a custom role
Once a custom role exists, assign it to a member using **Modify access** on the [Team Members](https://testingbot.com/members/sub) page. Your custom roles appear in the same list as the built-in tiers. For the full assignment workflow, see [Member roles](https://testingbot.com/support/team/rbac/member-roles).
## Edit or delete
You can edit or delete a custom role from the [Roles & Permissions](https://testingbot.com/members/roles) page.
- **Editing** a role updates every member who holds it immediately. Adding or removing a permission changes what those members can do right away.
- **Deleting** a role reverts its members to a built-in role, so no one is left without any access.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/team/rbac/member-roles
# Assign member roles
Every person on your TestingBot team has two roles that work together. An **IAM role** controls administrative authority: who can manage the account, billing, and team. An **RBAC role** controls product capability: what they can do inside the testing products.
Both roles are shown for each person on the [Team Members](https://testingbot.com/members/sub) page. This guide explains how to change them. For the full breakdown of every permission, see [Roles & Permissions](https://testingbot.com/support/team/rbac/roles-and-permissions) and the [RBAC overview](https://testingbot.com/support/team/rbac).
Role-based access control is available on the Enterprise plan. [See plans](https://testingbot.com/pricing).
## Modify access
Go to the [Team Members](https://testingbot.com/members/sub) page, open the actions menu next to the member you want to update, and choose **Modify access**. A dialog opens with separate selectors for the IAM role and the RBAC role, plus the member's concurrency limit.
Adjust the role selectors as needed and save. The new permissions apply immediately on the member's next request.
## IAM role
The IAM role decides how much administrative authority a member has over the account itself. Choose **Owner** , **Admin** , or **User**. Only the account owner can change a member's IAM role, and IAM roles are fixed (they cannot be customized).
- **Owner:** full administrative authority over the account, including transferring ownership and deleting the account. There is one owner per account.
- **Admin:** can manage team members, roles, billing, account settings, integrations, service accounts, and security. An Admin cannot transfer ownership or delete the account.
- **User:** no administrative authority. A User can still work in the products according to their RBAC role, but cannot manage the team, billing, or account settings.
The table below shows which administrative permissions each IAM role grants.
| Administrative permission | Owner | Admin | User |
| --- | --- | --- | --- |
| View team members | ✓ | ✓ | · |
| Manage team members | ✓ | ✓ | · |
| Manage roles | ✓ | ✓ | · |
| View billing | ✓ | ✓ | · |
| Manage billing | ✓ | ✓ | · |
| View account settings | ✓ | ✓ | · |
| Manage account settings | ✓ | ✓ | · |
| Manage integrations | ✓ | ✓ | · |
| Manage service accounts | ✓ | ✓ | · |
| Manage security | ✓ | ✓ | · |
| Transfer ownership | ✓ | · | · |
| Delete account | ✓ | · | · |
## RBAC role
The RBAC role decides what a member can do inside the products. Choose **Admin** , **User** , or **Viewer**. On Enterprise plans you can also assign a [custom role](https://testingbot.com/support/team/rbac/custom-roles). Any team admin can change a member's RBAC role.
- **Admin:** full product capability, including running and deleting tests across every product your plan includes.
- **User:** can view and run tests, but cannot delete tests.
- **Viewer:** read-only. A Viewer can view tests, reports, and integrations, but cannot run or delete tests.
The table below shows which product capabilities each RBAC role grants.
| Product capability | Admin | User | Viewer |
| --- | --- | --- | --- |
| Live Web Testing | ✓ | ✓ | ✓ |
| Live App Testing | ✓ | ✓ | ✓ |
| Automated Testing | ✓ | ✓ | ✓ |
| Visual Testing | ✓ | ✓ | ✓ |
| Accessibility Testing | ✓ | ✓ | ✓ |
| AI Testing | ✓ | ✓ | ✓ |
| View tests | ✓ | ✓ | ✓ |
| Run tests | ✓ | ✓ | · |
| Delete tests | ✓ | · | · |
| View integrations | ✓ | ✓ | ✓ |
| View reports | ✓ | ✓ | ✓ |
Each product capability also requires your plan to include that product. A member's effective product access is the combination of the plan including the product **and** the RBAC role granting access to it. The account owner and [service accounts](https://testingbot.com/support/team/service-accounts) always have full RBAC capability. Custom roles are RBAC-only and can never grant IAM permissions.
## Concurrency limit
From the same **Modify access** dialog you can set how many parallel tests the member is allowed to run. This caps the member's share of your account's total parallel capacity. For details on how concurrency is allocated across the team, see the concurrency section in [Team & Sub-Accounts](https://testingbot.com/support/team/sub-accounts#concurrency).
## Auditing
Every role change is recorded so you always know who changed what and when. Review these entries in the [Activity Log](https://testingbot.com/support/team/audit-logs).
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/team/rbac/product-access
# Manage product access
Product access controls which TestingBot products a role can use: Live Web Testing, Live App Testing, Automated Testing, Visual Testing, Accessibility Testing, and AI Testing. You manage these permissions per role under [Roles & Permissions](https://testingbot.com/members/roles), then assign roles to your [Team Members](https://testingbot.com/members/sub).
Product access is part of TestingBot's role-based access control (RBAC). For the full picture, see [Role-Based Access Control](https://testingbot.com/support/team/rbac) and [Roles & Permissions](https://testingbot.com/support/team/rbac/roles-and-permissions).
Role-based access control is available on the Enterprise plan. [See plans](https://testingbot.com/pricing).
## Two gates
Whether a team member can open a product depends on two gates. A member gets access only when **both** are satisfied:
- **Your plan includes the product.** Each product is part of your TestingBot subscription. If your plan does not include a product, no role can grant access to it.
- **The role has the matching permission.** The role assigned to the member must include the relevant `product..access` permission.
In the role editor, each product permission shows a **plan** badge to indicate the plan gate. The badge is a reminder that the permission only takes effect when your plan provides that product.
The account Owner and any service accounts always have full product capability, subject to the same plan gate. See [Service Accounts](https://testingbot.com/support/team/service-accounts).
## Built-in roles
The three built-in roles, **Admin** , **User** , and **Viewer** , all include access to every product your plan provides. The difference between them is not which products they can open, but what they can do inside those products: creating, running, and deleting tests.
- **Admin** has full capability, including deleting tests.
- **User** can view and run tests, but cannot delete them.
- **Viewer** is read-only: view tests and reports, but cannot run or delete.
For the complete capability breakdown, see [Roles & Permissions](https://testingbot.com/support/team/rbac/roles-and-permissions) and [Member Roles](https://testingbot.com/support/team/rbac/member-roles).
## Restrict to specific products
To give a team access to only some products rather than all of them, create an Enterprise [custom role](https://testingbot.com/support/team/rbac/custom-roles) and enable only the product permissions you want. For example, a custom role could grant Visual Testing and Accessibility Testing while leaving Automated Testing disabled.
Custom roles are RBAC-only: they control product and test capability, but they can never grant identity and access management (IAM) permissions such as managing billing or team members. Assign a custom role to a member from [Team Members](https://testingbot.com/members/sub) the same way you assign a built-in role.
## Available products
The six products and their permission keys are listed below. Each key also requires your plan to include that product, shown by the **plan** badge in the role editor.
- **Live Web Testing** : `product.live_web.access`
- **Live App Testing** : `product.live_app.access`
- **Automated Testing** : `product.automation.access`
- **Visual Testing** : `product.visual.access`
- **Accessibility Testing** : `product.accessibility.access`
- **AI Testing** : `product.ai.access`
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/team/account
# How to update my account details?
You can update your account details (email, first name, last name) by going to [Members \> Account](https://testingbot.com/members/user/edit).
In the form "Edit your account details" you can update your personal details.
## Resetting Security Tokens
You can find your Security Tokens (`Key` + `Secret`) by going to [Members \> Manage Credentials](https://testingbot.com/members/user/security).
These two identifiers are necessary to authenticate to our Selenium grid.
It is possible to reset these security tokens by clicking the "Generate new keys" link.
**Caution:** resetting these tokens means your existing tests with older key/secret credentials will stop running.
## Change Password
You can change your current password by logging into TestingBot and going to [Members \> Security](https://testingbot.com/members/user/security).
Click "Change Password". You will be asked to fill in your current password and a new password.
## Active Sessions
On the [Security page](https://testingbot.com/members/user/security) you can find an overview of the active sessions for your account.
Each entry contains the logged-in IP address, device name and last active time.
You can sign out of individual sessions, or sign out everywhere.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/team/audit-logs
# TestingBot Audit Logs
Audit logs provide a comprehensive record of all activities performed by team members in your TestingBot organization. These logs help you maintain security, ensure compliance and monitor user behavior across your testing infrastructure.
Only organization administrators can view and manage audit logs. Audit logs are retained for 400 days.
## How to Access Audit Logs
To view your organization's audit logs:
1. Log into your TestingBot account
2. Navigate to [Members → Team Activity](https://testingbot.com/members/sub/activity)
3. View the comprehensive activity log for your organization
The audit log interface provides filtering options to help you find specific events by date range, event type or team member.
## Event Types
TestingBot groups the tracked activities into the following categories:
### Authentication
| Event | Description |
| --- | --- |
| Login | User successfully logged into the TestingBot platform |
| Logout | User logged out of the TestingBot platform |
| Failed Login | A failed sign-in attempt was recorded for an account |
### Account Management
| Event | Description |
| --- | --- |
| Add Account | New team member was added to the organization |
| Remove Account | Team member was removed from the organization |
| Profile Update | User updated their profile information (name, email, etc.) |
| Email Changed | A user changed their account email address (old and new address recorded) |
| Role / Permission Changed | A team member's read-only (Viewer) permission was enabled or removed |
### Plan & Configuration
| Event | Description |
| --- | --- |
| Upgrade Level | Account plan was upgraded to a higher tier |
| Downgrade Level | Account plan was downgraded to a lower tier |
| Max Parallel Changed | Maximum parallel test sessions limit was modified |
| Max Parallel Device Changed | Maximum parallel device sessions limit was modified |
### Security
| Event | Description |
| --- | --- |
| Password Changed | User changed their account password |
| Change Keys | User regenerated their API access keys (Key + Secret) |
| 2FA Enabled | Two-factor authentication was enabled for the account |
| 2FA Disabled | Two-factor authentication was disabled for the account |
| Sessions Cleared | An administrator force-cleared a team member's active sessions |
| Audit Log Exported | A team administrator exported the audit log to CSV |
### Team Invitations
| Event | Description |
| --- | --- |
| Invite Sent | Team invitation was sent to a new member |
| Invite Accepted | Team member accepted their invitation and joined the organization |
| Invite Denied | Team invitation was declined or rejected |
| Invite Resent | A pending team invitation was resent to the recipient |
| Invite Revoked | A pending team invitation was cancelled before being accepted |
### Integrations
| Event | Description |
| --- | --- |
| Webhook Created | A webhook endpoint was created for the account |
| Webhook Updated | An existing webhook endpoint was updated |
| Webhook Deleted | A webhook endpoint was deleted |
| Integration Connected | A third-party integration (Jira, Slack or Microsoft Teams) was connected |
| Integration Disconnected | A third-party integration was disconnected |
| Notification Settings Changed | Notification or SMS alert settings were updated |
### Billing
| Event | Description |
| --- | --- |
| Subscription Cancelled | The account subscription was cancelled |
| Subscription Paused | A subscription pause was requested |
| Retention Discount Requested | A retention discount was requested from the cancellation flow |
## Filtering and Searching
The audit log interface provides several filtering options to help you find specific events:
- **Filter by Event Type:** Select specific event types to view only those activities
- **Filter by User:** Show activities performed by a specific team member
- **Filter by Date Range:** Specify a custom date range to narrow down the results
## Data Retention and Export
You can export all audit logs in `.CSV` format.
For compliance and record-keeping purposes, we recommend regularly exporting your audit logs if you need to maintain historical records beyond the 400-day retention period.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/team/sso
# Setting Up Single Sign On (SSO)
TestingBot supports Identity Provider (IdP)-initiated and Service Provider (SP)-initiated Single Sign-On (SSO) via the SAML 2.0 protocol. This allows your authorized employees to access TestingBot in a moderated fashion, as an alternative to using passwords.
To start using SSO, please make sure you have an [Identity Provider (IdP)](https://testingbot.com#idp) set up.
## Setting Up Identity Provider
TestingBot provides preconfigured SAML applications for a selection of Identity Providers (IdPs). These applications allow you to integrate SSO with TestingBot in a very easy way. Below is a list of SAML applications available:
- [Microsoft Entra](https://testingbot.com/support/team/sso/microsoft-entra)
- [Okta](https://testingbot.com/support/team/sso/okta)
To set up a custom Identity Provider, such as Salesforce, ForgeRock, Auth0 or others, please follow the steps below.
1. Retrieve the [SAML metadata](https://testingbot.com/users/saml/metadata) from TestingBot.
2. Log in to your identity provider's administrator panel.
3. If your IdP does not allow you to upload the metadata file, you can set up the integration manually.
4. Export SAML metadata of your newly created SAML application. Please [email it to us](https://testingbot.com/contact/new) so that we can set up the connection for you.
## Service Provider SAML Requirements
Below is a list of settings that are required by the TestingBot Service Provider:
| Setting Name | Value |
| --- | --- |
| Entity ID | `https://testingbot.com/users/saml/metadata` |
| Assertion Consumer Service (ACS URL, Reply URL) | `https://testingbot.com/users/saml/auth` |
| Name ID (Unique User Identifier) | `email` |
| Name ID format/policy | `urn:oasis:names:tc:SAML:2.0:nameid-format:transient` |
| Binding | `urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST` |
| Idp Cert Fingerprint Algorithm | `http://www.w3.org/2000/09/xmldsig#sha256` |
| Login URL | `https://testingbot.com/users/saml/sign_in` |
### SAML Claims
TestingBot Service Provider supports the following SAML custom claims:
- `email`
- `http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress`
- `http://schemas.xmlsoap.org/ws/2005/05/identity/claims/email`
- `first_name`
- `http://schemas.xmlsoap.org/ws/2005/05/identity/claims/givenname`
- `last_name`
- `http://schemas.xmlsoap.org/ws/2005/05/identity/claims/surname`
## SSO Options
We provide the following options with Single Sign On:
### Just-In-Time (JIT) Provisioning
If a user from your organization logs in through SSO, we can automatically create an account for this user on TestingBot, which will be associated to your team. Each user will be able to see the tests created by other members of your organization.
If this setting is disabled, users that are not yet registered with TestingBot will not be able to use TestingBot.
#### Enforce SSO (Big Bang)
When this setting is enabled, users in your organization must log in through SSO.
All other authentication methods will not be allowed.
#### Email verification
New users logging in through SAML for the first time will automatically have their account (and email address) verified.
#### Logout
TestingBot supports both SP-Initiated logout and IdP-initiated logout. If you'd like to set up SP-initiated logout, please [contact us](https://testingbot.com/contact/new) with your IdP SLO endpoint.
## Unsupported Features
We currently do not provide support for these features:
- SCIM (System for Cross-domain Identity Management)
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/team/sso/microsoft-entra
# Setting up SSO with Microsoft Entra
TestingBot has created a SAML app in Microsoft Entra which you can use to easily enable SSO with Entra and TestingBot.
1. Go to **Browse Microsoft Entra Gallery** and search for **TestingBot**.
2. Click the TestingBot app to go to the detail page. Click the **Create** button to add the app.
3. In the **Properties** section, make sure the **User Identifier** claim is set to a valid email address. You can adjust this later under **Attributes & Claims** in the Entra SAML configuration.
4. Click **Single Sign on** in the Manage section, then select **SAML** as the single sign-on method.
5. Make sure you fill in the required information in the **Basic SAML Configuration** , such as the **Identifier (Entity ID)** and **Reply URL (Assertion Consumer Service URL)** as listed on the [SSO setup page](https://testingbot.com/support/team/sso).
6. Download the **Federation Metadata XML** from the SAML Signing Certificate section, then send it to TestingBot via the [contact form](https://testingbot.com/contact/new) so we can finish setting up SSO on our side.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/team/sso/okta
# Setting up SSO in Okta
TestingBot has created a SAML app in Okta which you can install to easily enable SSO with Okta and TestingBot.
1. Log in to the **Okta administrator panel** , go to **Applications** and search for TestingBot in the **Browse App Integration Catalog**.
2. Click the TestingBot app to go to the detail page. Click the **Add Integration** button to add the TestingBot app.
3. Change the name of the application to your liking, or leave it as TestingBot, then click **Done**.
4. You can now assign one or more users or groups to use the TestingBot application.
5. Click the **Sign On** tab.
6. In the **Credentials Details** section, make sure that the value for **Application username format** is a valid email address. The default format for Okta username is an email address, unless it has been changed in the Okta Admin Console.
7. In the **SAML Signing Certificates** section, click **Actions** , and then **View IdP metadata**. Please [send the metadata to TestingBot](https://testingbot.com/contact/new).
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/team/service-accounts
# Service Accounts
A service account is an API-only identity that runs tests with its own key and secret. Unlike a [sub-account](https://testingbot.com/support/team/sub-accounts), a service account has no dashboard login: it exists only to authenticate automated tests against the TestingBot REST API and the browsers + device grid.
Every service account belongs to your account owner. Its tests run on your subscription, share your concurrency and appear under your account. It acts exactly like the tests of any team member. You manage service accounts from [Team Management \> Service Accounts](https://testingbot.com/members/service_accounts) (team admins only).
- Each service account has its own **TestingBot Key** and **TestingBot Secret**.
- A service account **cannot sign in** to the TestingBot dashboard and is not counted as a dashboard user.
- Its credentials can be rotated or revoked on their own, without affecting anyone's personal key/secret.
## When to Use a Service Account
Service accounts are the recommended way to run tests from shared, non-human contexts. Use one when you want to:
- Run tests from [CI/CD pipelines](https://testingbot.com/support/integrations/ci-cd) without tying them to a person's login.
- Give each pipeline, environment, or team its own labelled credentials so you can tell their runs apart.
- Rotate or revoke a set of credentials independently if they leak or a pipeline is retired, without disrupting your own access.
## Creating a Service Account
Go to [Team Management \> Service Accounts](https://testingbot.com/members/service_accounts) and click **Add Account**. Enter an [identifier](https://testingbot.com#identifier) (at least 3 characters) and click **Submit**.
The new service account appears in the table straight away with its generated **TestingBot Key** and **TestingBot Secret**. Use the eye icon to reveal a value and the copy icon to copy it. You can come back and reveal these credentials at any time, so there is no one-time "save it now" step.
## The Identifier
The identifier is a human-friendly label you give the service account, for example `ci-runner` or `nightly-regression`. It is used to mark the tests run by that service account on your dashboard and on the test detail page, so you can see at a glance which pipeline or environment started a test.
The identifier must be at least 3 characters and must be unique within your account. You can change it later at any time.
## Using the Credentials
Use the service account's key and secret anywhere you would normally use your own TestingBot key and secret.
### Browsers + Device Grid
Point your tests at the TestingBot hub using the service account's key and secret:
```bash
https://YOUR_KEY:YOUR_SECRET@hub.testingbot.com/wd/hub
```
Or pass them in your capabilities (`tb:options` / `key` and `secret`) exactly as you would with a personal key/secret.
### REST API
The [REST API](https://testingbot.com/support/api) uses HTTP Basic authentication with the key and secret:
```bash
curl -u YOUR_KEY:YOUR_SECRET https://api.testingbot.com/v1/tests
```
We recommend storing the key and secret as environment variables or CI secrets, never committing them to source control.
## Editing a Service Account
To rename a service account, open the actions menu (the **...** button) next to it and choose **Edit**. Change the identifier and click **Submit**. Editing the identifier does not change the key or secret.
## Rotating the Access Key
From the actions menu, choose **Rotate access key** to generate a fresh key and secret for the service account. This is useful if a credential may have leaked.
**Rotating takes effect immediately:** the previous key and secret stop working right away, so any automation still using the old credentials will start failing to authenticate. Update your pipelines with the new key and secret after rotating.
## Deleting a Service Account
From the actions menu, choose **Delete** to remove a service account. Its key and secret stop working immediately. Tests that were already run by the service account stay in your account and keep their history; they remain labelled with the identifier on the test detail page.
## How Test Runs Are Attributed
Tests started by a service account run on your account owner's subscription: they consume your plan's concurrency and minutes and are visible to your team, just like any other test. On the test detail page, the **Started by** field shows the service account's identifier so you can tell its runs apart from those of human team members.
## Service Accounts vs Sub-Accounts
Both run tests under your account, but they serve different purposes:
- **Sub-account:** a person on your team. It has a dashboard login, can view and manage tests, and can be given roles (Admin / Member / Viewer). See [Team & Sub-Accounts](https://testingbot.com/support/team/sub-accounts).
- **Service account:** an API-only identity for automation. It has no dashboard login, no role management, and no 2FA. It exists purely to run tests with a dedicated, rotatable key and secret.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/other
# Other Documentation
Reference pages for the TestingBot Selenium and Appium grid — configuration details, error code lookups, CI/CD build status widgets, proxy setup and miscellaneous integrations.
## Grid reference
- [Grid configuration](https://testingbot.com/support/other/configuration) — endpoints, regions and infrastructure overview.
- [Common errors and failures](https://testingbot.com/support/other/errors) — error code reference for Selenium and Appium tests.
- [Selenium WebDriver examples](https://testingbot.com/support/other/selenium2) — example scripts in Ruby, Python, PHP, Java and C#.
## CI/CD widgets
- [Status badges](https://testingbot.com/support/other/status-badges) — embed Selenium/Appium build status badges in GitHub README files.
- [Browser matrix widgets](https://testingbot.com/support/other/browser-matrix) — embed a live browser-coverage matrix from your test runs.
- [Share test results](https://testingbot.com/support/other/sharing) — view, share and embed Selenium test recordings.
## Integrations & advanced setup
- [Browsermob proxy / HAR files](https://testingbot.com/support/other/proxy) — capture network traffic during Selenium tests.
- [TestingBot on GitHub](https://testingbot.com/support/other/github) — open-source client libraries and example projects.
- [Sikuli image-based testing](https://testingbot.com/support/other/sikuli) — image-recognition testing alongside Selenium.
- [Firefox Unbranded for extensions](https://testingbot.com/support/other/firefox-unbranded) — install and test signed Firefox add-ons.
## Related
- [TestingBot Tunnel](https://testingbot.com/support/tunnel) — test internal websites and pre-production environments.
- [CI/CD integrations](https://testingbot.com/support/integrations/ci-cd) — Jenkins, GitHub Actions, GitLab CI, CircleCI and more.
- [Selenium capabilities reference](https://testingbot.com/support/web-automate/selenium/test-options) — all tb:options capabilities.
---
URL: https://testingbot.com/support/other/configuration
# TestingBot Grid Configuration
TestingBot provides a distributed cloud grid of real browsers and devices. Connect your Selenium, Appium, Puppeteer, Playwright, Cypress, XCUITest and Espresso tests to this grid through the endpoints documented below.
The Selenium and Appium grid runs the latest stable Selenium 4 server and is fully W3C-compliant. Your client SDK version (Selenium, Appium, Playwright, Puppeteer) is matched on the grid side for compatibility.
## Service status
Live incidents and scheduled maintenance are posted on the [TestingBot status page](https://status.testingbot.com/) (RSS feed available). Subscribe to the status page to receive notifications for grid availability, hub-endpoint issues and tunnel infrastructure events.
[](https://stats.pingdom.com/kwiiczwo3ltc) [](https://stats.pingdom.com/kwiiczwo3ltc)
## TestingBot IP Ranges
Below you will find an up-to-date list of the TestingBot IP ranges used during testing.
- 109.68.162.161
- 109.68.162.165
- 78.20.187.211
- 162.55.135.22
- 95.217.83.184
- 95.216.39.110
- 95.216.229.109
- 142.132.137.7
- 195.201.167.43
- 162.55.24.100
- 185.223.133.13
- 135.181.118.186
- 188.34.142.238
- 95.217.228.116
- 65.109.76.224
- 176.9.121.211
- 194.78.229.67
- 87.64.203.47
You can also fetch this up-to-date list from the [TestingBot REST-API](https://testingbot.com/support/api/devices#ipranges).
### Geolocation IP ranges
TestingBot provides [geolocation testing](https://testingbot.com/support/web-automate/selenium/test-options#geo), allowing you to test from various geographical places around the world. Below are the IP addresses we use for this.
- **AU** Australia: `35.213.216.75`
- **BE** Belgium: `109.68.162.161`
- **BR** Brazil: `35.215.244.91`
- **CA** Canada: `34.47.9.124`
- **CL** Chile: `34.176.132.166`
- **FR** France: `15.236.216.234`
- **DE** Germany: `188.245.92.209`
- **IN** India: `35.207.203.231`
- **IT** Italy: `15.161.57.62`
- **JP** Japan: `35.213.44.184`
- **NO** Norway: `51.120.9.213`
- **SG** Singapore: `18.138.255.42`
- **ZA** South Africa: `13.245.163.253`
- **SE** Sweden: `16.171.19.152`
- **CH** Switzerland: `34.65.52.54`
- **AE** United Arab Emirates: `20.174.25.155`
- **GB** United Kingdom: `35.214.33.251`
- **US** United States: `178.156.132.19`
## Selenium grid configuration
W3C-compliant Selenium 4 endpoint. Connect your Selenium WebDriver client (Java, Python, Ruby, Node.js, C#, PHP) to the URL below and pass your API key and secret inside the `tb:options` capability map.
| Key | Value |
| --- | --- |
| Selenium version | Latest stable Selenium 4 (W3C protocol). The grid is fully W3C-compliant. [Pin a specific Selenium version](https://testingbot.com/support/web-automate/selenium/test-options#seleniumversion) with the seleniumVersion capability if needed. |
| Hub endpoint | `https://hub.testingbot.com/wd/hub` |
| Port | 443 (HTTPS, recommended). Plain HTTP on port 80 and 4444 is also accepted but not recommended for production. |
| Default idle timeout | 130 seconds. [Change the idleTimeout capability](https://testingbot.com/support/web-automate/selenium/test-options#idletimeout) to raise or lower this per test. |
| Default screen resolution | 1280x1024. [Change the screenResolution capability](https://testingbot.com/support/web-automate/selenium/test-options#screenresolution) to pick a different resolution per test. |
## Appium grid configuration
Appium 2 endpoint for native mobile app testing and mobile-browser testing on iOS and Android simulators, emulators and real devices. Uses the same hub URL as Selenium.
| Key | Value |
| --- | --- |
| Appium version | Latest compatible Appium version for the specified device, OS and OS version. [Pin a specific Appium version](https://testingbot.com/support/app-automate/appium/options#appiumVersion) with the appiumVersion capability. |
| Hub endpoint | `https://hub.testingbot.com/wd/hub` |
| Port | 443 (HTTPS, recommended). Plain HTTP on port 80 and 4444 is also accepted but not recommended for production. |
| Default idle timeout | 130 seconds. [Change the idleTimeout capability](https://testingbot.com/support/web-automate/selenium/test-options#idletimeout) per test if needed. |
## Puppeteer and Playwright configuration
Puppeteer and Playwright connect over a WebSocket endpoint rather than the WebDriver HTTP hub. Use the appropriate `wss://` URL below and pass your API key, secret, browserName and browserVersion as query parameters or capabilities depending on your client. See [the Playwright support page](https://testingbot.com/support/web-automate/playwright) and [the Puppeteer support page](https://testingbot.com/support/web-automate/puppeteer) for complete examples.
| Key | Value |
| --- | --- |
| Playwright endpoint | `wss://cloud.testingbot.com/playwright` |
| Puppeteer endpoint | `wss://cloud.testingbot.com/puppeteer` |
| Playwright or Puppeteer version | TestingBot matches the server-side version to the client-side version you use, ensuring full compatibility. No version pin is needed on the capabilities. |
| Port | 443 (TLS-encrypted WebSocket). Plain ws:// on port 80 is also accepted but not recommended for production. |
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/other/sharing
### Available Topics:
- [Link to a Test](https://testingbot.com#linktest)
- [Link to a Build](https://testingbot.com#linkbuild)
- [Embed Test Info](https://testingbot.com#embedinfo)
- [Embed Test Video](https://testingbot.com#embedvideo)
# Sharing Tests
Below are various ways to share, embed or link to a test or build on TestingBot.
To share, view or link to a test, it is important to know its `sessionId`.
Selenium and Appium will return a `sessionId` when a client starts a new session.
This sesssionId is a unique identifier and changes per test. It is stored as a member variable of the instantiated Driver object. Most of the time it is named `sessionId` or `session_id`, depending on the client you are using.
## Link to a Test
You can easily share a single test result with someone in a safe way, without the other person needing an account.
To share a test result, you'll need to create the following url:
https://testingbot.com/tests/ **sessionId**?auth= **authentication-hash**
The authentication hash is created with md5 in the following format:
[Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#)
```ruby
Digest::MD5.hexdigest("#{client_key}:#{client_secret}:#{session_id}")
```
```java
MessageDigest m=MessageDigest.getInstance("MD5");
String s = "TESTINGBOT_KEY:TESTINGBOT_SECRET:session_id";
m.update(s.getBytes(),0,s.length());
System.out.println("MD5: "+new BigInteger(1,m.digest()).toString(16));
```
```php
echo md5("TESTINGBOT_KEY:TESTINGBOT_SECRET:$session_id");
```
```python
import hashlib
print(hashlib.md5("TESTINGBOT_KEY:TESTINGBOT_SECRET:#{session_id}".encode('utf-8')).hexdigest())
```
```javascript
var crypto = require('crypto');
crypto.createHash('md5').update("TESTINGBOT_KEY:TESTINGBOT_SECRET:sessionId").digest("hex");
```
```csharp
Convert.ToBase64String(new System.Security.Cryptography.MD5CryptoServiceProvider().ComputeHash(System.Text.Encoding.UTF8.GetBytes("TESTINGBOT_KEY:TESTINGBOT_SECRET:sessionId")));
```
An example link would then be:
https://testingbot.com/tests/9jf392fj3j2f3ojid8?auth=230i439uf3fojf
## Link to a Video
You can easily share a video of a specific test, please see the example below:
To build the URL with the correct authentication hash, the URL should be structured like this:
https://testingbot.com/tests/ **sessionId.mp4**?auth= **authentication-hash**
The authentication hash is created with MD5 in the following format:
[Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#)
```ruby
Digest::MD5.hexdigest("#{client_key}:#{client_secret}:#{session_id}")
```
```java
MessageDigest m=MessageDigest.getInstance("MD5");
String s = "TESTINGBOT_KEY:TESTINGBOT_SECRET:session_id";
m.update(s.getBytes(),0,s.length());
System.out.println("MD5: "+new BigInteger(1,m.digest()).toString(16));
```
```php
echo md5("TESTINGBOT_KEY:TESTINGBOT_SECRET:$session_id");
```
```python
import hashlib
print(hashlib.md5("TESTINGBOT_KEY:TESTINGBOT_SECRET:#{session_id}".encode('utf-8')).hexdigest())
```
```javascript
var crypto = require('crypto');
crypto.createHash('md5').update("TESTINGBOT_KEY:TESTINGBOT_SECRET:sessionId").digest("hex");
```
```csharp
Convert.ToBase64String(new System.Security.Cryptography.MD5CryptoServiceProvider().ComputeHash(System.Text.Encoding.UTF8.GetBytes("TESTINGBOT_KEY:TESTINGBOT_SECRET:sessionId")));
```
An example link would then be:
https://testingbot.com/tests/9jf392fj3j2f3ojid8.mp4?auth=230i439uf3fojf
## Link to a Build
With TestingBot, it is possible to group several tests under one build identifier.
In our member area, you can then see all the tests grouped by builds.
The build overview at `https://testingbot.com/builds/YOUR_TESTINGBOT_KEY` is only available for accounts whose test results are public. For a private account, share a link to a single build instead.
To link directly to a build, please see this example:
`https://testingbot.com/builds/YOUR_TESTINGBOT_KEY/build-identifier?auth=authentication-hash`
To generate the authentication-hash, you must concatenate user key, user secret and build-identifier, then generate the MD5 hash:
[Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#)
```ruby
Digest::MD5.hexdigest("#{client_key}:#{client_secret}:#{build_identifier}")
```
```java
MessageDigest m=MessageDigest.getInstance("MD5");
String s = "TESTINGBOT_KEY:TESTINGBOT_SECRET:build_identifier";
m.update(s.getBytes(),0,s.length());
System.out.println("MD5: "+new BigInteger(1,m.digest()).toString(16));
```
```php
echo md5("TESTINGBOT_KEY:TESTINGBOT_SECRET:$build_identifier");
```
```python
import hashlib
print(hashlib.md5("TESTINGBOT_KEY:TESTINGBOT_SECRET:#{build_identifier}".encode('utf-8')).hexdigest())
```
```javascript
var crypto = require('crypto');
crypto.createHash('md5').update("TESTINGBOT_KEY:TESTINGBOT_SECRET:build_identifier").digest("hex");
```
```csharp
Convert.ToBase64String(new System.Security.Cryptography.MD5CryptoServiceProvider().ComputeHash(System.Text.Encoding.UTF8.GetBytes("TESTINGBOT_KEY:TESTINGBOT_SECRET:build_identifier")));
```
The build-identifier is the custom string you can [send as an option to group several tests.](https://testingbot.com/support/web-automate/selenium/test-options#build)
## Embed Test Info
You can embed a test page in any CI detail page, test report or custom website. Please see the example below.
You add a SCRIPT tag in your page which links to the test. Replace sessionId with the correct sessionId of the test.
```php
```
To generate the authentication-token, you must concatenate user key, user secret and sessionId, then generate the MD5 hash:
[Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#)
```ruby
Digest::MD5.hexdigest("#{client_key}:#{client_secret}:#{sessionId}")
```
```java
MessageDigest m=MessageDigest.getInstance("MD5");
String s = "TESTINGBOT_KEY:TESTINGBOT_SECRET:sessionId";
m.update(s.getBytes(),0,s.length());
System.out.println("MD5: "+new BigInteger(1,m.digest()).toString(16));
```
```php
echo md5("TESTINGBOT_KEY:TESTINGBOT_SECRET:$sessionId");
```
```python
import hashlib
print(hashlib.md5("TESTINGBOT_KEY:TESTINGBOT_SECRET:#{sessionId}".encode('utf-8')).hexdigest())
```
```javascript
var crypto = require('crypto');
crypto.createHash('md5').update("TESTINGBOT_KEY:TESTINGBOT_SECRET:sessionId").digest("hex");
```
```csharp
Convert.ToBase64String(new System.Security.Cryptography.MD5CryptoServiceProvider().ComputeHash(System.Text.Encoding.UTF8.GetBytes("TESTINGBOT_KEY:TESTINGBOT_SECRET:sessionId")));
```
## Embed Video Info
Next to embedding test info pages, you can also embed the video of the test.
This might be useful to include in test reports or on CI result pages.
```php
```
To generate the authentication-token, you must concatenate user key, user secret and sessionId, then generate the MD5 hash:
[Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#)
```ruby
Digest::MD5.hexdigest("#{client_key}:#{client_secret}:#{sessionId}")
```
```java
MessageDigest m=MessageDigest.getInstance("MD5");
String s = "TESTINGBOT_KEY:TESTINGBOT_SECRET:sessionId";
m.update(s.getBytes(),0,s.length());
System.out.println("MD5: "+new BigInteger(1,m.digest()).toString(16));
```
```php
echo md5("TESTINGBOT_KEY:TESTINGBOT_SECRET:$sessionId");
```
```python
import hashlib
print(hashlib.md5("TESTINGBOT_KEY:TESTINGBOT_SECRET:#{sessionId}".encode('utf-8')).hexdigest())
```
```javascript
var crypto = require('crypto');
crypto.createHash('md5').update("TESTINGBOT_KEY:TESTINGBOT_SECRET:sessionId").digest("hex");
```
```csharp
Convert.ToBase64String(new System.Security.Cryptography.MD5CryptoServiceProvider().ComputeHash(System.Text.Encoding.UTF8.GetBytes("TESTINGBOT_KEY:TESTINGBOT_SECRET:sessionId")));
```
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/other/sikuli
# Sikuli Cloud Testing
Sikuli uses image recognition to identify components you want to interact with on the screen.
While Selenium is limited to interacting with browsers, Sikuli can be used to interact with native GUI components (for example: Windows dialogs, Linux programs or OS-X GUI).
At TestingBot, all our VMs are able to run your Sikuli scripts.
## 1. Download and Setup Sikuli
To get started, please [download Sikuli](http://www.sikulix.com).
Once downloaded, run the file to install Sikuli.
## 2. Create a new Sikuli script
Sikuli uses image-recognition to find the components to interact with.
In this example, we'll take a screenshot of a Chrome dialog asking to install a Chrome extension.
## 3. The Sikuli script
As you can see on the screenshot above, you can use `wait([image], FOREVER)` to have Sikuli wait for something to appear on the screen that looks like the `[image]`.
Once this happens, Sikuli will move on to the next step in the script: clicking the component it detected.
## 4. Integration with TestingBot
At TestingBot, we can run your Sikuli script at the start of your Selenium WebDriver test.
Simply create a zip-file with one or more of your `.sikuli` scripts inside. Then use this URL in your `DesiredCapabilities` when starting a test. This way you can run Sikuli tests in the cloud on all our [platforms](https://testingbot.com/support/web-automate/browsers).
For example:
```java
DesiredCapabilities caps = new DesiredCapabilities();
caps.setCapability("platform", "VISTA");
caps.setCapability("version", "9");
caps.setCapability("browserName", "internet explorer");
caps.setCapability("sikuli", "http://mywebsite.com/sikuli-scripts.zip");
```
This will make sure TestingBot will download your Sikuli script and run it at the start of your test.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/other/errors
# TestingBot Test Errors and How to Fix Them
Reference for the most common errors TestingBot returns when an automated or live test ends unexpectedly. Each section explains what triggered the error, the most common causes, and how to fix it. Errors appear on the test detail page in the dashboard and in the REST API response.
## Minutes depleted
### What it means
The test failed because your account does not have enough test minutes left. New tests cannot be started on TestingBot until minutes are restored.
### How to resolve
See [our pricing plans](https://testingbot.com/pricing) and upgrade your subscription to add minutes immediately, or wait for your monthly plan to reset. Free-tier accounts receive a fresh allowance each month. Enterprise plans can request additional minute pools mid-cycle.
## Test exceeded max duration
### What it means
The test was terminated because it exceeded the maximum duration allowed. The default cap is 1800 seconds (30 minutes).
### Common causes
- The test is genuinely long. Consider splitting it into multiple smaller, atomic tests.
- Tests on the cloud can be slower than local runs because of network round-trips and remote VM startup.
- The test is stuck in an endless loop, repeatedly sending commands.
### How to resolve
- Break the test into smaller pieces.
- Use the [maxDuration capability](https://testingbot.com/support/web-automate/selenium/test-options#maxduration) to raise the cap if you legitimately need a longer session.
- Check for endless loops and add explicit timeouts to waits.
## Test idle timeout
### What it means
TestingBot did not receive a new command from your Selenium or Appium script for more than 90 seconds (the default idle timeout). The session is terminated to prevent it from running forever and consuming all your minutes.
### Common causes
- A missing `driver.quit()` at the end of the test. The runner finishes but the session keeps the browser alive.
- The test script crashed, was interrupted, or lost internet connectivity.
- A legitimately slow step that takes more than 90 seconds between commands.
### How to resolve
- Ensure the test calls `driver.quit()` in a teardown or `finally` block so it always runs.
- Run the test from a stable network. CI runners with unstable egress are a common cause.
- Raise the timeout with the [idleTimeout capability](https://testingbot.com/support/web-automate/selenium/test-options#idletimeout) if you need more than 90 seconds between commands.
## New session cancelled before test start
### What it means
Your test runner requested a new session, but it closed the connection before TestingBot could allocate a slot and return a session id.
### Common causes
- Client connection timeout too short. iOS and Android simulators can take 60 to 120 seconds to boot.
- Running more tests than your plan's concurrency limit. Queued tests still consume the runner's connection time.
- Grid-side capacity issues during peak times. Check the [TestingBot status page](https://status.testingbot.com/).
### How to resolve
- Raise the connection timeout in your test runner. For WebdriverIO use `connectionRetryTimeout: 210000` (3.5 minutes). For Selenium HttpClient set the equivalent socket timeout.
- Confirm you are within the concurrent-session limit on your plan.
- Subscribe to the [status page RSS feed](https://status.testingbot.com/) for incident notifications.
## Manual session ended due to inactivity
### What it means
A Live (interactive) browser or device session was terminated because there was no mouse or keyboard input for an extended period. Live sessions are billed by the minute, so idle sessions are ended automatically to prevent runaway charges.
### How to resolve
Start a new Live session when you are ready to continue. If you need a long-running pinned session for QA workflows, consider switching to an automated test with a deliberate `maxDuration` setting.
## Capability not found or browser unavailable
### What it means
You will see a `CapabilityNotPresentOnTheGridException` (or a similar "browser not found" message) when the browser, version and platform combination you requested is not currently offered on the TestingBot grid.
### How to resolve
- Check the live [supported browser list](https://testingbot.com/support/web-automate/browsers) for available combinations.
- Use the version aliases `latest`, `latest-1`, `latest-2` when you do not need to pin to an exact version. TestingBot resolves them to the current top release.
- Double-check the spelling and case of `platformName` values. Selenium 4 W3C uses `WINDOWS`, `macOS`, `LINUX`, etc.
## Authentication errors (401)
### What it means
Your API key and secret were missing, malformed, or did not match the credentials on file. The grid rejects the session before any browser is allocated.
### How to resolve
- Verify your key and secret in [your account settings](https://testingbot.com/members/user/edit).
- Pass credentials inside `tb:options` (W3C or Selenium 4), not as top-level capabilities.
- If you have rotated keys, update every CI secret store that references the old pair.
## More help
Status incidents are posted at the [TestingBot status page](https://status.testingbot.com/) (RSS feed available). For anything not covered above, [contact our support team](https://testingbot.com/contact/new) with your session ID. The dashboard URL of the failed test is the fastest way for us to investigate.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/other/test-speed-optimization
# Test Speed Optimization
Every automated test session on TestingBot goes through the same phases: the session is requested, waits for a free slot if your account is at its concurrency limit, a browser or device is prepared (setup time), your commands run (execution time) and the session completes. This guide explains where that time goes and how to reduce it.
## See the duration breakdown for your tests
On every test detail page in the [member dashboard](https://testingbot.com/members/tests), hover over the **Duration** value to see a breakdown of the session: how much time was spent on setup versus execution, and which capabilities impacted the session speed. Each capability in the popover links to the matching section on this page.
## Optimize TestingBot Time
TestingBot Time is everything between requesting a session and its completion. It is the sum of two parts:
- **Setup time** : queue time plus preparing the browser, emulator, simulator or physical device, before your first command runs.
- **Execution time** : the time your test commands spend running on the browser or device.
### Reduce setup time
- **Stay within your concurrency limit.** Sessions started beyond your plan's parallel limit wait in a queue until a slot frees up. Spread test starts over time, or upgrade the number of parallel sessions if your suite regularly queues.
- **Prefer virtual devices for broad coverage.** Android emulators and iOS simulators are prepared faster than physical devices. Use virtual devices for functional coverage and reserve physical devices for final verification.
- **Only set the session capabilities you need.** Capabilities such as [timeZone](https://testingbot.com#timezone), [screen-resolution](https://testingbot.com#screen-resolution), [prerun](https://testingbot.com#prerun) and [firefox\_profile](https://testingbot.com#firefox-profile) each add work before the session is ready. The [capability sections below](https://testingbot.com#optimize-capabilities-impacting-session) explain the cost of each one.
- **Reuse uploaded apps.** For mobile app tests, upload your app once and reference it with its `tb://` app key instead of uploading the binary again for every session.
- **Set a realistic client connection timeout.** iOS simulators and Android emulators can take 60 to 120 seconds to boot. If your test runner gives up earlier, the session is cancelled and retried, which wastes the whole setup. See [new session cancelled before test start](https://testingbot.com/support/other/errors#session-cancelled).
### Reduce execution time
- **Use explicit waits instead of fixed sleeps.** A `sleep(5)` always costs 5 seconds; an explicit wait continues the moment the element is ready.
- **Keep tests atomic and run them in parallel.** Many short, independent tests finish faster than one long journey, because they spread across your parallel sessions and a failure does not invalidate unrelated steps.
- **Quit the session as soon as the test ends.** Always call `driver.quit()` in a teardown block. A session without it stays open until the [idle timeout](https://testingbot.com/support/web-automate/selenium/test-options#idletimeout) hits, adding up to 90 seconds per test.
- **Prefer fast selectors.** ID and CSS selectors are resolved faster than complex XPath expressions, especially on large pages.
- **Report status through the API.** Use [test annotations](https://testingbot.com/support/web-automate/selenium/annotating-tests) or the REST API to mark a test passed or failed instead of adding extra in-browser steps for reporting.
- **Avoid large downloads and uploads during the test.** Every file transferred through the browser session adds time. Keep fixtures small and mock heavy network responses where possible.
## Optimize Capabilities Impacting Session
Some capabilities add processing while the session is being prepared or while your commands run. The duration breakdown popover on the test detail page flags the ones detected in your session with a **high** , **medium** or **low** impact badge. Each section below explains the cost and how to reduce it.
### screenrecorder (high impact)
Video recording is enabled by default for every session. Encoding the video runs for the entire duration of the test and finalizing the file adds time when the session completes. If you do not need a recording, for example on large regression suites or consistently green builds, disable it with [`screenrecorder: false`](https://testingbot.com/support/web-automate/selenium/test-options#screenrecorder).
### tunnel (high impact)
With [TestingBot Tunnel](https://testingbot.com/support/tunnel), browser traffic is routed through the secure tunnel to your machine, so every request pays extra network latency. Start the tunnel once and share it across all tests in the run instead of starting one per test, run the tunnel process close to the servers under test, and only enable it for tests that genuinely need access to internal or staging environments.
### geoCountryCode (high impact)
[`geoCountryCode`](https://testingbot.com/support/web-automate/selenium/test-options#geo) routes all browser traffic through a proxy server in the selected country. Each request travels an extra round trip to that proxy. Only use it for tests that verify geolocation-specific behavior, and keep the rest of your suite on a direct connection.
### prerun (high impact)
A [`prerun`](https://testingbot.com/support/web-automate/selenium/test-options#prerun) executable is downloaded and executed on the test VM before your session starts, so its full runtime is added to setup time. Keep prerun scripts small and fast, and prefer building one-time configuration into the app or the test itself.
### firefox\_profile (high impact)
A custom [Firefox profile](https://testingbot.com/support/web-automate/selenium/firefox-profile) is uploaded and extracted on the VM during setup. The larger the profile, the longer the session takes to start. Keep the profile minimal: only the preferences and extensions the test actually needs.
### screenshot (medium impact)
With [`screenshot: true`](https://testingbot.com/support/web-automate/selenium/test-options#screenshot), TestingBot captures a screenshot during the test, which adds overhead per captured step. For large suites, disable automatic screenshots and take targeted screenshots from your test code only where they add debugging value.
### timeZone (medium impact)
Setting a custom [`timeZone`](https://testingbot.com/support/web-automate/selenium/test-options#timezones) reconfigures the operating system clock before the session starts. Only set it for tests that assert timezone-dependent behavior.
### deviceOrientation (medium impact)
Starting a mobile session in landscape requires rotating the device or emulator during setup. Use the default portrait orientation unless the test specifically covers landscape layouts, and rotate from within the test only when needed.
### realDevice (medium impact)
Physical devices are cleaned and prepared between sessions, which takes longer than booting an emulator or simulator. Use virtual devices for day-to-day functional coverage and schedule physical-device runs where real hardware matters: sensors, performance characteristics and vendor-specific behavior.
### screen-resolution (low impact)
Changing the [`screen-resolution`](https://testingbot.com/support/web-automate/selenium/test-options#screenresolution) switches the display mode of the VM during setup. Use the default resolution unless your test verifies a specific viewport, and prefer resizing the browser window from within the test when that is all you need.
### locale (low impact)
Custom `locale` or `language` values are configured on the device or emulator during setup. Only set them for localization tests.
---
URL: https://testingbot.com/support/other/status-badges
# Status Badges
Status Badges help you keep track of the status of your latest test build.
You can simply embed markdown code or HTML code to your project website to show the status of your most recent test build.
## Builds
In order to show a status badge, you need to make sure that you're running your tests in a group, called a **build**.
For example: you have a project that has a bunch of tests. After every commit you want to make sure the tests still pass.
The tests you want to run are grouped in a **build**.
To make sure your tests are grouped in the same build, use Selenium's Desired Capabilities.
Simple pass `"build" : "build-1XXX"` in your Desired Capabilities:
```java
DesiredCapabilities caps = new DesiredCapabilities();
caps.setCapability("platform", "WIN7");
caps.setCapability("browserName", "Chrome");
caps.setCapability("build", "build-1XXX"); // for example
```
In our [member area](https://testingbot.com/members) you will then see a new build, `build-1XXX`, which will contain all the tests that ran with the same build identifier.
## Marking a build as passed/failed
To mark a build as passed or failed, your test runner should be configured to send the success state of each test to TestingBot via the [TestingBot API](https://testingbot.com/support/api).
- If for every test in the build we know that the test succeeded, then we'll automatically mark the build as **Passing**.
- If a test failed, the build will automatically be marked as **Failed**.
- If you don't send us the success state for every test in the build, then we can not know the success state of the build. In this case we show an **Unknown** state.
## Status Badge
With our Status Badges, you can easily monitor the status of your most recent build.
The `auth` parameter is generated by concatenating your `api key` and `api secret` with md5.
Below are the different states that your badge can have:
Badge | Status || | **Passing** |
| | **Failure** |
| | **Unknown** We don't know if this build passed or failed. Please use our [REST-API](https://testingbot.com/support/api) to send the success state of your individual tests to TestingBot. |
| | **Error** The link to the badge might be invalid, or you might not have any builds yet. |
**Markdown:**
```markdown
[](https://testingbot.com/)
```
**HTML:**
```html
```
Not grouping your tests in builds yet? Open source projects can add the static **Tested with TestingBot** badge instead, which needs no setup. See [badges and logos for open source projects](https://testingbot.com/open-source#badges).
[Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#)
```ruby
Digest::MD5.hexdigest("#{client_key}:#{client_secret}")
```
```java
MessageDigest m=MessageDigest.getInstance("MD5");
String s = "TESTINGBOT_KEY:TESTINGBOT_SECRET";
m.update(s.getBytes(),0,s.length());
System.out.println("MD5: "+new BigInteger(1,m.digest()).toString(16));
```
```php
echo md5("TESTINGBOT_KEY:TESTINGBOT_SECRET");
```
```python
import hashlib
print(hashlib.md5("TESTINGBOT_KEY:TESTINGBOT_SECRET".encode('utf-8')).hexdigest())
```
```javascript
var crypto = require('crypto');
crypto.createHash('md5').update("TESTINGBOT_KEY:TESTINGBOT_SECRET").digest("hex");
```
```csharp
Convert.ToBase64String(new System.Security.Cryptography.MD5CryptoServiceProvider().ComputeHash(System.Text.Encoding.UTF8.GetBytes("TESTINGBOT_KEY:TESTINGBOT_SECRET")));
```
## Private Accounts
To display badges, you need to generate an **authentication token**. This token only unlocks the badge and build matrix images; it does not give access to your builds or tests, so the badge links to your builds in the member area.
To generate the authentication token, you must concatenate your TestingBot key, TestingBot secret and optionally an identifier. Finally, take the MD5 hash of the concatenated string:
[Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
```ruby
Digest::MD5.hexdigest("#{client_key}:#{client_secret}")
```
```java
MessageDigest m=MessageDigest.getInstance("MD5");
String s = "TESTINGBOT_KEY:TESTINGBOT_SECRET";
m.update(s.getBytes(),0,s.length());
System.out.println("MD5: "+new BigInteger(1,m.digest()).toString(16));
```
```php
echo md5("TESTINGBOT_KEY:TESTINGBOT_SECRET");
```
```python
import hashlib
print(hashlib.md5("TESTINGBOT_KEY:TESTINGBOT_SECRET".encode('utf-8')).hexdigest())
```
```javascript
var crypto = require('crypto');
crypto.createHash('md5').update("TESTINGBOT_KEY:TESTINGBOT_SECRET").digest("hex");
```
## Open Source Accounts
[Open Source](https://testingbot.com/open-source) accounts automatically have their tests and builds set to public. This means no authentication (auth parameter) is required to display badges or link to tests/builds.
## More
- If you like to show status-badges of **multiple projects** , please consider creating a new user account per project.
- We're also providing [browser-matrix images](https://testingbot.com/support/other/browser-matrix), displaying for which browsers your build passed/failed.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/other/browser-matrix
# Browser Matrix
With our Browser Matrix image you can quickly see the status for your latest test build for the various browsers/platforms.
You can simply add markdown code or HTML code to your project website to show the status of your most recent test build.
## Builds
In order to show a browser matrix image, you need to make sure that you're running your tests in a group, called a **build**.
For example: you have a project that has a bunch of tests. After every commit you want to make sure the tests still pass.
The tests you want to run are grouped in a **build**.
To make sure your tests are grouped in the same build, use Selenium's Desired Capabilities.
Simply pass `"build" : "build-1XXX"` in your Desired Capabilities:
```java
DesiredCapabilities caps = new DesiredCapabilities();
caps.setCapability("platformName", "WIN11");
caps.setCapability("browserName", "Chrome");
caps.setCapability("build", "build-1XXX"); // for example
```
In our [member area](https://testingbot.com/members/builds) you will then see a new build, `build-1XXX`, which will contain all the tests that ran with the same build identifier.
## Marking tests passed/failed
To mark tests as passed or failed, your test runner should be configured to send the success state of each test to TestingBot via the [TestingBot API](https://testingbot.com/support/api/tests#updatetest).
The browser matrix image will then show the success state for every browser/platform in your build.
## Browser Matrix
Below is an example of how the browser matrix image looks like, and information on how to embed this image on your website.
**Markdown:**
```markdown
[](https://testingbot.com/)
```
**HTML:**
```html
```
## Private Accounts
To display the matrix image, you need to generate an **authentication token**. This token only unlocks the badge and build matrix images; it does not give access to your builds or tests.
To generate the authentication token, you must concatenate your TestingBot key, TestingBot secret
and optionally an identifier. Finally, take the MD5 hash of the concatenated string:
[Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
```ruby
Digest::MD5.hexdigest("#{client_key}:#{client_secret}")
```
```java
MessageDigest m=MessageDigest.getInstance("MD5");
String s = "API_KEY:API_SECRET";
m.update(s.getBytes(),0,s.length());
System.out.println("MD5: "+new BigInteger(1,m.digest()).toString(16));
```
```php
echo md5("API_KEY:API_SECRET");
```
```python
import hashlib
print(hashlib.md5("API_KEY:API_SECRET".encode('utf-8')).hexdigest())
```
```javascript
var crypto = require('crypto');
crypto.createHash('md5').update("API_KEY:API_SECRET").digest("hex");
```
## Open Source Accounts
[Open Source](https://testingbot.com/open-source) accounts automatically have their tests and builds set to public. This means no authentication (auth parameter) is required to display matrix images or link to tests/builds.
## More
- If you like to show browser matrix images of **multiple projects** , please consider creating a new user account per project.
- We're also providing [status badges](https://testingbot.com/support/other/status-badges), displaying if your build passed/failed.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/tunnel
# Secure tunnel for local and staging tests
A free, open-source secure proxy that lets you run Selenium, Cypress, Playwright, Puppeteer and Appium tests against **localhost** , staging servers or any environment behind a firewall or VPN. No IP whitelisting required.
- v5.1
- Open Source
- Free for every plan
- Java 17+ or standalone
`java -jar testingbot-tunnel.jar TESTINGBOT_KEY TESTINGBOT_SECRET` bash
[Download v5.1](https://testingbot.com#downloads) [5-minute quickstart](https://testingbot.com/support/tunnel/quickstart) [View source on GitHub](https://github.com/testingbot/Testingbot-Tunnel)
- **Released:** 2026-10-27
- **License:** MIT
- **SHA-256 (zip):** `d44a19c0d3fe1d09beff043d32d04c805668147528ddd8af955d41d6a50a896e`
- **Maven:** `com.testingbot:TestingBotTunnel:5.1`
- **Docker:** `docker pull testingbot/tunnel:5.1`
- **Source:** [`github.com/testingbot/Testingbot-Tunnel`](https://github.com/testingbot/Testingbot-Tunnel)
## What is TestingBot Tunnel?
A Java application that proxies HTTP, HTTPS and WebSocket traffic between your network and the TestingBot browser and device grid.
_TestingBot Tunnel architecture: your client running Selenium, Playwright or Puppeteer connects through an SSH-encrypted tunnel to the TestingBot grid of browsers and devices, with optional access to your staging or internal network_
When the tunnel is running, every request your test makes to `localhost:4445/wd/hub` is forwarded over an SSH-encrypted channel to a dedicated VM in the TestingBot cloud. That VM then routes browser traffic back through the tunnel to your local server, so the cloud browser can reach pages on your laptop, your CI runner or your private staging environment without any firewall or IP-whitelisting changes.
At a glance
- **Encryption:** SSH
- **Protocols:** HTTP/1, /2, /3, WS, SSE
- **Outbound ports:** 443 + 22
- **Architecture:** Java 17, standalone builds
## Quickstart
Most teams have the tunnel running in under 5 minutes. The steps below use the JAR and assume Java 17+ is installed. No Java? Use a [self-contained build](https://testingbot.com#downloads). For Docker, Maven and the NodeJS launcher, see the [installation guide](https://testingbot.com/support/tunnel/installation).
01
Download the tunnel JAR
One file, no daemon, no service to install. The JAR runs everywhere Java 17+ runs.
```bash
curl -O https://testingbot.com/downloads/testingbot-tunnel.zip
unzip testingbot-tunnel.zip
```
02
Start the tunnel with your credentials
Wait for the message `You may start your tests`. The tunnel provisions a dedicated VM in the TestingBot cloud and opens an SSH-encrypted channel.
```bash
java -jar testingbot-tunnel.jar TESTINGBOT_KEY TESTINGBOT_SECRET
```
03
Point your tests at the local tunnel
That is the only change required. All Selenium, Appium, Playwright and Cypress capabilities work the same.
```diff
- https://hub.testingbot.com/wd/hub
+ http://localhost:4445/wd/hub
```
Need more detail? Read the [extended quickstart guide](https://testingbot.com/support/tunnel/quickstart) with language-specific examples.
## Downloads
Version 5.1, released 2026-10-27. The JAR needs Java 17 or newer. The self-contained builds bundle a trimmed Java runtime, so nothing has to be installed. Upgrading from 4.x? Read [what changes in 5.0](https://testingbot.com/support/tunnel/upgrading) first.
| Platform | Download | Size | SHA-256 |
| --- | --- | --- | --- |
| **Any OS with Java 17+** | [`testingbot-tunnel.zip`](https://testingbot.com/downloads/testingbot-tunnel.zip) | 7.3 MB | `d44a19c0d3fe1d09beff043d32d04c805668147528ddd8af955d41d6a50a896e` |
| **Linux x64** | [`testingbot-tunnel-linux-x64.tar.gz`](https://github.com/testingbot/Testingbot-Tunnel/releases/download/v5.1/testingbot-tunnel-linux-x64.tar.gz) | 31.1 MB | `5cb4753f9cd1fb2126b240dfdd1cd8967d1fadc71b155b0632ba9021b669ef05` |
| **Linux arm64** | [`testingbot-tunnel-linux-arm64.tar.gz`](https://github.com/testingbot/Testingbot-Tunnel/releases/download/v5.1/testingbot-tunnel-linux-arm64.tar.gz) | 30.9 MB | `0831415192c96c2dcbc25a36d45fd7e2bec6581b6f6777463dbae145d1d71b52` |
| **macOS Apple silicon** | [`testingbot-tunnel-macos-arm64.tar.gz`](https://github.com/testingbot/Testingbot-Tunnel/releases/download/v5.1/testingbot-tunnel-macos-arm64.tar.gz) | 27.4 MB | `48bd018fd204935be7e7090c821f381dd07052553822905c17833db0048f8c3b` |
| **macOS Intel** | [`testingbot-tunnel-macos-x64.tar.gz`](https://github.com/testingbot/Testingbot-Tunnel/releases/download/v5.1/testingbot-tunnel-macos-x64.tar.gz) | 28.1 MB | `a9cb077457dc73dc92c345acaa392d1aa4ae737e91791f237ec95003d03c8115` |
| **Windows x64** | [`testingbot-tunnel-windows-x64.zip`](https://github.com/testingbot/Testingbot-Tunnel/releases/download/v5.1/testingbot-tunnel-windows-x64.zip) | 29.0 MB | `01737338700fd3bc736f87a80637aa2cfc761a1a4012579ab3d2b54addd86935` |
Self-contained builds unpack to a folder with a `bin/testingbot-tunnel` launcher (`bin\testingbot-tunnel.cmd` on Windows) that takes the same options as the JAR. The macOS builds are signed and notarized.
```bash
tar -xzf testingbot-tunnel-macos-arm64.tar.gz
./testingbot-tunnel-macos-arm64/bin/testingbot-tunnel --version
```
Also available as
- Docker: `testingbot/tunnel:5.1`
- Maven Central: `com.testingbot:TestingBotTunnel:5.1`
- Every file with its `.sha256` on the [GitHub release](https://github.com/testingbot/Testingbot-Tunnel/releases/tag/v5.1)
## Key features
Fast
Static caching and compression keep round-trip latency low. [See monitoring](https://testingbot.com/support/tunnel/monitoring).
Secure
SSH-encrypted channel, dedicated ephemeral VM, automatic teardown. [Security model](https://testingbot.com/support/tunnel/security).
Robust
HTTP/1, /2, /3, WebSockets and SSE. Built for parallel workloads. [Troubleshoot](https://testingbot.com/support/tunnel/troubleshooting).
Simple
Single JAR, standalone binary or Docker image. Ready in under a minute. [Installation](https://testingbot.com/support/tunnel/installation).
## Documentation
Focused guides covering everything from your first test to production monitoring.
[Quickstart](https://testingbot.com/support/tunnel/quickstart)
Run your first test through the tunnel in 5 minutes.
[Installation](https://testingbot.com/support/tunnel/installation)
JAR, standalone builds, Docker, Maven, NodeJS.
[Upgrading to 5.0](https://testingbot.com/support/tunnel/upgrading)
Java 17, loopback listeners, quieter logs.
[Configuration](https://testingbot.com/support/tunnel/configuration)
Config files and TESTINGBOT\_\* variables.
[CLI reference](https://testingbot.com/support/tunnel/commandline)
Every flag, every environment variable.
[Examples](https://testingbot.com/support/tunnel/examples)
Headers, identifiers, basic auth, custom DNS.
[Multiple tunnels](https://testingbot.com/support/tunnel/multiple)
Parallel test setups using tunnel identifiers.
[Upstream proxy](https://testingbot.com/support/tunnel/upstream-proxy)
HTTP and SOCKS5 proxies, PAC files.
[Authentication](https://testingbot.com/support/tunnel/authentication)
Basic auth and Kerberos for sites and proxies.
[Traffic rules](https://testingbot.com/support/tunnel/traffic-rules)
Allow-lists, fast-fail, localhost policy, DNS.
[SSL bumping](https://testingbot.com/support/tunnel/ssl-bumping)
Certificates, --nobump and corporate CAs.
[Header rewriting](https://testingbot.com/support/tunnel/headers)
Add, replace or remove HTTP headers.
[Security](https://testingbot.com/support/tunnel/security)
SSH encryption, dedicated VMs, certificates.
[Health checks](https://testingbot.com/support/tunnel/health-checks)
/readyz, Docker and Kubernetes probes.
[Logging](https://testingbot.com/support/tunnel/logging)
HTTP logs, JSON output, request IDs.
[Monitoring](https://testingbot.com/support/tunnel/monitoring)
Prometheus metrics, Grafana dashboards.
[Embedding (Java)](https://testingbot.com/support/tunnel/embedding)
Start and stop the tunnel from your tests.
[Troubleshooting](https://testingbot.com/support/tunnel/troubleshooting)
Ports, SSL bumping, slow speeds, disconnects.
[FAQ](https://testingbot.com/support/tunnel/faq)
Common questions about ports, devices, WS.
[Changelog](https://testingbot.com/support/tunnel/changelog)
Release notes and version history.
[Source on GitHub](https://github.com/testingbot/Testingbot-Tunnel)
Open-source project, MIT licensed.
## System requirements
- Java 17+, or none with a standalone build
- RAM 4 GB minimum
- CPU 2 cores
- OS Windows / macOS / Linux / BSD
Outbound only
The tunnel needs outbound port **443** (HTTPS) and **22** (SSH) to `*.testingbot.com`. No inbound ports required.
For heavy parallel testing, scale CPU and RAM with concurrent sessions, or run [multiple tunnels](https://testingbot.com/support/tunnel/multiple).
## Graceful shutdown
Stop the tunnel cleanly so the cloud can deregister your VM and free your tunnel slot.
Option 1
Press `Ctrl+C`
In the terminal that runs the tunnel.
Option 2
Delete the PID file
`~/.testingbot-tunnel.pid`
Option 3
Send a signal
`kill -SIGUSR1 `
## Stopping a tunnel
Stop a running tunnel from your terminal, the dashboard or the API, for example when a CI job exited without cleaning up.
CLI
Press `Ctrl+C`
Or use any of the [graceful shutdown](https://testingbot.com#shutdown) options above.
Dashboard
Tunnels page, **Stop** button
Open [/members/tunnels](https://testingbot.com/members/tunnels) and click Stop on the tunnel's row.
API
`DELETE /v1/tunnel/{id}`
Get the id from `GET /v1/tunnel/list`. See the [API reference](https://testingbot.com/support/api#apitunneldelete).
Who can stop a tunnel
On a team, every member can see all of the team's tunnels. A tunnel can be stopped by:
- the person who started it (the tunnel's **Owner** column),
- the team **owner** ,
- any team member with the **Admin** role.
The same rule applies to the dashboard and the API. Other members see the tunnel but no Stop button.
## Tunnel clients
Launch and stop the tunnel programmatically from your test runner or CI pipeline.
Two official launchers handle the tunnel lifecycle so you do not have to manage the JAR yourself:
- **NodeJS:** [testingbot-tunnel-launcher](https://github.com/testingbot/testingbot-tunnel-launcher) — start and stop the tunnel from any npm-based test runner.
- **GitHub Actions:** [testingbot-tunnel-action](https://github.com/testingbot/testingbot-tunnel-action) — drop-in workflow step that boots the tunnel before your tests and tears it down after. See the [GitHub Actions guide](https://testingbot.com/support/integrations/ci-cd/github-actions).
The tunnel also ships as a Docker image and as a Maven dependency. See the [installation guide](https://testingbot.com/support/tunnel/installation) for all options.
[](https://testingbot.com/assets/tunnel-diagram.png "How the TestingBot Tunnel works")
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/tunnel/quickstart
# Your first tunneled test in 5 minutes
This guide gets a Selenium, Cypress, Playwright, Puppeteer or Appium test running against your local development server. If you have not already, [create a free TestingBot account](https://testingbot.com/users/sign_up) before you start.
_TestingBot Tunnel architecture: your client running Selenium, Playwright or Puppeteer connects through an SSH-encrypted tunnel to the TestingBot grid of browsers and devices, with optional access to your staging or internal network_
## Prerequisites
- JAVA 17 or higher. Check with `java -version`. No Java? Use a [standalone build](https://testingbot.com/support/tunnel/installation#standalone) or [Docker](https://testingbot.com#docker).
- ACCOUNT A TestingBot account. The free trial is enough.
- CREDENTIALS Your TestingBot key and secret from [account settings](https://testingbot.com/members/user/edit).
- LOCAL SERVER Anything that responds on a local port, e.g. `localhost:3000`.
## Download the tunnel
Download the latest release and unzip it. Place `testingbot-tunnel.jar` anywhere on your `PATH`, or run it from the folder you unzipped it in.
```bash
curl -O https://testingbot.com/downloads/testingbot-tunnel.zip
unzip testingbot-tunnel.zip
```
Prefer Docker or Maven? See the [installation guide](https://testingbot.com/support/tunnel/installation) for every install method.
## Start the tunnel
Pass your TestingBot key and secret. You can also export them as environment variables so they never appear in your shell history.
```bash
java -jar testingbot-tunnel.jar TESTINGBOT_KEY TESTINGBOT_SECRET
```
You will see log output as the tunnel provisions a dedicated VM in the TestingBot cloud and opens an SSH-encrypted channel. After 30 to 60 seconds, you will see:
```log
You may start your tests
```
The tunnel is now listening on `localhost:4445` for your Selenium and Appium clients.
## Run your first test
Change the hub URL in your test runner from the public TestingBot hub to your local tunnel endpoint. That is the only change required.
```diff
- https://hub.testingbot.com/wd/hub
+ http://localhost:4445/wd/hub
```
Run your tests as you normally would, and the cloud browser will reach your `localhost` via the tunnel.
## Language examples
The same first test in five common languages. Each one points the WebDriver at the local tunnel hub.
[Java](https://testingbot.com#) [Python](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Ruby](https://testingbot.com#) [C#](https://testingbot.com#)
```java
DesiredCapabilities caps = new DesiredCapabilities();
caps.setCapability("browserName", "chrome");
caps.setCapability("platformName", "WIN11");
WebDriver driver = new RemoteWebDriver(
new URL("http://KEY:SECRET@localhost:4445/wd/hub"),
caps);
driver.get("http://localhost:3000");
System.out.println(driver.getTitle());
driver.quit();
```
```python
from selenium import webdriver
options = webdriver.ChromeOptions()
options.set_capability("platformName", "WIN11")
driver = webdriver.Remote(
command_executor="http://KEY:SECRET@localhost:4445/wd/hub",
options=options,
)
driver.get("http://localhost:3000")
print(driver.title)
driver.quit()
```
```javascript
const { remote } = require("webdriverio");
const browser = await remote({
hostname: "localhost",
port: 4445,
path: "/wd/hub",
user: "KEY",
key: "SECRET",
capabilities: { browserName: "chrome", platformName: "WIN11" },
});
await browser.url("http://localhost:3000");
console.log(await browser.getTitle());
await browser.deleteSession();
```
```ruby
require "selenium-webdriver"
caps = Selenium::WebDriver::Remote::Capabilities.new
caps["browserName"] = "chrome"
caps["platformName"] = "WIN11"
driver = Selenium::WebDriver.for(:remote,
url: "http://KEY:SECRET@localhost:4445/wd/hub",
capabilities: caps)
driver.navigate.to "http://localhost:3000"
puts driver.title
driver.quit
```
```csharp
var options = new ChromeOptions();
options.AddAdditionalCapability("platformName", "WIN11", true);
IWebDriver driver = new RemoteWebDriver(
new Uri("http://KEY:SECRET@localhost:4445/wd/hub"),
options);
driver.Navigate().GoToUrl("http://localhost:3000");
Console.WriteLine(driver.Title);
driver.Quit();
```
## Run with Docker
Prefer not to install Java locally? The official `testingbot/tunnel` image on Docker Hub bundles everything you need. It supports both `amd64` and `arm64` and is the easiest way to run the tunnel from CI or a containerised dev environment.
1. **Pull the image** from Docker Hub.
2. **Start the tunnel** , passing your TestingBot key and secret as environment variables.
3. **Point your tests** at `http://localhost:4445/wd/hub`, exactly the same as the JAR version.
```bash
docker pull testingbot/tunnel:5.1
docker run --rm -it \
--network host \
-e TESTINGBOT_BIND_ADDRESS=127.0.0.1 \
-e TESTINGBOT_KEY=YOUR_KEY \
-e TESTINGBOT_SECRET=YOUR_SECRET \
testingbot/tunnel:5.1
```
`--network host` only works fully on Linux. On macOS and Windows, publish the ports explicitly, on the loopback address, so your tests can reach the tunnel on `localhost:4445` without exposing it to your network:
```bash
docker run --rm -it \
-p 127.0.0.1:4445:4445 -p 127.0.0.1:8087:8087 -p 127.0.0.1:8003:8003 \
-e TESTINGBOT_KEY=$TESTINGBOT_KEY \
-e TESTINGBOT_SECRET=$TESTINGBOT_SECRET \
testingbot/tunnel:5.1
```
Once the container prints `You may start your tests`, the tunnel is ready. Stop it gracefully with `Ctrl+C`. For Compose, Maven and the NodeJS launcher, see the [installation guide](https://testingbot.com/support/tunnel/installation).
## Next steps
[Multiple tunnels](https://testingbot.com/support/tunnel/multiple)
Add identifiers to run more than one tunnel at a time.
[Upstream proxy](https://testingbot.com/support/tunnel/upstream-proxy)
Forward through a corporate or GeoIP proxy.
[Monitoring](https://testingbot.com/support/tunnel/monitoring)
Configure Prometheus and Grafana.
[Security model](https://testingbot.com/support/tunnel/security)
Review SSH encryption and credential handling.
[Troubleshooting](https://testingbot.com/support/tunnel/troubleshooting)
If something goes wrong, this is where to look.
[FAQ](https://testingbot.com/support/tunnel/faq)
Ports, devices, WebSockets and more.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/tunnel/installation
# Install the tunnel
Pick the flavour that fits your environment best. The single-file JAR works everywhere with Java 17+. The standalone builds for Linux, macOS and Windows need no Java at all. Docker is the easiest way to run the tunnel in CI. The Maven artifact and NodeJS launcher let you embed the tunnel in your existing build tooling.
## Before you install
- JAVA 17 or higher for the JAR. Verify with `java -version`. Not needed for the [standalone builds](https://testingbot.com#standalone) or Docker.
- NETWORK Outbound access to `*.testingbot.com` on ports 443 and 22.
- CREDENTIALS A TestingBot key and secret from [account settings](https://testingbot.com/members/user/edit).
## JAR
One file, no daemon, no service to install. Runs on any OS with Java 17 or newer.
```bash
curl -O https://testingbot.com/downloads/testingbot-tunnel.zip
unzip testingbot-tunnel.zip
java -jar testingbot-tunnel.jar TESTINGBOT_KEY TESTINGBOT_SECRET
```
Verify the download checksum before running:
```bash
shasum -a 256 testingbot-tunnel.jar
# expected: 171fb4012ed2375313c5b6a63c305fa1c6e33e0371c3eb4c7304ecee935492f8
```
## Standalone builds
Self-contained archives for Linux, macOS and Windows that bundle a trimmed Java runtime. Unpack and run, nothing else to install. Each archive has a `.sha256` file next to it on the [GitHub release](https://github.com/testingbot/Testingbot-Tunnel/releases/tag/v5.1).
Linux
[x64](https://github.com/testingbot/Testingbot-Tunnel/releases/download/v5.1/testingbot-tunnel-linux-x64.tar.gz) · [arm64](https://github.com/testingbot/Testingbot-Tunnel/releases/download/v5.1/testingbot-tunnel-linux-arm64.tar.gz)
macOS
[Apple silicon](https://github.com/testingbot/Testingbot-Tunnel/releases/download/v5.1/testingbot-tunnel-macos-arm64.tar.gz) · [Intel](https://github.com/testingbot/Testingbot-Tunnel/releases/download/v5.1/testingbot-tunnel-macos-x64.tar.gz). Signed and notarized.
Windows
[x64](https://github.com/testingbot/Testingbot-Tunnel/releases/download/v5.1/testingbot-tunnel-windows-x64.zip)
Linux and macOS:
```bash
curl -LO https://github.com/testingbot/Testingbot-Tunnel/releases/download/v5.1/testingbot-tunnel-linux-x64.tar.gz
curl -LO https://github.com/testingbot/Testingbot-Tunnel/releases/download/v5.1/testingbot-tunnel-linux-x64.tar.gz.sha256
sha256sum -c testingbot-tunnel-linux-x64.tar.gz.sha256 # macOS: shasum -a 256 -c
tar -xzf testingbot-tunnel-linux-x64.tar.gz
./testingbot-tunnel-linux-x64/bin/testingbot-tunnel TESTINGBOT_KEY TESTINGBOT_SECRET
```
Windows (PowerShell):
```powershell
Invoke-WebRequest -OutFile testingbot-tunnel-windows-x64.zip https://github.com/testingbot/Testingbot-Tunnel/releases/download/v5.1/testingbot-tunnel-windows-x64.zip
Expand-Archive testingbot-tunnel-windows-x64.zip -DestinationPath .
.\testingbot-tunnel-windows-x64\bin\testingbot-tunnel.cmd TESTINGBOT_KEY TESTINGBOT_SECRET
```
The launcher takes exactly the same options as the JAR, so every example in these docs works by replacing `java -jar testingbot-tunnel.jar` with `bin/testingbot-tunnel`.
## Docker
The official image is published as `testingbot/tunnel` on Docker Hub and supports both amd64 and arm64.
```bash
docker pull testingbot/tunnel:5.1
docker run --rm -it \
--network host \
-e TESTINGBOT_BIND_ADDRESS=127.0.0.1 \
-e TESTINGBOT_KEY=your_key \
-e TESTINGBOT_SECRET=your_secret \
testingbot/tunnel:5.1
```
The image listens on every interface inside the container, so that published ports work. With `--network host` the container shares your machine's interfaces, which is why the command above sets `TESTINGBOT_BIND_ADDRESS=127.0.0.1`: the Selenium relay carries your credentials and should not be reachable from your network.
`--network host` only works fully on Linux. On macOS and Windows, publish the ports explicitly, on the loopback address:
```bash
docker run --rm -it \
-p 127.0.0.1:4445:4445 -p 127.0.0.1:8087:8087 -p 127.0.0.1:8003:8003 \
-e TESTINGBOT_KEY=$TESTINGBOT_KEY \
-e TESTINGBOT_SECRET=$TESTINGBOT_SECRET \
testingbot/tunnel:5.1
```
Almost every tunnel option can be passed as a `TESTINGBOT_*` environment variable, see [configuration](https://testingbot.com/support/tunnel/configuration#env). The image has a built-in health check, see [health checks](https://testingbot.com/support/tunnel/health-checks#docker).
## Maven
Add the tunnel as a dependency to your `pom.xml` and start it from a JUnit or TestNG fixture. See [embedding](https://testingbot.com/support/tunnel/embedding) for the API.
```xml
com.testingbotTestingBotTunnel5.1
```
## NodeJS launcher
For npm-based projects, use the `testingbot-tunnel-launcher` package to start and stop the tunnel programmatically. The launcher handles downloading the right JAR for your platform.
```bash
npm install --save-dev testingbot-tunnel-launcher
```
```javascript
const testingbotTunnel = require("testingbot-tunnel-launcher");
testingbotTunnel({
apiKey: process.env.TESTINGBOT_KEY,
apiSecret: process.env.TESTINGBOT_SECRET,
}, (err, tunnel) => {
if (err) throw err;
// your tests here
tunnel.close(() => console.log("tunnel closed"));
});
```
Source on GitHub: [testingbot-tunnel-launcher](https://github.com/testingbot/testingbot-tunnel-launcher).
## GitHub Action
If your tests run in GitHub Actions, the official `testingbot/testingbot-tunnel-action` handles the entire tunnel lifecycle for you. It pulls the latest Docker image, starts the tunnel before your test step, waits for the ready signal, and tears it down when the workflow ends.
Add it as a step in your workflow, before the step that runs your tests:
```yaml
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: testingbot/testingbot-tunnel-action@v1
with:
key: ${{ secrets.TB_KEY }}
secret: ${{ secrets.TB_SECRET }}
tunnelIdentifier: github-action-tunnel
- name: Run tests
run: npm test
env:
TB_KEY: ${{ secrets.TB_KEY }}
TB_SECRET: ${{ secrets.TB_SECRET }}
```
The action supports every tunnel flag as a workflow input: `tunnelIdentifier`, `noBump`, `noCache`, `proxy`, `extraHeaders`, `fastFailRegexps` and more. It also uploads the tunnel log as a workflow artifact by default for easy debugging.
Full reference and example workflows: [GitHub Actions integration guide](https://testingbot.com/support/integrations/ci-cd/github-actions). Source: [testingbot-tunnel-action](https://github.com/testingbot/testingbot-tunnel-action).
## Sanity check with --doctor
Once the tunnel is installed, run the built-in health check. It verifies that the required local ports are free, that outbound connectivity to TestingBot works, and that your credentials are valid.
```bash
java -jar testingbot-tunnel.jar --doctor
```
Pass the same options you normally start the tunnel with: `--doctor` checks the configuration it is given, including upstream proxies, Kerberos and any `--cacert-file`.
If all checks pass, continue with the [quickstart](https://testingbot.com/support/tunnel/quickstart). If anything fails, the [troubleshooting guide](https://testingbot.com/support/tunnel/troubleshooting) covers the most common causes.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/tunnel/examples
# Configuration recipes
Practical recipes for the most common TestingBot Tunnel configurations. Start with the basic setup, then add identifiers, headers, proxies and basic auth as your tests need them.
## Getting started
Download the tunnel and start it from the command line.
```bash
java -jar testingbot-tunnel.jar key secret
```
Then update your test configuration to route traffic through the tunnel by changing the hub URL:
```diff
- urlhub = "https://key:secret@hub.testingbot.com/wd/hub"
+ urlhub = "http://key:secret@localhost:4445/wd/hub"
```
By connecting to `localhost:4445`, all test commands are routed through the tunnel to TestingBot.
## Tunnel identifiers
Run multiple tunnels simultaneously and direct specific tests to a particular tunnel.
```bash
java -jar testingbot-tunnel.jar key secret -i myTunnel
```
Then specify the identifier in your test's desired capabilities:
```json
{
"browserName": "firefox",
"browserVersion": "latest",
"platformName": "WIN11",
"tunnel-identifier": "myTunnel"
}
```
See [running multiple tunnels](https://testingbot.com/support/tunnel/multiple) for parallel and high-availability setups.
## Custom headers
Inject custom HTTP headers into every request the tunnel makes. Useful for authentication tokens, feature flags, A/B test cookies and debugging headers.
```bash
java -jar testingbot-tunnel.jar key secret \
--extra-headers '{"MyOwnHeader": "HeaderValue"}'
```
## Upstream proxy
Route tunnel traffic through an upstream proxy server. Common use cases:
- Corporate networks that require all outbound traffic to pass through a proxy.
- Geo-location testing using HTTP proxies in different regions.
Use `-Y` for the proxy address and `-z` for credentials:
```bash
java -jar testingbot-tunnel.jar key secret -Y myproxy:9000 -z username:password
```
For PAC files and more advanced setups, see the [upstream proxy guide](https://testingbot.com/support/tunnel/upstream-proxy).
## Shared tunnels
Tunnels are private by default. Enable the `--shared` flag to make a tunnel available to other team members on the same TestingBot account.
```bash
java -jar testingbot-tunnel.jar key secret --shared
```
Team members can then run their tests against the shared tunnel without starting their own. Combine with `--tunnel-identifier` to give the shared tunnel a memorable name.
## Basic authentication
Automatically supply credentials for websites that require HTTP Basic Authentication. The tunnel injects these credentials for matching hosts, so your tests do not have to handle authentication dialogs.
Use the `-a` flag with the format `host:port:username:password`. For standard HTTP sites use port 80:
```bash
java -jar testingbot-tunnel.jar key secret -a mywebsite.com:80:myusername:mypassword
```
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/tunnel/multiple
# More than one tunnel, in parallel
Your account supports up to **2** simultaneous tunnels. They can be tunnels you start yourself or shared tunnels from team members on the same account.
## Why use multiple tunnels?
Parallel
Higher throughput
Spread test traffic across tunnels for higher throughput.
HA
High availability
Run tunnels on different machines for redundancy.
Isolate
Environment isolation
Dedicate one tunnel per environment (staging, QA, dev).
Team
Team workflows
Let team members run tunnels for their own tests without conflicts.
## Setting up tunnel identifiers
Use the `--tunnel-identifier` flag to give each tunnel a unique name.
```bash
java -jar testingbot-tunnel.jar key secret --tunnel-identifier myTunnel
```
Then specify the identifier in the desired capabilities of each test that should be routed through that tunnel:
```json
{
"browserName": "firefox",
"browserVersion": "latest",
"platformName": "WIN11",
"tunnel-identifier": "myTunnel"
}
```
## Parallel test isolation
When two test suites both rely on the tunnel (for example a staging suite and a QA suite), give each suite its own tunnel and identifier. This prevents one suite from saturating the other's bandwidth and makes monitoring per-suite traffic straightforward via the [metrics endpoint](https://testingbot.com/support/tunnel/monitoring).
## High availability
Run identically-configured tunnels on two or more machines using the same identifier. The TestingBot grid load-balances test traffic across all available tunnels with that identifier and routes around any tunnel that becomes unhealthy.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/tunnel/upstream-proxy
# Route through an upstream proxy
On many networks the only route out is a proxy. The tunnel can send its own connection to TestingBot, and the traffic it carries for your tests, through an upstream HTTP or SOCKS5 proxy. It can also use a different proxy per destination. Common uses are corporate proxies and GeoIP proxies for region-specific testing.
- HTTP
- SOCKS5
- PAC
- Kerberos
## Basic configuration
Pass the proxy with `--proxy`, and credentials with `--proxy-userpwd`.
```bash
java -jar testingbot-tunnel.jar --proxy --proxy-userpwd :
```
| Form | Proxy type |
| --- | --- |
| host:port | HTTP proxy |
| http://host:port | HTTP proxy |
| socks5://host:port | SOCKS5 proxy, with optional username and password |
Unless you set [`--proxy-testingbot`](https://testingbot.com#split), the proxy carries everything: the browser's requests to your sites, HTTPS `CONNECT` tunnels, WebSocket upgrades, and the tunnel's own connection to TestingBot.
## Corporate proxies
Forward both the control channel to `*.testingbot.com` and the test traffic through your authenticated proxy.
Export `TESTINGBOT_PROXY_USERPWD` instead of passing `-z` on the command line so the credentials never appear in the process list.
```bash
export TESTINGBOT_PROXY_USERPWD=alice:hunter2
java -jar testingbot-tunnel.jar --proxy corp-proxy.example.com:8080
```
Proxy needs Kerberos instead of a password? Add `--proxy-auth-scheme negotiate`. See [Kerberos proxies](https://testingbot.com/support/tunnel/authentication#kerberos-proxy).
## A separate proxy for TestingBot
Where egress is filtered by destination, the proxy allowed out to the internet is often not the one that reaches your internal test targets. `--proxy-testingbot` gives TestingBot's own traffic, the API and the tunnel's SSH connection, a different proxy from the one test traffic uses.
```bash
java -jar testingbot-tunnel.jar \
--proxy 10.0.0.9:8080 \
--proxy-testingbot corp-egress:3128 --proxy-testingbot-userpwd user:password
```
Credentials are deliberately not shared: `--proxy-userpwd` belongs to one proxy operator and is not sent to another. Give each proxy its own.
## Routing with a PAC file
`--pac-local` points the tunnel at a proxy auto-config file and lets it decide, per destination, whether to go direct or through a proxy. This is how your browser already handles a corporate network, applied to the tunnel.
```bash
java -jar testingbot-tunnel.jar --pac-local /etc/corp.pac
java -jar testingbot-tunnel.jar --pac-local https://corp.example/proxy.pac
```
Check what a file decides for a URL before relying on it:
```bash
java -jar testingbot-tunnel.jar --pac-local corp.pac --pac-test https://internal.corp/page
```
- **Use `https://` or a local file.** A PAC file decides where traffic goes. Fetched over plain `http://`, it is whatever the network says it is, so a plain `http://` URL is refused unless you pin its contents with `--pac-local-sha256`. Redirects are refused rather than followed.
- **A purpose-built interpreter, no JavaScript engine.** It supports what PAC files use: functions, variables, conditionals, loops, the usual operators, and the standard helpers (`isPlainHostName`, `dnsDomainIs`, `shExpMatch`, `isInNet`, `dnsResolve`, `myIpAddress` and the date and time predicates).
- **Unsupported syntax is reported with its line number** , rather than guessed at. Object literals, regular expressions and `new` are not supported. A misread PAC file would silently send traffic to the wrong place.
- **No credentials for PAC-chosen proxies.** A proxy named only by the PAC file does not receive `--proxy-userpwd`. The tunnel logs that it withheld them.
```bash
java -jar testingbot-tunnel.jar --pac-local http://wpad.corp/proxy.pac \
--pac-local-sha256 3b1f...64-hex-characters
```
## PAC for the remote browser
`--pac` is a different option: it tells the **cloud browser** which PAC URL to use. It does not change where the tunnel itself sends traffic.
```bash
java -jar testingbot-tunnel.jar --pac https://corp.example.com/proxy.pac
```
| Option | Evaluated by | Decides |
| --- | --- | --- |
| --pac-local | The tunnel, on your machine | Where the tunnel sends its own outgoing traffic |
| --pac | The remote browser | Which proxy the cloud browser uses |
## TLS-intercepting proxies
If the proxy intercepts TLS and re-signs it with an internal certificate authority, Java will not trust it and the tunnel cannot register itself. Point it at the authority.
```bash
java -jar testingbot-tunnel.jar --proxy proxy.corp:8080 --cacert-file /etc/ssl/corp-ca.pem
```
The certificate is added to the platform's trust store rather than replacing it. `--doctor` prints the subjects it loaded. See [SSL bumping and certificates](https://testingbot.com/support/tunnel/ssl-bumping#cacert).
## WebSockets through a proxy
`--ws-proxy-mode` controls how a `ws://` upgrade crosses `--proxy`.
| Mode | Behaviour |
| --- | --- |
| connect | **Default.** Ask the proxy to open a tunnel with `CONNECT` and upgrade inside it. This is what RFC 6455 specifies and what browsers do. |
| get | Send the upgrade to the proxy as an ordinary request. For proxies that forward `Upgrade` but only allow `CONNECT` to port 443. |
`wss://` is always a `CONNECT`, so this option does not affect it.
## Things to know
- Run `--doctor` with your proxy options. It tests the route out and, for Kerberos, every step of the Negotiate chain.
- A failing proxy shows up in the cloud browser with an `X-TestingBot-Error` of `upstream-proxy-unreachable`, `upstream-proxy-auth-failed` or `upstream-proxy-refused`. See [reading errors](https://testingbot.com/support/tunnel/traffic-rules#errors).
- WebSocket and SSE traffic requires `--nobump --nocache` in addition to the proxy flags.
- See the [CLI reference](https://testingbot.com/support/tunnel/commandline#flags-proxy) for every proxy-related flag.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/tunnel/security
# Built for production traffic
Every tunnel session runs inside its own ephemeral virtual machine, all traffic is encrypted in transit, and credentials never have to appear on the command line.
- SSH encrypted
- Dedicated VM
- Ephemeral
- MIT licensed
## Encrypted channel
The tunnel opens a single outbound SSH connection from your machine to a TestingBot edge endpoint. All test traffic (HTTP, HTTPS, WebSocket, SSE) is multiplexed over that SSH connection.
Inbound ports on your network are not required. The tunnel only initiates outbound connections to ports **443** (HTTPS) and **22** (SSH) on `*.testingbot.com`.
## Pinning the tunnel server's host key
The tunnel's control connection is SSH, and your account secret is its password. Whatever answers on the tunnel port receives it, so the tunnel can verify the server's SSH host key before it sends anything.
```bash
java -jar testingbot-tunnel.jar --ssh-host-key SHA256:xxxx... --ssh-host-key-policy require
```
- `--ssh-host-key` pins the SHA-256 fingerprint the server must present, in the form `ssh-keygen -lf -E sha256` prints. Repeat it, or comma-separate several, where the server may present more than one key. MD5 fingerprints are refused.
- Where the TestingBot API has a fingerprint for your tunnel server, it sends it over its authenticated HTTPS connection and the tunnel checks it. A pin you give outranks it.
- With neither, the tunnel connects anyway and logs a warning on every connect, which is what 4.x tunnels did. `--ssh-host-key-policy require` refuses to connect instead.
- `--doctor` reports which of these cases applies before anything connects.
## Local listeners bind loopback
The Selenium relay, local proxy, metrics port and `--web` server do not authenticate. Since 5.0 they listen on `127.0.0.1` only.
That matters because the Selenium relay attaches your TestingBot key and secret to everything it forwards, and the local proxy will connect anywhere your machine can. If tests run on another machine, set `--bind-address 0.0.0.0` and restrict the ports with a firewall. See [upgrading](https://testingbot.com/support/tunnel/upgrading#bind).
## Limiting what the tunnel can reach
A tunnel can reach whatever the machine running it can reach. On a CI runner or jump host, narrow that to what your tests need.
```bash
java -jar testingbot-tunnel.jar --allow-hosts 'staging.example.com,*.internal.example' --localhost-policy deny
```
Both are enforced on plain HTTP, `CONNECT` and WebSocket upgrades. The localhost check is applied to the address actually dialled, so a DNS answer pointing at `127.0.0.1` cannot get around it. See [traffic rules](https://testingbot.com/support/tunnel/traffic-rules).
## Dedicated VM lifecycle
Every tunnel session provisions a dedicated virtual machine in the TestingBot cloud. The VM exists only for the lifetime of your tunnel.
01
Provision
A fresh VM is allocated when your tunnel starts.
02
Use
Only your account (or your team if `--shared`) can route traffic.
03
Teardown
The VM is destroyed when the tunnel shuts down. No state persists.
VMs are not reused between accounts. There is no cross-tenant traffic on the tunnel VM.
## Certificates and SSL bumping
By default, the tunnel performs SSL interception (also called SSL bumping) for outbound HTTPS so that the cloud browser can validate certificates seamlessly. The TestingBot CA used for bumping is preinstalled on every cloud VM and device.
For apps that pin certificates, you have two options:
Trust the TB CA
[Download the certificate](https://testingbot.com/downloads/testingbot-ssl-bumping.crt) and add it to your app's trust store.
Disable bumping
Start the tunnel with `--nobump` and HTTPS is forwarded transparently with no decryption.
## Credential handling
Your TestingBot key and secret authenticate the tunnel against the TestingBot grid. Treat them like passwords.
The tunnel accepts credentials from three sources, in order of precedence:
1. Positional arguments on the command line.
2. Environment variables: `TESTINGBOT_KEY` and `TESTINGBOT_SECRET`.
3. A `.testingbot` file in your `$HOME` directory.
For CI pipelines and shared machines, prefer the environment-variable approach so credentials never appear in shell history, process lists or build logs. See the [CLI reference](https://testingbot.com/support/tunnel/commandline#env) for the full list.
## Credentials stay out of logs
Tunnel logs end up in CI artifacts and support tickets, so credentials are redacted before anything is written.
- Credential-bearing headers such as `Authorization`, `Proxy-Authorization` and cookies.
- The `user:password@` part of URLs and credential-shaped query values.
- Capabilities logged with `--log-http forwarder:body`, by key name.
- Upstream proxy credentials are only ever sent to the proxy they were issued for, never to a proxy named by a PAC file. See [authentication](https://testingbot.com/support/tunnel/authentication#scoping).
## Metrics endpoint
The tunnel exposes a Prometheus-compatible metrics endpoint on port 8003 by default. The endpoint is unauthenticated unless you set `--metrics-auth user:password` (or `TESTINGBOT_METRICS_AUTH`).
Since 5.0 the endpoint binds `127.0.0.1` by default. If you set `--bind-address 0.0.0.0`, enable metrics auth and restrict the port with a firewall rule. The `/healthz` and `/readyz` endpoints are not behind metrics auth, and disclose only up or down. See the [monitoring guide](https://testingbot.com/support/tunnel/monitoring).
## Data residency
Test traffic flowing through the tunnel is routed via TestingBot edge endpoints in the EU. No traffic is stored after the tunnel ends. For specific data-residency or compliance information, please see the [TestingBot trust center](https://trust.testingbot.com).
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/tunnel/monitoring
# Watch your tunnel from Grafana
Production tunnels are easier to operate when you can see their health at a glance. TestingBot Tunnel ships with a built-in Prometheus-compatible metrics endpoint, so you can scrape it from any monitoring stack you already run.
- Prometheus
- Grafana ready
- Port 8003
## Enable the metrics endpoint
The metrics endpoint is enabled by default on `http://localhost:8003`. Override the port with `--metrics-port`.
```bash
java -jar testingbot-tunnel.jar --metrics-port 9100
```
If the tunnel host is reachable from the network (for example a shared CI runner), protect the endpoint with HTTP Basic Auth:
```bash
java -jar testingbot-tunnel.jar --metrics-auth ops:s3cret
```
Export `TESTINGBOT_METRICS_AUTH` instead of passing credentials on the command line. See the [security guide](https://testingbot.com/support/tunnel/security#metrics).
Since 5.0 the metrics port binds `127.0.0.1`. For Prometheus on another host, start the tunnel with `--bind-address 0.0.0.0`. The same port also serves `/healthz` and `/readyz` for liveness and readiness checks, see [health checks](https://testingbot.com/support/tunnel/health-checks).
## Available metrics
Tunnel-specific series are prefixed with `testingbot_`. The endpoint also exposes the standard JVM and process metrics from the Prometheus Java client. The full label set is always documented at the `/metrics` endpoint itself.
### Tunnel state
| Metric | Type | Meaning |
| --- | --- | --- |
| testingbot\_tunnel\_up | gauge | `1` when the tunnel is connected, `0` while reconnecting or down. The single most important alerting signal. |
| testingbot\_tunnel\_info | info | Static labels with the tunnel build (`version`, `id`, `name`). Use for dashboard headers and version filters. |
| testingbot\_tunnel\_uptime\_seconds | counter | Seconds since this tunnel process started. Drops on restart, useful for detecting flapping. |
| testingbot\_tunnel\_reconnects\_total | counter | Total tunnel reconnects since startup. Sustained increases indicate an unstable upstream link. |
| testingbot\_active\_connections | gauge | Number of in-flight client connections through the tunnel right now. |
| testingbot\_tunnel\_connect\_duration\_seconds | histogram | Time taken to establish the tunnel itself (cold-start latency). Buckets `_bucket`, `_sum`, `_count`. |
### HTTP traffic
| Metric | Type | Meaning |
| --- | --- | --- |
| testingbot\_http\_requests\_total | counter | HTTP requests proxied since startup. Labels: `method`, `code`. Use `rate()` for throughput, filter on `code=~"5.."` for errors. |
| testingbot\_http\_request\_duration\_seconds | histogram | End-to-end HTTP latency. Compute p50/p95/p99 with `histogram_quantile()`. |
| testingbot\_https\_connect\_total | counter | HTTPS CONNECT sessions established. Labels: `code`. |
| testingbot\_https\_connect\_duration\_seconds | histogram | CONNECT handshake latency, suitable for `histogram_quantile()`. |
| testingbot\_https\_connect\_errors\_total | counter | CONNECT errors. Label: `reason` (TLS, target unreachable, timeout, ...). |
| testingbot\_proxy\_bytes\_transferred\_total | counter | Total bytes proxied (both directions). Use `rate()` for throughput in B/s. |
| testingbot\_errors\_total | counter | Generic proxy errors. Label: `name` (the error class). Useful for alerting on burst increases. |
## JVM and process metrics
The endpoint also exposes the standard `jvm_*` and `process_*` series from the Prometheus Java client. These help you size the tunnel host, detect memory pressure and catch garbage-collection pauses.
| Metric | Type | Meaning |
| --- | --- | --- |
| jvm\_memory\_bytes\_used | gauge | Bytes of heap and non-heap memory in use. Label: `area` (`heap` or `nonheap`). |
| jvm\_memory\_bytes\_max | gauge | Maximum bytes available per memory area. Pair with `_used` to compute headroom. |
| jvm\_memory\_pool\_bytes\_used | gauge | Per-pool memory usage (Eden, Survivor, Old Gen, Metaspace, ...). Label: `pool`. |
| jvm\_gc\_collection\_seconds\_count | counter | Number of GC collections. Label: `gc` (collector name). |
| jvm\_gc\_collection\_seconds\_sum | counter | Total time spent in GC. Use `rate()` to see GC pressure over time. |
| jvm\_threads\_current | gauge | Live thread count. Watch for runaway growth. |
| jvm\_threads\_daemon | gauge | Daemon thread count. |
| jvm\_threads\_peak | gauge | Peak thread count since process start. |
| jvm\_classes\_loaded | gauge | Currently loaded classes. |
| jvm\_buffer\_pool\_used\_bytes | gauge | Direct and mapped NIO buffer usage. Label: `pool`. |
| process\_cpu\_seconds\_total | counter | Total CPU time consumed by the process. Use `rate()` for CPU utilisation. |
| process\_resident\_memory\_bytes | gauge | RSS memory of the tunnel process as reported by the OS. |
| process\_open\_fds | gauge | Open file descriptors. Compare with `process_max_fds` to spot leaks. |
| process\_start\_time\_seconds | gauge | Unix epoch when the process started. Useful for restart detection. |
All names above are the canonical Prometheus Java client names. Open `http://localhost:8003/metrics` to see the full live list and their `# HELP` / `# TYPE` annotations.
## Prometheus scrape config
Add a job to your `prometheus.yml` that targets the tunnel host.
```yaml
scrape_configs:
- job_name: testingbot_tunnel
static_configs:
- targets: ["tunnel-host.example.com:8003"]
basic_auth:
username: ops
password_file: /etc/prometheus/testingbot_metrics_password
```
## Grafana dashboard
A ready-made Grafana dashboard ships with the tunnel source. Import it once and you have a full Overview / HTTP / Tunnel-health view in seconds.
Download `testingbot-tunnel.json` from the [grafana-dashboard examples folder](https://github.com/testingbot/Testingbot-Tunnel/tree/master/examples/grafana-dashboard) on GitHub. In Grafana go to **Dashboards → New → Import** , paste the JSON or upload the file, pick your Prometheus data source, and click **Import**.
[ ](https://testingbot.com/assets/support/tunnel/grafana-dashboard-bf173520c136608330e3e59c0444883d41815ea3bd031cf063f3147f3ef65bf2.png)
_The bundled dashboard rendered against a live tunnel_
The bundled dashboard groups panels into two rows:
- **Overview:** Tunnel status (UP/DOWN), build version, active connections, uptime and reconnect counter.
- **HTTP:** request rate by status class, HTTPS CONNECT rate, p50/p95/p99 latency and response throughput.
Want a turnkey setup? The repo also includes a [docker-compose example](https://github.com/testingbot/Testingbot-Tunnel/tree/master/examples/docker-compose-prometheus-grafana) that spins up Prometheus + Grafana already wired to scrape a local tunnel and load the dashboard automatically. Useful for local development or a single-machine CI runner.
```bash
git clone https://github.com/testingbot/Testingbot-Tunnel.git
cd Testingbot-Tunnel/examples/docker-compose-prometheus-grafana
docker compose up -d
# Grafana on http://localhost:3000 · Prometheus on http://localhost:9090
```
If you prefer to build the dashboard yourself, here are the most useful PromQL queries:
Tunnel up
max(testingbot\_tunnel\_up)
Active connections
sum(testingbot\_active\_connections)
2xx request rate
rate(testingbot\_http\_requests\_total{code=~"2.."}[5m])
5xx error rate
rate(testingbot\_http\_requests\_total{code=~"5.."}[5m])
p95 HTTP latency
histogram\_quantile(0.95, sum by (le) (rate(testingbot\_http\_request\_duration\_seconds\_bucket[5m])))
Bytes throughput
rate(testingbot\_proxy\_bytes\_transferred\_total[5m])
Reconnects
sum(testingbot\_tunnel\_reconnects\_total)
JVM heap usage
sum by (area) (jvm\_memory\_bytes\_used)
Process CPU
rate(process\_cpu\_seconds\_total[5m])
Threads
jvm\_threads\_current
## Alert ideas
- **Tunnel restart:** fire when `testingbot_uptime_seconds` drops to under 60 seconds.
- **Error spike:** fire when the 5-minute error rate exceeds your normal baseline.
- **Stalled connections:** fire when `testingbot_connections` stays at zero during normal CI hours.
- **Endpoint unreachable:** fire on Prometheus `up == 0` for the tunnel target.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/tunnel/upgrading
# Upgrading from 4.x to 5.0
Tunnel 5.0 adds 28 options and removes none, so your existing command lines keep working. Six things do change. The first three are the ones most people notice.
- v5.0
- No options removed
- Java 17
## What changes
| Area | 4.x | 5.0 | To keep 4.x behaviour |
| --- | --- | --- | --- |
| **Java** | 11 | **17** | None: hard requirement, or use a [self-contained build](https://testingbot.com/support/tunnel/installation#standalone) |
| **What listeners bind** | every interface | `127.0.0.1` | `--bind-address 0.0.0.0` |
| **Per-request logging** | one line per request | only failures and 5xx | `--log-http url` |
| **Selenium relay logging** | always logged | honours `--log-http` | `--log-http forwarder:url` |
| **Docker image** | n/a | sets `TESTINGBOT_BIND_ADDRESS=0.0.0.0` | None: published ports keep working |
| **Embedding the JAR** | Jetty 11 + Servlet API | Jetty 12 core handlers | Source change, see [below](https://testingbot.com#embedding) |
## Java 17 is required
Tunnel 5.0 is built on Jetty 12, which needs Java 17. The JAR's classes cannot be loaded by an older JVM.
On an older Java the tunnel now says so plainly, instead of failing with `A JNI error has occurred`. Check what you have:
```bash
java -version
```
If you cannot upgrade Java on the machine, download the [self-contained build](https://testingbot.com/support/tunnel/installation#standalone) for Linux, macOS or Windows. It bundles a trimmed Java runtime, so nothing has to be installed. The [Docker image](https://testingbot.com/support/tunnel/installation#docker) already ships Java 17.
## Listeners bind loopback
In 4.x, the Selenium relay (`4445`), the local proxy (`8087`), the metrics port (`8003`) and `--web` (`8080`) accepted connections from any machine that could route to yours.
None of those listeners authenticates. The Selenium relay attaches your TestingBot key and secret to everything it forwards, and the local proxy will connect anywhere your machine can, including its own loopback. In 5.0 they bind `127.0.0.1`.
Same machine
Nothing changes
Tests that connect to `localhost:4445` on the tunnel machine work as before. This is the normal setup.
Different machine
Add `--bind-address 0.0.0.0`
Then restrict who can reach the ports with a firewall rule.
```bash
java -jar testingbot-tunnel.jar --bind-address 0.0.0.0
```
Docker
The image sets `TESTINGBOT_BIND_ADDRESS=0.0.0.0` for you, because a loopback bind inside a container makes published ports unreachable. Narrow access on the host side of the publish instead: `-p 127.0.0.1:4445:4445`.
## Successful requests no longer log
`--log-http` now defaults to `errors`: the request line and headers are logged only for failed or 5xx responses. The Selenium relay follows the same setting, where it previously logged every request.
```bash
# one line per request, as in 4.x
java -jar testingbot-tunnel.jar --log-http url
# browser traffic quiet, Selenium relay verbose
java -jar testingbot-tunnel.jar --log-http proxy:none,forwarder:url
```
See the [logging guide](https://testingbot.com/support/tunnel/logging) for per-module levels and JSON output.
## Readiness is stricter
`/readyz` reports ready only after the startup self-test passes, and after a reconnect only once both forwarding directions are back.
A half-working tunnel that used to look healthy now returns `503` until it can actually carry traffic. A self-test that failed for a transient reason is retried while the SSH connection stays up. If your orchestration waits on `/readyz`, nothing needs to change, it just becomes more accurate. See [health checks](https://testingbot.com/support/tunnel/health-checks).
## Embedding the JAR
Only relevant if you start the tunnel from your own Java code rather than as a process.
5.0 is built on Jetty 12's core Handler API with no Servlet layer, so code that reached into the tunnel's Jetty internals needs a source change. The supported surface is `boot()`, `awaitReady(Duration)` and `stop()`, described in the [embedding guide](https://testingbot.com/support/tunnel/embedding).
## Upgrade checklist
1. Install Java 17+, or switch to a [self-contained build](https://testingbot.com/support/tunnel/installation#standalone) or `testingbot/tunnel:5.0`.
2. If tests connect from another host, add `--bind-address 0.0.0.0` and firewall the ports.
3. If you parse the tunnel log per request, add `--log-http url`.
4. Bump Maven to `com.testingbot:TestingBotTunnel:5.0` if you embed the tunnel.
5. Run `--doctor` with your usual flags. In 5.0 it diagnoses the configuration it is actually given.
```bash
java -jar testingbot-tunnel.jar --doctor
```
Then look at what 5.0 adds: [traffic rules](https://testingbot.com/support/tunnel/traffic-rules), [config files](https://testingbot.com/support/tunnel/configuration), [Kerberos proxies](https://testingbot.com/support/tunnel/authentication) and [SSH host key pinning](https://testingbot.com/support/tunnel/security#host-key). The full list is in the [changelog](https://testingbot.com/support/tunnel/changelog).
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/tunnel/configuration
# Flags, files and environment
Tunnel options can be set three ways: on the command line, in a `--config` file, or as a `TESTINGBOT_*` environment variable. That lets a container or CI job be configured without assembling a long command line.
- Since v5.0
## Three ways, one order
When an option is set in more than one place, the **command line** wins, then the **config file** , then the **environment**.
```bash
# these three are equivalent
java -jar testingbot-tunnel.jar --se-port 4446
echo "se-port = 4446" > tunnel.conf
java -jar testingbot-tunnel.jar --config tunnel.conf
TESTINGBOT_SE_PORT=4446 java -jar testingbot-tunnel.jar
```
The environment comes last because it is the most ambient of the three, and the least likely to be what someone meant when they also typed a flag or wrote a file.
## Config file
A properties file. Keys are the long option names without the leading dashes. Lines starting with `#` are comments.
```properties
# testingbot-tunnel.conf
# Run with: java -jar testingbot-tunnel.jar --config testingbot-tunnel.conf
tunnel-identifier = staging
se-port = 4445
metrics-port = 8003
# Upstream proxy: host:port (HTTP), http://host:port or socks5://host:port
proxy = corp-proxy.internal:8080
# Only reach these hosts; everything else gets 403
allow-hosts = staging.example.com,*.internal.example
# Domains to refuse (comma separated regular expressions).
# Backslashes must be doubled in this file.
fast-fail-regexps = .*\\.doubleclick\\.net,.*google-analytics\\.com
# Boolean options: true to enable, omit or false to leave off
shared = true
nobump = false
```
- Boolean options accept `true`, `yes`, `on` or `1`. Anything else, including `false`, leaves the option off.
- A boolean set to `false` in the file is not overridden by the environment, so `shared = false` stays off even if `TESTINGBOT_SHARED=true` is set.
- The file is read as a Java properties file, so a backslash is an escape character. Write a regular expression's `\.` as `\\.`, or it reaches the tunnel as a bare `.`.
- An unknown key stops the tunnel at startup rather than being ignored, so a typo does not silently drop a setting.
- Anything given explicitly on the command line still wins over the file.
- Run `--help` to list every option, or see the [CLI reference](https://testingbot.com/support/tunnel/commandline).
When to use a file
A file is the easiest way to manage long `--allow-hosts` or `--fast-fail-regexps` lists, to keep a tunnel's setup in version control, and to share one setup between developers. Keep credentials out of it: use the environment for those.
## Environment variables
Long options have a `TESTINGBOT_*` alias, derived from its name: upper-case it, replace dashes with underscores and prefix `TESTINGBOT_`. The few exceptions are listed below the table.
| Option | Environment variable |
| --- | --- |
| --se-port | `TESTINGBOT_SE_PORT` |
| --tunnel-identifier | `TESTINGBOT_TUNNEL_IDENTIFIER` |
| --allow-hosts | `TESTINGBOT_ALLOW_HOSTS` |
| --bind-address | `TESTINGBOT_BIND_ADDRESS` |
| --log-http | `TESTINGBOT_LOG_HTTP` |
| --shared | `TESTINGBOT_SHARED=true` |
A few options have no environment alias because they make no sense on every run: `--help`, `--version`, `--doctor`, `--ready` and `--config`. `TESTINGBOT_AUTH` is special: it takes a comma-separated list of `host:port:user:password` entries, one per `--auth`.
This is what makes the [Docker image](https://testingbot.com/support/tunnel/installation#docker) configurable from a compose file or a Kubernetes manifest:
```bash
docker run --rm \
-e TESTINGBOT_KEY=... -e TESTINGBOT_SECRET=... \
-e TESTINGBOT_TUNNEL_IDENTIFIER=ci-$BUILD_ID \
-e TESTINGBOT_ALLOW_HOSTS=staging.example.com \
testingbot/tunnel:5.1
```
## Where the tunnel reads your key and secret
- Positional arguments: `java -jar testingbot-tunnel.jar KEY SECRET`
- `client-key` and `client-secret` in a `--config` file
- The `TESTINGBOT_KEY` and `TESTINGBOT_SECRET` environment variables
- A `~/.testingbot` file containing `key:secret`
Get your key and secret from your [account page](https://testingbot.com/members/user/edit).
## Keeping credentials out of the process list
Anything on the command line is visible to other users of the machine in `ps`. Options that carry a secret are better set in the environment.
| Option | Environment variable | Format |
| --- | --- | --- |
| KEY SECRET | `TESTINGBOT_KEY`, `TESTINGBOT_SECRET` | your account credentials |
| --auth | `TESTINGBOT_AUTH` | `host:port:user:password`, comma-separated for several |
| --proxy-userpwd | `TESTINGBOT_PROXY_USERPWD` | `user:password` |
| --proxy-testingbot-userpwd | `TESTINGBOT_PROXY_TESTINGBOT_USERPWD` | `user:password` |
| --metrics-auth | `TESTINGBOT_METRICS_AUTH` | `user:password` |
Positional credentials
The positional `KEY SECRET` form is still supported, but it puts your secret in `ps` for as long as the tunnel runs. On a shared machine or CI runner, prefer `TESTINGBOT_KEY` / `TESTINGBOT_SECRET` or `~/.testingbot`. A flag always wins over the matching variable.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/tunnel/authentication
# Credentials the tunnel presents for you
Some of what a test needs to reach sits behind a login prompt, and on corporate networks the route out does too. The tunnel can authenticate on your test's behalf, to **sites** and to **upstream proxies** , with HTTP Basic or Kerberos.
- Basic
- Kerberos / SPNEGO
| | Basic | Kerberos (Negotiate) |
| --- | --- | --- |
| **Sites** your test opens | `--auth` | `--krb5-hosts` |
| **Upstream proxy** for test traffic | `--proxy-userpwd` | `--proxy-auth-scheme negotiate` |
| **Proxy to TestingBot** | `--proxy-testingbot-userpwd` | `--proxy-auth-scheme negotiate` |
## Basic auth for protected sites
Staging sites often sit behind HTTP Basic authentication. Rather than putting credentials in every test URL, let the tunnel answer the prompt.
```bash
java -jar testingbot-tunnel.jar -a staging.example.com:80:myuser:mypassword
```
- The format is `host:port:user:password`. Pass several entries for several sites.
- To keep the password out of `ps`, use `TESTINGBOT_AUTH` instead, with entries comma-separated.
- This works on plain HTTP. HTTPS reaches the site as an encrypted `CONNECT` tunnel that the local tunnel cannot add headers to.
```bash
export TESTINGBOT_AUTH=staging.example.com:80:myuser:mypassword,admin.example.com:80:ops:secret
java -jar testingbot-tunnel.jar
```
## Basic auth for upstream proxies
```bash
export TESTINGBOT_PROXY_USERPWD=alice:hunter2
java -jar testingbot-tunnel.jar --proxy proxy.corp:8080
```
The same credentials are used for SOCKS5 proxies (`--proxy socks5://host:port`). If TestingBot's own traffic leaves through a different proxy, give that one its own credentials with `--proxy-testingbot-userpwd`; see [where credentials go](https://testingbot.com#scoping).
## Kerberos proxies (SPNEGO / Negotiate)
Many corporate proxies require Kerberos rather than a username and password. The tunnel speaks Negotiate on every path out: browser traffic, HTTPS CONNECT and its own connection to TestingBot.
```bash
kinit alice@CORP.EXAMPLE
java -jar testingbot-tunnel.jar --proxy proxy.corp:8080 --proxy-auth-scheme negotiate
```
| Option | Purpose |
| --- | --- |
| --proxy-auth-scheme \ | How to authenticate to the upstream proxy. Default `basic`. |
| --proxy-spn \ | Service principal of the proxy. Defaults to `HTTP/`. Accepts `HTTP/host` or `HTTP@host`. |
| --krb5-keytab \ | Keytab to log in from, for machines where nobody runs `kinit`. Requires `--krb5-principal`. |
| --krb5-principal \ | Principal to log in as with the keytab, for example `svc-tunnel@CORP.EXAMPLE`. |
Without a keytab, credentials come from your existing Kerberos ticket cache.
## Unattended Kerberos with a keytab
On a build agent or in a container, nobody is there to run `kinit`. Give the tunnel a keytab for a service account instead.
```bash
java -jar testingbot-tunnel.jar \
--proxy proxy.corp:8080 \
--proxy-auth-scheme negotiate \
--krb5-keytab /etc/tunnel/svc-tunnel.keytab \
--krb5-principal svc-tunnel@CORP.EXAMPLE
```
Treat the keytab like a password: it can obtain tickets as that principal.
## Kerberos for intranet sites
`--krb5-hosts` sends the same Kerberos credentials to _sites_, for intranet applications that log users in with Kerberos.
```bash
java -jar testingbot-tunnel.jar --krb5-hosts intranet.corp.example,wiki.corp.example
```
- It is an explicit list and never a wildcard. A service ticket names the user, so every host that may receive one has to be written down.
- Empty by default, so no site receives Kerberos credentials unless you name it.
- Plain HTTP only. HTTPS reaches the target as an opaque `CONNECT` tunnel.
## Where credentials go
Proxy credentials are only ever sent to the proxy they were issued for, matched on protocol, host and port.
- `--proxy-userpwd` goes to `--proxy` and nowhere else.
- `--proxy-testingbot` does **not** inherit `--proxy-userpwd`. The two proxies usually belong to different operators, so give it its own `--proxy-testingbot-userpwd`.
- A proxy chosen by a [PAC file](https://testingbot.com/support/tunnel/upstream-proxy#pac-local) receives no credentials. The tunnel logs that it withheld them, so the resulting `407` explains itself.
```bash
java -jar testingbot-tunnel.jar \
--proxy 10.0.0.9:8080 --proxy-userpwd testuser:testpass \
--proxy-testingbot corp-egress:3128 --proxy-testingbot-userpwd egress:secret
```
## Checking a setup
Every Negotiate misconfiguration looks like the same `407` from the proxy. `--doctor` walks the whole chain and reports exactly which step fails.
```bash
java -jar testingbot-tunnel.jar --proxy proxy.corp:8080 --proxy-auth-scheme negotiate --doctor
```
Run it with the same options you start the tunnel with. It checks the configuration it is actually given.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/tunnel/traffic-rules
# Decide what the tunnel may reach
A tunnel can reach whatever the machine running it can reach. These options narrow that down to what your tests actually need, and steer requests to a different destination without touching your tests.
- Since v5.0
- HTTP, CONNECT and WebSocket
Enforced on every way out
`--allow-hosts`, `--fast-fail-regexps` and `--localhost-policy` apply to plain HTTP requests, HTTPS `CONNECT` tunnels and WebSocket upgrades alike. A refused request gets a `403` that says which rule refused it.
## Allow only named hosts
`--allow-hosts` names what the tunnel is _for_. Everything else is refused.
```bash
java -jar testingbot-tunnel.jar --allow-hosts 'staging.example.com,*.internal.example'
```
- An entry is an exact host or `*.suffix` for its subdomains.
- `*.internal.example` covers `app.internal.example` but not `internal.example` itself. List the apex separately if you need it.
- Omit the option and any host is reachable, as in 4.x.
This is the safer of the two approaches. A deny list only stops destinations somebody thought of in advance. An allow list keeps a tunnel on a CI runner from becoming a route into everything else on that network.
## Fast-fail domains
`--fast-fail-regexps` refuses requests to hosts matching any of a comma-separated list of regular expressions. The browser gets an immediate answer instead of waiting on a request that would go through the tunnel.
```bash
# skip analytics and ad networks to speed up page loads
java -jar testingbot-tunnel.jar -F '.*google-analytics\.com,.*doubleclick\.net'
```
**New in 5.0:** prefix an entry with `!` to make an exception. This blocks everything except `ok.com` and its subdomains:
```bash
java -jar testingbot-tunnel.jar -F '.*,!ok\.com'
```
An exception has to cover a whole host or subdomain, so it cannot be satisfied by a name that merely ends with it: `ok.com.attacker.net` is _not_ excepted by `!ok\.com`.
Which one to use
To speed tests up by skipping third-party noise, use `--fast-fail-regexps`. To limit what the tunnel may reach, use `--allow-hosts`. You can combine them: a host has to pass both.
## Localhost policy
`--localhost-policy deny` refuses tunnel traffic that would reach this machine's own loopback interface.
```bash
java -jar testingbot-tunnel.jar --localhost-policy deny
```
Use it for a tunnel that is meant to reach a staging network, and nothing running on the machine itself, such as a shared CI runner or a jump host. The check is applied to the address actually dialled, not only to the name in the URL, so a DNS answer that points a public-looking name at `127.0.0.1` cannot walk past it.
The default is `allow`, which is what you want when testing a development server on your laptop. The Selenium relay is not affected. For how cloud browsers reach your local server, see [Can I use localhost?](https://testingbot.com/support/tunnel/faq#localhost)
## Remap destinations with --connect-to
Dial a different host or port than the request asks for, while leaving the URL, the `Host` header and the TLS SNI name untouched. The format matches curl's `--connect-to`: `HOST1:PORT1:HOST2:PORT2`.
```bash
# test https://www.example.com against the staging instance
java -jar testingbot-tunnel.jar --connect-to 'www.example.com:443:staging-lb.internal:443'
# send every request on port 80 to a local dev server on 3000
java -jar testingbot-tunnel.jar --connect-to ':80:127.0.0.1:3000'
```
- An empty `HOST1` or `PORT1` matches anything. An empty `HOST2` or `PORT2` leaves that half of the destination unchanged.
- Comma-separate several rules. The first rule that matches wins.
- Bracket IPv6 literals: `example.com:443:[::1]:8443`.
This lets a test drive the production hostname against another instance, with the same virtual host and certificate name the real site uses. Editing `/etc/hosts` is the usual alternative, but it needs root, affects every process on the machine and cannot change a port.
## Custom DNS
`--dns` resolves hostnames through servers you name instead of the system resolver. Useful for split-horizon setups where the machine's default resolver does not know internal names.
```bash
java -jar testingbot-tunnel.jar --dns 10.0.0.53,10.0.0.54:5353
```
| Option | Effect |
| --- | --- |
| --dns \ | Host or `host:port`, comma separated. The first is primary and the rest are tried in order when it does not answer. Falls back to the system resolver if none can. |
| --dns-round-robin | Spread queries across the servers instead of treating the first as primary, for a pool whose members are equal. |
| --dns-timeout \ | How long to wait for each server. Default `5`. |
## Connection timeouts
| Option | Default | Effect |
| --- | --- | --- |
| --http-dial-timeout \ | 5 for the API, 15 for tunnelled connections | How long to wait for a TCP connection to a target or upstream proxy. Lower it where unreachable hosts should fail fast. Also bounds the SSH control connection. |
| --http-idle-timeout \ | 120 for HTTP, 300 for CONNECT | How long a connection may sit with no traffic before it is closed. Raise it for long-lived idle connections such as long polling. |
## Reading refusals and errors
When the tunnel cannot or will not complete a request, it says why three ways: the HTTP status, an `X-TestingBot-Error` header, and a one-line explanation in the body that shows up in the cloud browser's window.
| X-TestingBot-Error | Status | Meaning |
| --- | --- | --- |
| not-allowed | 403 | The host is not in `--allow-hosts`. |
| denied-by-fast-fail | 403 | Blocked by `--fast-fail-regexps`. |
| denied-localhost | 403 | Blocked by `--localhost-policy deny`. |
| dns-error | 502 | The hostname could not be resolved from the tunnel machine. |
| connection-refused | 502 | Nothing is listening on that port. |
| connection-timeout | 504 | Timed out connecting to the target. |
| host-unreachable | 502 | No network route to the target from the tunnel machine. |
| tls-error | 502 | The TLS handshake with the target failed. |
| upstream-proxy-\* | 502 | The [upstream proxy](https://testingbot.com/support/tunnel/upstream-proxy) was unreachable, rejected the credentials, or refused the destination. |
| loop-detected | 508 | The request targets the tunnel's own proxy port. |
The same reason string is the `reason` label on the proxy error counter in [Prometheus metrics](https://testingbot.com/support/tunnel/monitoring), so a spike on a dashboard and a header in a bug report name the same thing.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/tunnel/ssl-bumping
# HTTPS through the tunnel
Two different certificate questions come up with a tunnel. Does the **cloud browser** trust the certificate your site presents? And does the **tunnel itself** trust the network it runs on? This page covers both.
## What SSL bumping does
By default, HTTPS traffic through the tunnel is intercepted and re-signed with the TestingBot CA. This is called SSL bumping.
Every TestingBot VM and device already trusts the TestingBot CA. Because the cloud browser sees a certificate it trusts, staging sites with self-signed or expired certificates load without certificate warnings, with no changes to your servers or tests.
Bumping on
Default
Self-signed, expired and internal-CA certificates just work.
Bumping off
`--nobump`
HTTPS is relayed as opaque bytes. The browser sees your server's real certificate, which it has to trust on its own.
## When to disable bumping
```bash
java -jar testingbot-tunnel.jar --nobump
```
- **WebSockets and Server-Sent Events.** Streaming protocols need `--nobump` together with `--nocache`. See the [FAQ](https://testingbot.com/support/tunnel/faq#websockets).
- **Certificate pinning** in a mobile app, see below.
- **Mutual TLS** , where your server expects a client certificate that a re-signing proxy cannot present.
- **Testing the certificate itself** , for example HSTS, certificate transparency or an expiry warning.
`--nobump` applies to the whole tunnel. If only one environment needs it, run a separate tunnel for that environment with its own [tunnel identifier](https://testingbot.com/support/tunnel/multiple).
## Certificate pinning
An app that pins its server certificate will reject the re-signed certificate. You have two options.
Trust the TestingBot CA
[Download the certificate](https://testingbot.com/downloads/testingbot-ssl-bumping.crt) and add it to your test build's trust store or pin set.
Disable bumping
Start the tunnel with `--nobump` so the app sees your server's real certificate.
## Networks that intercept TLS
The other direction: some corporate proxies intercept outgoing TLS and re-sign it with an internal certificate authority. Java does not trust that authority, so the tunnel cannot even register itself with TestingBot.
Point the tunnel at the authority's certificate, in PEM format. Repeat the option for several.
```bash
java -jar testingbot-tunnel.jar --proxy proxy.corp:8080 --cacert-file /etc/ssl/corp-ca.pem
```
- The certificate is **added** to the platform's trust store, not used instead of it, so public sites keep working.
- `--doctor` prints the subjects of the certificates it loaded, so you can confirm it picked up the right one.
- This affects what the tunnel process trusts. It does not change what the cloud browser trusts.
More on corporate networks: [upstream proxies](https://testingbot.com/support/tunnel/upstream-proxy) and [proxy authentication](https://testingbot.com/support/tunnel/authentication).
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/tunnel/headers
# Edit headers on the way through
`--header` edits the headers the tunnel sends to your site. `--response-header` edits the headers your site sends back to the cloud browser. Both are repeatable, and neither needs a change to your application.
- Since v5.0
- Plain HTTP
## Rule grammar
Each `--header` or `--response-header` takes one rule. Repeat the option for more.
| Rule | Effect |
| --- | --- |
| name: value | Set the header, replacing whatever the peer sent. |
| name; | Set the header to an empty value. |
| -name | Remove the header. |
| -name\* | Remove every header whose name starts with `name`. |
## Request headers
Headers sent from the tunnel to your site.
```bash
# tell the backend this traffic comes from automated tests
java -jar testingbot-tunnel.jar --header 'X-Test-Run: nightly-42'
# remove every X-Forwarded-* header before it reaches your server
java -jar testingbot-tunnel.jar --header '-X-Forwarded*'
```
## Response headers
Headers sent from your site back to the cloud browser. Useful for dropping a policy that blocks a test page without changing the server.
```bash
# a strict CSP blocks a test helper script
java -jar testingbot-tunnel.jar --response-header '-Content-Security-Policy'
# stop a staging site's HSTS from sticking in the browser
java -jar testingbot-tunnel.jar --response-header '-Strict-Transport-Security'
# allow the page to be framed by a test harness
java -jar testingbot-tunnel.jar --response-header '-X-Frame-Options'
```
## --extra-headers
`--extra-headers` adds request headers from a JSON object. It predates `--header` and still works.
```bash
java -jar testingbot-tunnel.jar --extra-headers '{"X-Environment":"staging","X-Tenant":"acme"}'
```
For new setups, prefer `--header`: it can also replace and remove headers, and each rule is a separate option, which is easier to manage in a [config file](https://testingbot.com/support/tunnel/configuration) or CI variable.
## HTTPS and headers
Plain HTTP only
HTTPS arrives at the local tunnel as a `CONNECT` request and is relayed as opaque, encrypted bytes, so the local tunnel cannot see or change its headers. Header rules, `--extra-headers` and [`--auth`](https://testingbot.com/support/tunnel/authentication#basic-sites) apply to plain HTTP traffic.
If you need a header on an HTTPS site in a test, set it from the test itself, for example with your framework's request interception or a browser extension.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/tunnel/health-checks
# Know when the tunnel can carry traffic
Starting the tunnel process is not the same as the tunnel being able to forward requests. A remote tunnel server has to be provisioned, the SSH connection established and a self-test passed. Health checks let your CI job or orchestrator wait for exactly that moment, and notice when it is lost.
- Since v5.0
- Docker HEALTHCHECK
- Kubernetes probes
## Endpoints
Served on the metrics port, `8003` by default (change with `--metrics-port`).
| Path | Answers | Use it for |
| --- | --- | --- |
| /healthz | `200 {"status":"ok"}` whenever the process is answering, including mid-reconnect | Liveness: restart the tunnel only if this fails |
| /readyz | `200` when the tunnel is forwarding, `503` otherwise | Readiness: start tests, or route traffic, only when this is 200 |
| / | JSON status: version, uptime, requests, bytes transferred | Dashboards and quick inspection |
| /metrics | Prometheus exposition | [Monitoring](https://testingbot.com/support/tunnel/monitoring) |
```bash
curl -i http://127.0.0.1:8003/readyz
```
- `/readyz` turns 200 only after the startup self-test passes, and after a reconnect only once both forwarding directions are back.
- During a reconnect, `/healthz` stays 200 while `/readyz` returns 503. The tunnel is recovering by itself, so it should be taken out of rotation, not restarted.
- The health endpoints are deliberately not behind `--metrics-auth`. Container probes have no good way to carry credentials, and the endpoints disclose nothing beyond up or down.
- Like every local listener, the metrics port binds `127.0.0.1` unless you set `--bind-address 0.0.0.0`.
## The --ready command
Asks a running tunnel whether it is ready, then exits `0` (ready) or `1` (not ready, or nothing listening). No `curl` or `wget` needed.
```bash
java -jar testingbot-tunnel.jar --ready
echo $? # 0 when ready
# a tunnel started with --metrics-port 9003
java -jar testingbot-tunnel.jar --ready --metrics-port 9003
```
`--ready` queries `/readyz` on `--metrics-port`, so pass the same port you started the tunnel with.
## Waiting for the tunnel in CI
Start the tunnel in the background, then block until it is ready before running tests.
```bash
java -jar testingbot-tunnel.jar &
TUNNEL_PID=$!
for i in $(seq 1 60); do
java -jar testingbot-tunnel.jar --ready 2>/dev/null && break
sleep 2
done
java -jar testingbot-tunnel.jar --ready || { echo "tunnel not ready"; exit 1; }
npm test
kill $TUNNEL_PID
```
On GitHub Actions the [testingbot-tunnel-action](https://testingbot.com/support/integrations/ci-cd/github-actions) does this for you.
## Docker
The `testingbot/tunnel` image has a built-in `HEALTHCHECK` that runs `--ready`, so `docker ps` shows _healthy_ once the tunnel is actually forwarding.
```bash
docker run -d --name tb-tunnel \
-e TESTINGBOT_KEY=... -e TESTINGBOT_SECRET=... \
-p 127.0.0.1:4445:4445 \
testingbot/tunnel:5.1
docker inspect --format '{{.State.Health.Status}}' tb-tunnel
```
With Docker Compose, other services can wait for it:
```yaml
services:
tunnel:
image: testingbot/tunnel:5.1
environment:
TESTINGBOT_KEY: ${TESTINGBOT_KEY}
TESTINGBOT_SECRET: ${TESTINGBOT_SECRET}
tests:
build: .
environment:
SELENIUM_URL: http://tunnel:4445/wd/hub
depends_on:
tunnel:
condition: service_healthy
```
Changing the metrics port
The image's `HEALTHCHECK` assumes the default metrics port. If you set `--metrics-port` or `TESTINGBOT_METRICS_PORT`, override the healthcheck to match.
## Kubernetes
Use `/healthz` as the liveness probe and `/readyz` as the readiness probe. Tunnel options are available as `TESTINGBOT_*` environment variables, so no command line is needed.
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: testingbot-tunnel
spec:
replicas: 1
selector:
matchLabels: { app: testingbot-tunnel }
template:
metadata:
labels: { app: testingbot-tunnel }
spec:
containers:
- name: tunnel
image: testingbot/tunnel:5.1
env:
- name: TESTINGBOT_KEY
valueFrom: { secretKeyRef: { name: testingbot, key: key } }
- name: TESTINGBOT_SECRET
valueFrom: { secretKeyRef: { name: testingbot, key: secret } }
- name: TESTINGBOT_TUNNEL_IDENTIFIER
value: k8s-staging
ports:
- containerPort: 4445 # Selenium relay
- containerPort: 8003 # metrics and health
startupProbe:
httpGet: { path: /readyz, port: 8003 }
periodSeconds: 5
failureThreshold: 36
readinessProbe:
httpGet: { path: /readyz, port: 8003 }
periodSeconds: 10
livenessProbe:
httpGet: { path: /healthz, port: 8003 }
periodSeconds: 30
```
Who can reach port 4445
The image binds every interface so that the Service can reach it. The Selenium relay attaches your TestingBot credentials to everything it forwards, so restrict access to it with a `NetworkPolicy` and do not expose it outside the cluster.
Running more than one tunnel, for example one per environment? See [multiple tunnels](https://testingbot.com/support/tunnel/multiple) for how identifiers route tests to the right one.
## --readyfile
`--readyfile FILE` touches a file once the tunnel is ready. It still works, but it cannot express “was ready, has since dropped”.
The file is created once and not removed while the process lives, so a tunnel that lost its SSH connection still looks ready on disk. Prefer `--ready` or `/readyz`, which ask the process for its current state.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/tunnel/logging
# Quiet when it works, detailed when it fails
By default the tunnel logs only requests that fail or return a 5xx, with their headers. That keeps the log readable in normal use and self-diagnosing when a test breaks. When you need more, turn up browser traffic and the Selenium relay independently.
- Since v5.0
## Where logs go and how much
| Option | Effect |
| --- | --- |
| -l,--logfile \ | Also write the log to a file. |
| --log-level \ | `error`, `warn`, `info` (default), `debug` or `trace`. |
| -d,--debug | Alias for `--log-level debug`. |
| --log-http \ | Per-request HTTP logging, see below. |
| --log-format \ | Plain text (default) or one JSON object per record. |
| --request-id-header \ | Header carrying the correlation id. Default `X-Request-Id`. |
```bash
java -jar testingbot-tunnel.jar --logfile tunnel.log --log-level debug
```
## HTTP request logging
`--log-http` controls how much of each proxied request is written to the log.
| Mode | Logs |
| --- | --- |
| errors | **Default.** Request line and headers, only for failed or 5xx responses. |
| none | Nothing per request. |
| url | One line per request. This was the 4.x behaviour. |
| headers | Every request with its headers. |
```bash
java -jar testingbot-tunnel.jar --log-http url
```
## Browser traffic and the Selenium relay, separately
The tunnel carries two kinds of traffic: the **proxy** (requests the cloud browser makes to your sites) and the **forwarder** (the Selenium relay on port 4445 that your test talks to). Set a level for each with `proxy:` and `forwarder:`.
```bash
# debugging a WebDriver problem: silence page traffic, show every command
java -jar testingbot-tunnel.jar --log-http proxy:none,forwarder:headers
# debugging a page that will not load: the opposite
java -jar testingbot-tunnel.jar --log-http proxy:headers,forwarder:none
```
`forwarder:body` also logs the relay's request body. That body is WebDriver capabilities, which routinely carry your access key, so it is redacted before it is written (see [redaction](https://testingbot.com#redaction)). `proxy:body` is refused: browser traffic has to stream and cannot be buffered for logging.
## Request IDs
Every request carries a correlation id. It is printed in brackets on each log line and passed to your server in the `X-Request-Id` header.
If the incoming request already has that header, its value is reused. That lets you match a line in the tunnel log to a line in your application's access log. Use a different header name if your stack expects one:
```bash
java -jar testingbot-tunnel.jar --request-id-header X-Correlation-Id
```
## JSON output
`--log-format json` writes one JSON object per record, so a log collector does not have to guess where a multi-line message or a stack trace ends.
```bash
java -jar testingbot-tunnel.jar --log-format json --logfile tunnel.jsonl
```
Useful when the tunnel runs in Docker or Kubernetes and its output is shipped to Loki, Elasticsearch, Datadog or CloudWatch.
## What is redacted
Tunnel logs often end up attached to support tickets and CI artifacts, so credentials are removed before anything is written.
- Header values that carry credentials, such as `Authorization`, `Proxy-Authorization`, cookies and `TB-Credentials`.
- Credentials in URLs: the `user:password@` part and credential-shaped query values.
- In `forwarder:body`, values whose key names look like credentials. A body that cannot be parsed, because of an unknown content type or because it is too large, is described rather than printed.
Sharing a log with support? Include the output of `--doctor` as well, it usually saves a round trip. See [troubleshooting](https://testingbot.com/support/tunnel/troubleshooting).
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/tunnel/embedding
# Run the tunnel inside your JVM
The tunnel runs as a library as well as a process. Start it from a JUnit or TestNG fixture, wait until it can carry traffic, and stop it when the suite ends. No separate process to manage, no shell script in CI.
- Since v5.0
- Java 17+
## Dependency
```xml
com.testingbotTestingBotTunnel5.1test
```
Use the regular artifact, not the shaded JAR
`com.testingbot:TestingBotTunnel:5.1` is the library artifact: Jetty, Apache HttpClient, JSch and its other dependencies are declared in its POM, so Maven or Gradle resolves them alongside your own and can settle version conflicts. The downloadable `testingbot-tunnel.jar` (the `shaded` classifier on Maven Central) is built to run on its own: it bundles those libraries under their original package names, so on your classpath they would duplicate, and possibly clash with, the versions your project already uses.
## Lifecycle
Configure an `App` with the same settings the command line takes, then `boot()`, `awaitReady()` and `stop()`.
```java
import com.testingbot.tunnel.App;
import java.time.Duration;
App tunnel = new App();
tunnel.setClientKey(System.getenv("TESTINGBOT_KEY"));
tunnel.setClientSecret(System.getenv("TESTINGBOT_SECRET"));
tunnel.setSeleniumPort(4445);
tunnel.setJettyPort(8087); // local proxy, --localproxy
tunnel.setMetricsPort(8003);
try {
tunnel.boot(); // throws TunnelFailedException instead of exiting the JVM
if (tunnel.awaitReady(Duration.ofMinutes(2))) {
// run the tests
} else {
throw new IllegalStateException("tunnel not ready: " + tunnel.getSetupFailure());
}
} finally {
tunnel.stop();
}
```
| Method | What it does |
| --- | --- |
| boot() | Creates the tunnel. Returns once it has been _created_, which is not the same as it being able to carry traffic: if the tunnel server is still starting, the rest happens in the background. |
| awaitReady(Duration) | Waits for the same state `/readyz` reports. Returns `false` as soon as setup gives up, so you do not wait out the timeout for a tunnel that has already failed. |
| getSetupFailure() | Why setup failed, for your error message. |
| isReady() | The current readiness, without waiting. |
| stop() | Shuts the tunnel down and releases everything it held. |
Most command-line options have a matching setter, for example `setTunnelIdentifier`, `setShared`, `setBindAddress` and `setProxy`.
## One tunnel per test class
```java
import com.testingbot.tunnel.App;
import java.time.Duration;
import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.BeforeAll;
class CheckoutIT {
static App tunnel;
@BeforeAll
static void startTunnel() throws Exception {
tunnel = new App();
tunnel.setClientKey(System.getenv("TESTINGBOT_KEY"));
tunnel.setClientSecret(System.getenv("TESTINGBOT_SECRET"));
tunnel.setTunnelIdentifier("checkout-it");
tunnel.boot();
if (!tunnel.awaitReady(Duration.ofMinutes(2))) {
throw new IllegalStateException("tunnel not ready: " + tunnel.getSetupFailure());
}
}
@AfterAll
static void stopTunnel() {
if (tunnel != null) tunnel.stop();
}
// tests use http://localhost:4445/wd/hub with tunnel-identifier "checkout-it"
}
```
## What you can rely on
- **Nothing calls `System.exit`.** Every failure while booting throws `TunnelFailedException`. A failure in the background, where there is nobody to throw to, releases everything and leaves a stopped tunnel whose `isReady()` and `/readyz` say so.
- **`stop()` gives everything back.** The Selenium relay, local proxy and metrics port are unbound, the SSH session and its timers are cancelled, shutdown hooks are unregistered, the ready file is removed, and the JVM's default `Authenticator` is put back the way it was found. Starting a tunnel per job does not leak.
- **Readiness is per tunnel.** Two tunnels in one JVM each answer for themselves, on their own metrics port.
- **Your logging is left alone.** Console handlers, log levels and `--log-format` are set up by the command-line entry point, not by `boot()`.
## Shared within a JVM
Three things are process-wide, because what they are built on is.
- The Prometheus registry behind `/metrics` and the request and byte counters on `/`, which add up across tunnels in the same JVM.
- `Authenticator.setDefault`, which the JDK uses for proxy and SOCKS credentials. The tunnel's authenticator answers only for the proxy it was given and delegates everything else to whatever was installed before it.
- Uptime, measured from the first tunnel to boot.
Not on the JVM? The [NodeJS launcher](https://testingbot.com/support/tunnel/installation#node) and the [GitHub Action](https://testingbot.com/support/integrations/ci-cd/github-actions) manage the tunnel process for you.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/tunnel/commandline
# Every flag, every variable
The complete commandline reference for TestingBot Tunnel. Pass credentials positionally, via environment variables, or from a `.testingbot` file in your `$HOME`.
## Three ways to authenticate
Method 1
Positional args
`java -jar testingbot-tunnel.jar KEY SECRET`
Method 2
Env vars
`TESTINGBOT_KEY` + `TESTINGBOT_SECRET`
Method 3
.testingbot file
In `$HOME`, containing `key:secret`.
## Environment variables
Use environment variables to keep sensitive credentials out of your shell history and the system process list.
| Variable | Replaces / purpose |
| --- | --- |
| TESTINGBOT\_KEY | Your TestingBot API key. Replaces the first positional argument. |
| TESTINGBOT\_SECRET | Your TestingBot API secret. Replaces the second positional argument. |
| TESTINGBOT\_AUTH | Basic-auth credentials in the form `host:port:user:password`. Replaces `-a`. |
| TESTINGBOT\_PROXY\_USERPWD | Upstream proxy credentials as `user:password`. Replaces `-z`. |
| TESTINGBOT\_METRICS\_AUTH | HTTP-basic credentials for the metrics endpoint as `user:password`. Replaces `--metrics-auth`. |
| TESTINGBOT\_PROXY\_TESTINGBOT\_USERPWD | Credentials for `--proxy-testingbot` as `user:password`. |
**Since 5.0, long options have an environment alias:** `--se-port` reads `TESTINGBOT_SE_PORT`, `--allow-hosts` reads `TESTINGBOT_ALLOW_HOSTS`, and so on. The exceptions are `--help`, `--version`, `--doctor`, `--ready` and `--config`, which have no alias, and `TESTINGBOT_AUTH`, which takes a comma-separated list of `--auth` entries. Options can also come from a `--config` file. Precedence is command line, then config file, then environment. See [configuration](https://testingbot.com/support/tunnel/configuration).
## All command-line options
Run `--help` for the list your version supports. Options marked _new in 5.0_ need tunnel 5.0 or later.
| Option | Description |
| --- | --- |
| -h,--help | Display help text and exit. |
| -v,--version | Display the tunnel version and exit. |
| --doctor | Run diagnostics with the options given and exit: DNS, connectivity to TestingBot, free ports, upstream proxy, Kerberos and `--cacert-file` subjects. |
| --config \ | Read options from a properties file. Keys are long option names without dashes. _New in 5.0._ See [configuration](https://testingbot.com/support/tunnel/configuration). |
| -i,--tunnel-identifier \ | Tag the tunnel with an identifier. When you have multiple tunnels, set this identifier in your test capabilities to pick which tunnel handles the test. |
| -s,--shared | Share this tunnel with other users on your team. Tunnels are private by default. |
| -P,--se-port \ | Local Selenium port your tests connect to. Default `4445`. |
| -j,--localproxy \ | Port the local proxy listens on. Default `8087`. |
| --bind-address \ | `127.0.0.1` (default) or `0.0.0.0`. Interface for every local listener: the Selenium relay, local proxy, metrics port and `--web`. _New in 5.0;_ 4.x bound every interface. See [upgrading](https://testingbot.com/support/tunnel/upgrading#bind). |
| -p,--hubport \ | Connect to port 80 on the TestingBot hub instead of the default port 4444. |
| -x,--noproxy | Do not start the local proxy. Requires you to run your own proxy on port 8087. |
| -q,--nocache | Bypass the caching proxy running on the tunnel VM. Required for WebSocket and SSE proxying. |
| -w,--web \ | Serve a local directory over a built-in webserver. Handy for static-file testing. |
| -f,--readyfile \ | Touch this file when the tunnel is ready. Prefer `--ready` for health checks. |
## Traffic rules, headers and site auth
| Option | Description |
| --- | --- |
| --allow-hosts \ | Only reach these hosts; everything else is refused with 403. `*.suffix` matches subdomains. _New in 5.0._ |
| -F,--fast-fail-regexps \ | Comma-separated regular expressions for hosts the tunnel should refuse. Prefix an entry with `!` to make an exception, e.g. `.*,!ok\.com`. |
| --localhost-policy \ | Whether tunnel traffic may reach this machine's loopback interface. Default `allow`. _New in 5.0._ |
| --connect-to \ | Dial `H2:P2` for requests to `H1:P1`, keeping the URL, Host header and TLS SNI. Comma separated. _New in 5.0._ |
| --dns \ | Resolve hostnames with these DNS servers, `host` or `host:port`. The first is primary, the rest are fallbacks. |
| --dns-round-robin | Spread queries across the `--dns` servers instead of using the first as primary. |
| --dns-timeout \ | Timeout per DNS server. Default `5`. |
| --http-dial-timeout \ | TCP connect timeout to targets and upstream proxies. Default 5 for the API, 15 for tunnelled connections. |
| --http-idle-timeout \ | Close connections idle this long. Default 120 for HTTP, 300 for CONNECT. |
| -a,--auth \ | Perform HTTP Basic Authentication for specific hosts. Plain HTTP only. See [authentication](https://testingbot.com/support/tunnel/authentication). |
| --krb5-hosts \ | Send Kerberos (SPNEGO) credentials to these sites. Never wildcarded. Plain HTTP only. |
| --header \ | Edit a request header: `name: value` sets, `name;` empties, `-name` removes, `-name*` removes by prefix. Repeatable. Plain HTTP only. See [header rewriting](https://testingbot.com/support/tunnel/headers). |
| --response-header \ | Edit a response header, same grammar as `--header`. For example `-Content-Security-Policy`. |
| --extra-headers \ | Add request headers from a JSON object. |
| --pac \ | PAC URL for the **remote browser** to use. Not the same as `--pac-local`. |
## Upstream proxy
| Option | Description |
| --- | --- |
| -Y,--proxy \<[scheme://]host:port\> | Upstream proxy for egress: `host:port`, `http://host:port` or `socks5://host:port`. Also used for the connection to TestingBot unless `--proxy-testingbot` is set. |
| -z,--proxy-userpwd \ | Credentials for `--proxy`. Env: `TESTINGBOT_PROXY_USERPWD`. |
| --proxy-auth-scheme \ | How to authenticate to the upstream proxy. `negotiate` is Kerberos/SPNEGO. |
| --proxy-spn \ | Service principal for Negotiate. Default `HTTP/`. |
| --krb5-keytab \ | Keytab for unattended Kerberos. Requires `--krb5-principal`. |
| --krb5-principal \ | Principal to log in as with the keytab. |
| --proxy-testingbot \ | Separate proxy for reaching TestingBot itself: the API and the SSH connection. |
| --proxy-testingbot-userpwd \ | Credentials for `--proxy-testingbot`. Not inherited from `--proxy-userpwd`. |
| --pac-local \ | Choose this tunnel's egress per destination from a PAC file. A plain `http://` URL needs `--pac-local-sha256`. See [PAC files](https://testingbot.com/support/tunnel/upstream-proxy#pac-local). |
| --pac-local-sha256 \ | SHA-256 the PAC document must have. |
| --pac-test \ | Evaluate `--pac-local` for one URL, print the result and exit. |
| --ws-proxy-mode \ | How a `ws://` upgrade traverses `--proxy`. Default `connect`. |
## TLS and SSH
| Option | Description |
| --- | --- |
| -b,--nobump | Do not perform SSL bumping. Required for WebSocket and SSE proxying. See [SSL bumping](https://testingbot.com/support/tunnel/ssl-bumping). |
| --cacert-file \ | Trust an additional CA (PEM), for proxies that intercept TLS. Repeatable. Added to the platform trust store. |
| --ssh-host-key \ | Pin the SHA-256 fingerprint the tunnel server's SSH host key must have. Repeatable. See [security](https://testingbot.com/support/tunnel/security#host-key). |
| --ssh-host-key-policy \ | Without a fingerprint: `warn` (default) connects and says so, `require` refuses. |
## Logging and health
| Option | Description |
| --- | --- |
| -l,--logfile \ | Write logs to a file in addition to stdout. |
| --log-level \ | One of `error`, `warn`, `info` (default), `debug`, `trace`. |
| -d,--debug | Alias for `--log-level debug`. |
| --log-http \ | `none`, `url`, `headers` or `errors` (default). Per module: `proxy:none,forwarder:headers`. See [logging](https://testingbot.com/support/tunnel/logging). |
| --log-format \ | One JSON object per record with `json`. |
| --request-id-header \ | Correlation id header. Default `X-Request-Id`. |
| --metrics-port \ | Port for the JSON status, Prometheus `/metrics`, `/healthz` and `/readyz`. Default `8003`. |
| --metrics-auth \ | Protect `/metrics` with HTTP Basic Authentication. Health endpoints stay open. |
| --ready | Query a running tunnel's `/readyz` and exit 0 (ready) or 1. See [health checks](https://testingbot.com/support/tunnel/health-checks). |
See practical configurations on the [examples](https://testingbot.com/support/tunnel/examples) page, or the full [installation guide](https://testingbot.com/support/tunnel/installation) for Docker, Maven and the NodeJS launcher.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/tunnel/troubleshooting
# When something does not look right
Most tunnel problems fall into one of four categories: a missing prerequisite, a network or firewall block, a wrong flag combination or a credential mismatch. This page walks through the common symptoms, what causes them, and how to fix them.
## Run --doctor first
The built-in health check validates ports, outbound connectivity and credentials in one step. Run it before anything else when something looks wrong.
```bash
java -jar testingbot-tunnel.jar --doctor
```
`--doctor` exits with a non-zero status if any check fails, which makes it usable as a CI pre-flight step.
## Issues you may encounter
### Address already in use on port 4445 or 8087
Another process is bound to one of the tunnel's local ports. Identify it with `lsof -i :4445` on macOS or Linux, or `netstat -ano | findstr :4445` on Windows. Stop the offending process or change the tunnel's local ports with `--se-port 4446` and `--localproxy 8088`.
### --doctor reports "Cannot connect to hub.testingbot.com:443"
The tunnel cannot reach TestingBot. Verify outbound port 443 to `*.testingbot.com` is open. If you are behind a corporate proxy, route tunnel traffic through it with `--proxy host:port` and `--proxy-userpwd user:password`. See the [upstream proxy guide](https://testingbot.com/support/tunnel/upstream-proxy).
### Authentication failed: 401 Unauthorized
Your TestingBot key or secret is wrong, swapped, or has been rotated. Copy them again from [your account settings](https://testingbot.com/members/user/edit). If you use environment variables, make sure `TESTINGBOT_KEY` and `TESTINGBOT_SECRET` are exported in the same shell that runs the tunnel.
### SSL handshake fails inside the cloud browser
Your test target uses certificate pinning or a non-standard CA chain. Either [download the TestingBot CA certificate](https://testingbot.com/downloads/testingbot-ssl-bumping.crt) and add it to the trust store on the cloud VM, or disable SSL bumping with the `--nobump` flag.
### WebSocket connection fails or upgrades hang
WebSocket and SSE proxying requires both `--nobump` and `--nocache` to be set. The local caching proxy and the SSL-bumping interceptor cannot forward streaming protocols, so they must be disabled. These flags are prerequisites, not optional fix-it switches.
### Tests are unusually slow through the tunnel
Bypass non-essential domains with `--fast-fail-regexps`. A typical setting like the one below often halves the proxied request count:
```bash
java -jar testingbot-tunnel.jar -F ".*google-analytics\.com.*,.*facebook\.com.*,.*doubleclick\.net.*"
```
If your network has limited bandwidth, run [multiple tunnels in parallel](https://testingbot.com/support/tunnel/multiple) on different machines and let TestingBot load-balance across them.
### Tunnel disconnects every few minutes
A firewall or NAT is killing idle connections. Watch the connection counters at `http://localhost:8003` to see reconnect frequency, as covered in the [monitoring guide](https://testingbot.com/support/tunnel/monitoring). If JVM memory is the cause, increase the heap:
```bash
java -Xmx2g -jar testingbot-tunnel.jar key secret
```
### Java version error: "class file has wrong version"
TestingBot Tunnel 5.0 requires Java 17 or higher (4.x required Java 11). If you cannot upgrade Java, use a [self-contained build](https://testingbot.com/support/tunnel/installation#standalone), which bundles its own runtime. Otherwise verify with `java -version` and install Java 17 LTS if needed. See the [installation guide](https://testingbot.com/support/tunnel/installation).
### localhost works on desktop VMs but not on mobile devices
Mobile simulators, emulators and physical devices cannot resolve the literal hostname `localhost`. Add a custom hostname to `/etc/hosts` on the tunnel host, such as `127.0.0.1 mywebsite.local`, and reference `http://mywebsite.local` in your tests.
### Tunnel shows as active in dashboard after I stopped it
The tunnel was terminated abruptly (`kill -9` or hard machine shutdown) and could not deregister itself. Wait up to 5 minutes for the TestingBot grid to time it out, or shut down gracefully next time: press `Ctrl+C`, delete `~/.testingbot-tunnel.pid`, or send `kill -SIGUSR1 `.
## Reading the logs
Re-run the tunnel with `--log-level debug` and `--logfile` to capture detailed output to a file.
```bash
java -jar testingbot-tunnel.jar --log-level debug --logfile tunnel.log key secret
```
The log file is the first thing support will ask for. `--debug` additionally prints request and response headers, useful when diagnosing application-level issues.
## Contact support
If none of the steps above resolve your issue, reach out to [TestingBot support](https://testingbot.com/contact/new) with:
- The exact tunnel version (`java -jar testingbot-tunnel.jar --version`).
- Your `--doctor` output.
- The debug log file from a failing run.
- The flags you started the tunnel with.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/tunnel/faq
# Common questions, direct answers
The most common questions about installing, configuring and operating TestingBot Tunnel. If your question is not answered here, see the [troubleshooting guide](https://testingbot.com/support/tunnel/troubleshooting) or [contact support](https://testingbot.com/contact/new).
## How do I use the tunnel with manual (live) testing?
When starting a manual test session, select which tunnel to use from the session configuration screen. This works across all platforms: desktop VMs, simulators, emulators and physical devices.
Team members can start shared tunnels (with the `--shared` flag) that everyone on the team can use during manual testing.
## How do I handle SSL certificate pinning in my mobile app?
By default, TestingBot Tunnel performs SSL interception (also called SSL bumping) to handle common certificate issues like self-signed or expired certificates. All TestingBot VMs and devices are configured to trust our CA certificate.
If your app uses certificate pinning, [download the TestingBot CA certificate](https://testingbot.com/downloads/testingbot-ssl-bumping.crt) and add it to your app's trusted certificates. Alternatively, disable SSL interception by starting the tunnel with the `--nobump` flag.
## How many tunnels can I run simultaneously?
The number of simultaneous tunnels depends on your account plan. The default limit is 2 concurrent tunnels. [Upgrade your plan](https://testingbot.com/pricing) if you need more.
## Can I use localhost with the tunnel?
**The reliable answer, everywhere:** use `local.testingbot.com` instead of `localhost`. It resolves to `127.0.0.1`, so it reaches exactly the same server on your machine, and it works on desktop VMs, simulators, emulators and physical devices alike.
```bash
http://local.testingbot.com:3000
https://local.testingbot.com:8080
```
Browsers refuse to send `localhost` through a proxy at all — Chrome, Edge and Firefox all treat it as a special name that means “this machine”. Over `https://` that means the request never leaves the browser and never reaches your tunnel, usually showing up as `ERR_SSL_PROTOCOL_ERROR`. Any name that is not `localhost` avoids the problem entirely.
**Plain `http://localhost` on desktop VMs:** this does work, on these ports: `80, 443, 3000, 3001, 3030, 3400, 8080`. For additional ports, use the `localHttpPorts` or `localHttpsPorts` capabilities with an array of port numbers.
**Mobile devices:** simulators, emulators and physical devices cannot resolve `localhost` at all. Use `local.testingbot.com`, or add your own hostname to the hosts file on the machine running the tunnel:
```bash
# Add to /etc/hosts on your tunnel machine
127.0.0.1 mywebsite.local
```
Then use `http://mywebsite.local` in your tests. The tunnel will route requests to your local server.
## Can I use WebSockets or SSE with the tunnel?
Yes, TestingBot Tunnel v4 and later supports WebSockets (`ws://` and `wss://`) and Server-Sent Events (SSE).
Prerequisite flags
WebSocket and SSE proxying **requires** the `--nobump` and `--nocache` flags. These are prerequisites, not optional fix-it switches: the local caching proxy and the SSL-bumping interceptor cannot forward streaming protocols.
```bash
java -jar testingbot-tunnel.jar --nocache --nobump
```
Hostname for WebSocket
WebSocket connections cannot use `localhost` directly. Use `local.testingbot.com`, which resolves to `127.0.0.1` and needs no setup:
```bash
ws://local.testingbot.com:3000
wss://local.testingbot.com:3000
```
Or create your own hostname in `/etc/hosts` on the machine running the tunnel:
```bash
127.0.0.1 testingbot.com
```
Then connect using `ws://testingbot.com:3000`.
## How can I improve tunnel performance?
Use the `--fast-fail-regexps` (or `-F`) option to bypass the tunnel for domains that do not affect your tests, such as analytics, ads or third-party trackers:
```bash
java -jar testingbot-tunnel.jar -F ".*google-analytics\.com.*,.*facebook\.com.*"
```
This reduces the number of requests proxied through the tunnel, improving overall test speed. For high-traffic test runs, consider [running multiple tunnels in parallel](https://testingbot.com/support/tunnel/multiple).
## Can I use the same tunnel for both virtual and real devices?
Yes, a single tunnel instance works with desktop VMs, simulators, emulators and physical devices. There is no need to start separate tunnels for different device types.
## Does TestingBot Tunnel work behind a corporate firewall or VPN?
Yes. The tunnel only needs outbound access to `*.testingbot.com` on port 443 (HTTPS) and port 22 (SSH). It does not require any inbound ports.
For corporate networks that force all traffic through an HTTP proxy, route tunnel traffic through the proxy with `--proxy` and `--proxy-userpwd`:
```bash
java -jar testingbot-tunnel.jar --proxy corp-proxy.example.com:8080 --proxy-userpwd user:password
```
See the [upstream proxy guide](https://testingbot.com/support/tunnel/upstream-proxy) for PAC and SOCKS variations.
## What ports does TestingBot Tunnel use?
| Direction | Port | Purpose |
| --- | --- | --- |
| Outbound | 443 | HTTPS to `*.testingbot.com` |
| Outbound | 22 | SSH tunnel |
| Local bind | 4445 | Selenium hub (override `--se-port`) |
| Local bind | 8087 | Local proxy (override `--localproxy`) |
| Local bind | 8003 | Metrics endpoint (override `--metrics-port`) |
No inbound ports are required.
## Is TestingBot Tunnel free?
Yes. TestingBot Tunnel is free for all TestingBot accounts including the free trial. It is open source under an MIT license, [source on GitHub](https://github.com/testingbot/Testingbot-Tunnel).
## How do I update TestingBot Tunnel to the latest version?
Download the latest JAR from the [tunnel home page](https://testingbot.com/support/tunnel), unzip and replace your existing `testingbot-tunnel.jar`. If you use Docker, run `docker pull testingbot/tunnel`. If you use Maven, bump the `com.testingbot:TestingBotTunnel` version in your `pom.xml`. See the [changelog](https://testingbot.com/support/tunnel/changelog) for the current release notes.
Moving from 4.x to 5.0? Version 5.0 needs Java 17, or a [self-contained build](https://testingbot.com/support/tunnel/installation#standalone) that bundles its own runtime, and its local listeners bind `127.0.0.1` by default. Read [upgrading from 4.x](https://testingbot.com/support/tunnel/upgrading) first.
## Why does my tunnel disconnect or time out?
The most common causes are: an idle network connection being killed by a corporate firewall or NAT timeout, JVM out of memory on the tunnel host, or a transient outbound connectivity issue.
Run the tunnel with `--doctor` to validate ports and credentials, watch the metrics endpoint on port 8003 for connection counters, and check the log file specified by `--logfile` for stack traces. The [troubleshooting guide](https://testingbot.com/support/tunnel/troubleshooting) walks through the most frequent root causes.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/tunnel/changelog
# Release history
Every released version of TestingBot Tunnel, newest first. New features, improvements and bug fixes.
1. ## v5.1
October 2026 Current
- Several SSH connections to the tunnel server: `--ssh-connections N` (1-8, default 4). One slow or dropped connection no longer holds up the others, and each reconnects on its own. On a tunnel server that cannot spread traffic over them, the tunnel uses one connection, as before
- Uploads through the tunnel are up to 10x faster at real-world latency: each forwarded connection's window is 2 MB instead of 128 KB, which capped every connection at 128 KB per round trip
- A dead connection is noticed in about 6 seconds instead of up to 15 minutes after a network change (VPN reconnect, Wi-Fi switch), and the tunnel reconnects straight away
- Browser traffic keeps flowing through reconnects: the tunnel re-requests and checks its connection to the tunnel server, and repairs it if it stopped delivering
- PAC files (`--pac-local`): a failover list such as `PROXY a; DIRECT` falls through to the next entry when a proxy cannot be reached, `isInNet()` resolves host names, and an invalid PAC file is reported before a tunnel server is started
- `--noproxy` without `--localproxy` uses port 8087, as its help text says
- An incomplete response from the TestingBot API is reported plainly instead of as a stack trace
- Startup no longer logs two failed warm-up requests
- The Windows self-contained build is available again
2. ## v5.0
October 2026
- **Breaking:** requires Java 17 (was 11). On an older Java the tunnel now says so plainly. Or use a new [self-contained build](https://testingbot.com/support/tunnel/installation#standalone) that needs no Java
- **Breaking:** local listeners (Selenium relay, local proxy, metrics port, `--web`) bind `127.0.0.1` instead of every interface. Use `--bind-address 0.0.0.0` to accept connections from other machines. The Docker image sets this for you
- **Breaking:** successful requests no longer log a line. `--log-http` defaults to `errors`; `--log-http url` restores the 4.x behaviour. See [upgrading from 4.x](https://testingbot.com/support/tunnel/upgrading)
- New self-contained downloads for Linux x64/arm64, macOS Intel/Apple silicon (signed and notarized) and Windows x64, with a `.sha256` for every file
- Built on Jetty 12
- Restrict what the tunnel can reach: `--allow-hosts`, `!` exceptions in `--fast-fail-regexps` and `--localhost-policy deny`, enforced on HTTP, CONNECT and WebSocket upgrades. See [traffic rules](https://testingbot.com/support/tunnel/traffic-rules)
- Corporate networks: `--pac-local` to choose egress per destination from a PAC file, Kerberos/SPNEGO with `--proxy-auth-scheme negotiate` and `--krb5-hosts`, a separate `--proxy-testingbot`, `--cacert-file` for TLS-intercepting proxies, SOCKS5 upstream proxies, `--connect-to`, `--ws-proxy-mode`, multi-server `--dns` and `--http-dial-timeout` / `--http-idle-timeout`
- Security: pin the tunnel server's SSH host key with `--ssh-host-key` and `--ssh-host-key-policy require`; upstream proxy credentials are only sent to the proxy they belong to; credentials are redacted from logs
- Operations: `/healthz` and `/readyz` endpoints, `--ready` for health checks, and a `HEALTHCHECK` in the Docker image. `/readyz` now reports ready only after the startup self-test passes
- Logging: `--log-http` per module (`proxy:`, `forwarder:`), `--log-format json` and request-id correlation
- Configuration: `--config` file and a `TESTINGBOT_*` environment variable for nearly every option
- `--header` and `--response-header` to add, replace or remove headers on plain HTTP
- Failed requests now say why, with an `X-TestingBot-Error` header and a readable message
- More resilient reconnects, and `--doctor` diagnoses the configuration it is actually given
- The tunnel can be embedded as a Java library with `boot()`, `awaitReady()` and `stop()`. See [embedding](https://testingbot.com/support/tunnel/embedding)
3. ## v4.9
August 2026
- Fixed reverse port forwarding on macOS: every request through the tunnel failed with a `502 Bad Gateway` while the tunnel reported itself healthy
- The tunnel now verifies at startup and continuously that reverse forwarding can deliver traffic, and reports failures prominently instead of silently ignoring them
- A failed startup self-test no longer prints "The Tunnel is ready"
- `--log-level debug` no longer writes credentials to the log
- Fixed HTTPS CONNECT host handling for `--fast-fail-regexps` matching and logging
- Updated dependencies (security fixes)
4. ## v4.8
May 2026
- Added a Prometheus `/metrics` endpoint exposing HTTP request rate, duration and size histograms, HTTPS CONNECT counters, SSH tunnel state and reconnects, active connections, error counters and JVM/process metrics
- Added `--metrics-auth user:password` to protect the metrics endpoint with HTTP Basic Authentication
- Added `TESTINGBOT_AUTH`, `TESTINGBOT_PROXY_USERPWD` and `TESTINGBOT_METRICS_AUTH` environment variables so credentials stay out of the process list
- Sensitive headers are now stripped from debug logging
- Improved basic authentication handling and logging
- Updated dependencies
5. ## v4.7
March 2026
- Fixed an issue with WebSocket proxying
6. ## v4.6
January 2026
- Fixed an issue with `--debug`
7. ## v4.5
January 2026
- Added `-s`, `--shared` to share a tunnel with colleagues on your team. Tunnels are not shared by default
- Updated dependencies
- Code improvements
8. ## v4.4
October 2025
- Improved `--proxy` and `--proxy-userpwd` reliability
- Updated dependencies
9. ## v4.3
August 2025
- Enhanced `--doctor` diagnostic checks
- Improved connection monitoring and automatic retry mechanisms
- Updated Jetty and SSH libraries
- Dropped support for Java 8 (Java 11+ now required)
10. ## v4.0
December 2024
- Added WebSocket and Server-Sent Events (SSE) support
- Added `--nobump` option to disable SSL bumping
- Upgraded to Jetty 11
11. ## v3.5
May 2023
- Upgraded secure connection ciphers for improved security
- Updated dependencies
12. ## v3.2
February 2021
- Updated dependencies for security and compatibility
13. ## v3.1
February 2020
- Fixed tunnel polling to prevent indefinite connection attempts
- Built with Java 8 for broader compatibility
- Internal code improvements
14. ## v3.0
September 2019
- Updated Jetty dependencies
- Various stability improvements
15. ## v2.9
January 2019
- Various minor fixes to timeout handling and `--doctor` checks
16. ## v2.8
June 2018
- Fixed null pointer exception in connection handler
17. ## v2.7
June 2018
- Added upstream HTTPS proxy support via CONNECT method (`--proxy`)
- Renamed `--jettyport` option to `--localport`
- Enabled `--nocache` integration with Manual Testing API
- Removed hardcoded port restrictions
18. ## v2.6
April 2018
- Added option to inject custom HTTP headers into requests
- Added automatic basic authentication for configured hosts
- Clarified `--nocache` behavior: disables all HTTP request caching
- Removed deprecated `--boost` and `--ssl` options
- Fixed incorrect Java version detection
19. ## v2.5
September 2017
- The tunnel now reports its version to TestingBot
- Added `--pac` option to configure a PAC (Proxy Auto-Config) URL for test sessions
20. ## v2.4
August 2017
- Added `--tunnel-identifier` to run multiple tunnels on the same account
- Added local directory serving for static file testing
- Added embedded REST API for tunnel statistics (default port 8003)
- Improved logging with configurable output format, file logging, and debug levels
- Improved connection error handling with automatic reconnection
21. ## v2.3
October 2016
- Added `--doctor` command to troubleshoot configuration issues
- Debug mode now outputs HTTP headers for easier troubleshooting
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/web-automate/mobile
# Mobile Browser Testing with Selenium
TestingBot lets you run Selenium WebDriver tests against **mobile Safari on iOS** (iPhone and iPad) and **mobile Chrome on Android** , on both [simulators/emulators](https://testingbot.com/support/web-automate/browsers) and [real physical devices](https://testingbot.com/support/app-automate/devices) in our EU data centre. Mobile sessions are routed through Appium automatically; no separate Appium install needed on your side.
Looking for native app testing instead of mobile browsers? See [App Automate](https://testingbot.com/support/app-automate) for Appium, XCUITest, Espresso and Maestro on real devices.
## Configuring capabilities
TestingBot's mobile simulators and emulators can be accessed using Appium. You can use the device picker below to select the capabilities you want to use for your tests.
 iOS 18.6 › iPhone 16
Loading environments...
Please wait while we load the available devices and platforms.
W3C Protocol
W3C is the newest WebDriver protocol, TestingBot recommends using W3C.
JSONWP
JSONWP is the legacy protocol which is no longer actively maintained.
Java Python NodeJS C# Ruby PHP
```java
MutableCapabilities capabilities = new MutableCapabilities();
capabilities.setCapability(CapabilityType.BROWSER_NAME, "safari");
capabilities.setCapability(CapabilityType.BROWSER_VERSION, "18.6");
capabilities.setCapability("appium:deviceName", "iPhone 16");
capabilities.setCapability(CapabilityType.PLATFORM_NAME, "iOS");
```
You can configure more options by using the [Appium capabilities generator](https://testingbot.com/support/app-automate/appium/capabilities).
## Portrait/Landscape
It's possible to rotate the device before and during your test. Please see these examples:
**Set orientation before test (capability):**
```ruby
caps = {
platformName: "iOS",
"appium:deviceName": "iPhone 15",
"appium:orientation": "LANDSCAPE" # or PORTRAIT
}
```
**Rotate during test:**
```java
((AppiumDriver) driver).rotate(ScreenOrientation.LANDSCAPE);
```
## Specifying Appium Version
TestingBot will use the most recent, compatible, Appium version according to the device, OS and version you specify.
If you'd like to specify your own Appium version, you can do this with the `appiumVersion` capability.
To learn more about the available Appium versions, please see our [Appium versions documentation](https://testingbot.com/support/app-automate/appium/appium-versions).
[Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Python](https://testingbot.com#) [C#](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#)
Pass the `appiumVersion` capability inside `tb:options`. The examples below target an iPhone running mobile Safari, for Android Chrome change `platformName` to `"Android"`, `browserName` to `"chrome"`, and pick an Android `deviceName`.
```java
XCUITestOptions options = new XCUITestOptions();
options.setPlatformName("iOS");
options.setDeviceName("iPhone 16");
options.setPlatformVersion("18.6");
options.setBrowserName("safari");
Map tbOptions = new HashMap<>();
tbOptions.put("key", "API_KEY");
tbOptions.put("secret", "API_SECRET");
tbOptions.put("appiumVersion", "2.4.1");
options.setCapability("tb:options", tbOptions);
IOSDriver driver = new IOSDriver(
new URL("https://hub.testingbot.com/wd/hub"), options);
```
```python
from appium import webdriver
from appium.options.ios import XCUITestOptions
options = XCUITestOptions()
options.platform_name = "iOS"
options.device_name = "iPhone 16"
options.platform_version = "18.6"
options.browser_name = "safari"
options.set_capability("tb:options", {
"key": "API_KEY",
"secret": "API_SECRET",
"appiumVersion": "2.4.1"
})
driver = webdriver.Remote("https://hub.testingbot.com/wd/hub", options=options)
```
```javascript
const { remote } = require('webdriverio');
const driver = await remote({
hostname: 'hub.testingbot.com',
port: 443,
protocol: 'https',
path: '/wd/hub',
user: 'API_KEY',
key: 'API_SECRET',
capabilities: {
platformName: 'iOS',
browserName: 'safari',
'appium:deviceName': 'iPhone 16',
'appium:platformVersion': '18.6',
'tb:options': {
appiumVersion: '2.4.1'
}
}
});
```
```csharp
using OpenQA.Selenium.Appium;
using OpenQA.Selenium.Appium.iOS;
var options = new AppiumOptions();
options.PlatformName = "iOS";
options.BrowserName = "safari";
options.AddAdditionalAppiumOption("deviceName", "iPhone 16");
options.AddAdditionalAppiumOption("platformVersion", "18.6");
options.AddAdditionalAppiumOption("tb:options", new Dictionary
{
["key"] = "API_KEY",
["secret"] = "API_SECRET",
["appiumVersion"] = "2.4.1"
});
var driver = new IOSDriver(
new Uri("https://hub.testingbot.com/wd/hub"), options);
```
```ruby
require 'appium_lib'
caps = {
caps: {
platformName: 'iOS',
browserName: 'safari',
'appium:deviceName': 'iPhone 16',
'appium:platformVersion': '18.6',
'tb:options' => {
'appiumVersion' => '2.4.1'
}
},
appium_lib: {
server_url: 'https://API_KEY:API_SECRET@hub.testingbot.com/wd/hub'
}
}
driver = Appium::Driver.new(caps).start_driver
```
```php
$capabilities = [
'platformName' => 'iOS',
'browserName' => 'safari',
'appium:deviceName' => 'iPhone 16',
'appium:platformVersion' => '18.6',
'tb:options' => [
'key' => 'API_KEY',
'secret' => 'API_SECRET',
'appiumVersion' => '2.4.1',
],
];
$driver = RemoteWebDriver::create(
'https://hub.testingbot.com/wd/hub',
$capabilities
);
```
## Testing internal or staging mobile sites
Use the [TestingBot Tunnel](https://testingbot.com/support/tunnel) to run mobile-browser tests against websites that aren't publicly reachable (local dev servers, staging environments behind a VPN, internal QA sites). Start the tunnel with an identifier, then pass `tunnelIdentifier` in your `tb:options` capabilities so the mobile device routes traffic through your tunnel.
See the [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for setup details on Java, Docker and command-line installs.
## Next steps
- [Mobile browser & OS list](https://testingbot.com/support/web-automate/browsers) — current iOS/Android browsers available on the grid.
- [Real device list](https://testingbot.com/support/app-automate/devices) — physical iPhones, iPads and Android phones/tablets.
- [Appium capabilities reference](https://testingbot.com/support/app-automate/appium/capabilities) — full list of mobile capabilities supported.
- [Appium versions](https://testingbot.com/support/app-automate/appium/appium-versions) — pin a specific Appium release.
- [App Automate](https://testingbot.com/support/app-automate) — native mobile-app testing with Appium, XCUITest, Espresso and Maestro.
- [Real device testing overview](https://testingbot.com/real-device-testing) — how the real-device cloud works.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/web-automate/codeless-automation
# Codeless Web Testing
Codeless Web Automated Testing allows you to run your pre-recorded Selenium tests on the TestingBot cloud grid.
- [Add a test](https://testingbot.com/support/web-automate/codeless-automation/add-test)
- [Test Suites](https://testingbot.com/support/web-automate/codeless-automation/suites)
- [Schedule a test](https://testingbot.com/support/web-automate/codeless-automation/schedule)
- [Test alerts](https://testingbot.com/support/web-automate/codeless-automation/alerts)
- [Supported commands](https://testingbot.com/support/web-automate/codeless-automation/commands)
- [Variables](https://testingbot.com/support/web-automate/codeless-automation/variables)
- [FAQ](https://testingbot.com/support/web-automate/codeless-automation/faq)
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/web-automate/codeless-automation/add-test
# Add Codeless Automation test
TestingBot can create and run automated web tests against your website on regular intervals and alert you when a test fails.
There are 2 options to create a codeless web automation test:
- You can [record your actions](https://testingbot.com#record) with Selenium IDE, [upload the steps](https://testingbot.com#upload) and replay the steps on TestingBot.
- You can choose to describe tests in natural language and have the [TestingBot AI agent](https://testingbot.com/support/ai/codeless) create and run the test for you.
## Record your actions
### Installing Selenium IDE
To get started, please make sure to [download the Selenium IDE](https://github.com/SeleniumHQ/selenium-ide/releases/). This is an Electron app that will run on Windows and macOS and will allow you to record and playback actions on a website and allow you to save these actions as Selenium scripts (`.side` files).
You can also install the package through NPM: `npm install -g selenium-ide`
### Recording a test
#### Creating a test
Once you have the Selenium IDE program running, you can create a new testcase by clicking the plus icon (+). Enter a name for your test. Next, open the URL you want to test in the right side of the Selenium IDE. Once the page is loaded, interact with the page. You will see new commands are added to the left sidebar, which means the IDE is recording all your actions.
#### Adding verifications
For example, if you tap an element on the webpage, you will see a new `Click` command appear. At any given point, you can right-click an element on the page you are testing and add a verification, for example: `Record Wait For Element Present`.
### Saving test
Once you've finished recording your test, you can choose to save it as a `.side` file, which is basically a JSON file that describes all the actions and verifications you added during the recording.
Choose `Save Project As` and save the file. You can now [upload this file](https://testingbot.com#upload) to TestingBot and instruct TestingBot to run this test on a scheduled interval, running on the browsers & operating systems you want.
## Upload a codeless test
Once you have a `.side` file, you can upload it in the [Codeless Test](https://testingbot.com/members/lab) area of TestingBot.
Click the upload button in the **Import a Selenium Test** section and upload the file.
Once uploaded, you will be redirected to a test configuration page. Here you can configure various options:
### Steps
We will display the same steps that were recorded before. You have the option to re-order the steps, edit steps or remove steps.
### Browsers
A selection of browsers & operating systems will be available to choose from. Select one or more browsers that you want to run your test on.
### Settings
You can configure various settings for your test:
- Schedule when the test should run; a specific time, interval, day. Or maybe every 15 minutes. You indicate how often the test will run.
- Manage timeouts, screen resolution, screenshots and videos.
- Daily and hourly reporting of your tests via email.
- Automatically retry a failed test and if the retry succeeds, ignore the failed test.
- Receive an alert via email or text message when a test fails.
- Test from various geographical locations
## Run Test
Now that the test has been added to TestingBot, you can choose to run it immediately.
Click the **Run test** button on the top right of the edit test page, or from within the test overview page.
TestingBot will run the test on the browser(s) you specified and alert you when a test failed.
Each test comes with screenshots, a video and generated log files.
## API Reference
You can use the [TestingBot API](https://testingbot.com/support/api) to add, edit, remove and [run Codeless Automation tests](https://testingbot.com/support/api/codeless#runlabtest).
Use our [integrations](https://testingbot.com/support/integrations) to improve your workflow. For example, configure a [Slack Bot](https://testingbot.com/support/integrations/slack) to notify you when a codeless automation test failed.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/web-automate/codeless-automation/suites
# Add a Test Suite
In our TestLab we offer the possibility to upload test-suites (called **Projects** in Selenium IDE) generated by [Selenium IDE](https://www.selenium.dev/selenium-ide/).
A test suite is a collection of tests, generated by the Selenium IDE extension.
To upload the suite, please upload the exported (`.side`) file:
## Managing Test Suites
Once uploaded to our TestLab, you will see page detailing the Project you just uploaded.
It will include a list of the tests that this project contains.
On this page, you can change on which browser(s) the tests should run, change alert/scheduling options and more.
Changing a setting in the project affects all tests for this project.
If you like to only change a single test, click through to the test edit page and make the necessary changes there.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/web-automate/codeless-automation/schedule
# Schedule a test
When you have [added a test](https://testingbot.com/support/web-automate/codeless-automation/add-test), you can specify when we should run the test, and how many times we should run it (interval).
You can have your test either run once, every day, every week, repeat every `..` time or use a custom CRON format.
Once scheduled, you can specify when you want to be [alerted of failed tests](https://testingbot.com/support/web-automate/codeless-automation/alerts).
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/web-automate/codeless-automation/alerts
# Alerts
When a test in Codeless Automation fails, we can notify you by sending you an email, Webhook, Push Notification or SMS/text message.
You can choose to be alerted immediately, or once a day.
## Alert Types
We offer 3 ways to alert you:
- Email
- SMS/Text message
- Webhook to a URL you specify.
The message you will receive will contain the URL, number of errors and an error message trace.
## Email Alerts
To receive an alert via email, fill in your email address.
We will send an email with the failure of the test.
You can customize the error message by filling in your own text.
We offer these variables to customize your error text: `%test_name%` and `%browser%`.
## SMS Text Message
Fill in your phone number (with country code) to receive an SMS message when a test fails.
You can customize the error message by filling in your own text.
We offer these variables to customize your error text: `%test_name%` and `%browser%`.
There's also the possibility to be alerted when the test recovers.
## Webhook
We will call a URL you specify whenever a test fails.
The URL needs to be publicly available, otherwise TestingBot cannot reach the URL.
The URL will receive a POST request from us with the following fields:
- `success` (boolean)
- `job_id` (int) - internal id
- `test_ids[]` (string[]) webdriver sessionIds
- `errors[]` (string[]) error messages
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/web-automate/codeless-automation/commands
# Supported Selenium IDE Commands
Below is a list of the commands we currently support in our Codeless Automation.
## Browsing
- **close**
Simulates the user clicking the "close" button in the titlebar of a popup window or tab.
- **closeAndWait**
Simulates the user clicking the "close" button in the titlebar of a popup window or tab.
- **goBack**
Simulates the user clicking the "back" button on their browser.
- **goBackAndWait**
Simulates the user clicking the "back" button on their browser.
- **open**
(url)
Opens an URL in the test frame. This accepts both relative and absolute URLs. The "open" command waits for the page to load before proceeding, ie. the "AndWait" suffix is implicit. Note: The URL must be on the same domain as the runner HTML due to security restrictions in the browser (Same Origin Policy). If you need to open an URL on another domain, use the Selenium Server to start a new browser session on that domain.
- **openAndWait**
(url)
Opens an URL in the test frame. This accepts both relative and absolute URLs. The "open" command waits for the page to load before proceeding, ie. the "AndWait" suffix is implicit. Note: The URL must be on the same domain as the runner HTML due to security restrictions in the browser (Same Origin Policy). If you need to open an URL on another domain, use the Selenium Server to start a new browser session on that domain.
- **openWindow**
(url - windowID)
Opens a popup window (if a window with that ID isn't already open). After opening the window, you'll need to select it using the selectWindow command. This command can also be a useful workaround for bug SEL-339. In some cases, Selenium will be unable to intercept a call to window.open (if the call occurs during or before the "onLoad" event, for example). In those cases, you can force Selenium to notice the open window's name by using the Selenium openWindow command, using an empty (blank) url, like this: openWindow("", "myFunnyWindow").
- **openWindowAndWait**
(url - windowID)
Opens a popup window (if a window with that ID isn't already open). After opening the window, you'll need to select it using the selectWindow command. This command can also be a useful workaround for bug SEL-339. In some cases, Selenium will be unable to intercept a call to window.open (if the call occurs during or before the "onLoad" event, for example). In those cases, you can force Selenium to notice the open window's name by using the Selenium openWindow command, using an empty (blank) url, like this: openWindow("", "myFunnyWindow").
- **refresh**
Simulates the user clicking the "Refresh" button on their browser.
- **refreshAndWait**
Simulates the user clicking the "Refresh" button on their browser.
- **selectFrame**
(locator)
Selects a frame within the current window. (You may invoke this command multiple times to select nested frames.) To select the parent frame, use "relative=parent" as a locator; to select the top frame, use "relative=top". You can also select a frame by its 0-based index number; select the first frame with "index=0", or the third frame with "index=2". You may also use a DOM expression to identify the frame you want directly, like this: dom=frames["main"].frames["subframe"]
- **selectFrameAndWait**
(locator)
Selects a frame within the current window. (You may invoke this command multiple times to select nested frames.) To select the parent frame, use "relative=parent" as a locator; to select the top frame, use "relative=top". You can also select a frame by its 0-based index number; select the first frame with "index=0", or the third frame with "index=2". You may also use a DOM expression to identify the frame you want directly, like this: dom=frames["main"].frames["subframe"]
- **selectWindow**
(windowID)
Selects a popup window using a window locator; once a popup window has been selected, all commands go to that window. To select the main window again, use null as the target. Window locators provide different ways of specifying the window object: by title, by internal JavaScript "name," or by JavaScript variable. title=My Special Window: Finds the window using the text that appears in the title bar. Be careful; two windows can share the same title. If that happens, this locator will just pick one. name=myWindow: Finds the window using its internal JavaScript "name" property. This is the second parameter "windowName" passed to the JavaScript method window.open(url, windowName, windowFeatures, replaceFlag) (which Selenium intercepts). var=variableName: Some pop-up windows are unnamed (anonymous), but are associated with a JavaScript variable name in the current application window, e.g. "window.foo = window.open(url);". In those cases, you can open the window using "var=foo". If no window locator prefix is provided, we'll try to guess what you mean like this: 1.) if windowID is null, (or the string "null") then it is assumed the user is referring to the original window instantiated by the browser). 2.) if the value of the "windowID" parameter is a JavaScript variable name in the current application window, then it is assumed that this variable contains the return value from a call to the JavaScript window.open() method. 3.) Otherwise, selenium looks in a hash it maintains that maps string names to window "names". 4.) If that fails, we'll try looping over all of the known windows to try to find the appropriate "title". Since "title" is not necessarily unique, this may have unexpected behavior. If you're having trouble figuring out the name of a window that you want to manipulate, look at the Selenium log messages which identify the names of windows created via window.open (and therefore intercepted by Selenium). You will see messages like the following for each window as it is opened: debug: window.open call intercepted; window ID (which you can use with selectWindow()) is "myNewWindow" In some cases, Selenium will be unable to intercept a call to window.open (if the call occurs during or before the "onLoad" event, for example). (This is bug SEL-339.) In those cases, you can force Selenium to notice the open window's name by using the Selenium openWindow command, using an empty (blank) url, like this: openWindow("", "myFunnyWindow").
- **selectWindowAndWait**
(windowID)
Selects a popup window using a window locator; once a popup window has been selected, all commands go to that window. To select the main window again, use null as the target. Window locators provide different ways of specifying the window object: by title, by internal JavaScript "name," or by JavaScript variable. title=My Special Window: Finds the window using the text that appears in the title bar. Be careful; two windows can share the same title. If that happens, this locator will just pick one. name=myWindow: Finds the window using its internal JavaScript "name" property. This is the second parameter "windowName" passed to the JavaScript method window.open(url, windowName, windowFeatures, replaceFlag) (which Selenium intercepts). var=variableName: Some pop-up windows are unnamed (anonymous), but are associated with a JavaScript variable name in the current application window, e.g. "window.foo = window.open(url);". In those cases, you can open the window using "var=foo". If no window locator prefix is provided, we'll try to guess what you mean like this: 1.) if windowID is null, (or the string "null") then it is assumed the user is referring to the original window instantiated by the browser). 2.) if the value of the "windowID" parameter is a JavaScript variable name in the current application window, then it is assumed that this variable contains the return value from a call to the JavaScript window.open() method. 3.) Otherwise, selenium looks in a hash it maintains that maps string names to window "names". 4.) If that fails, we'll try looping over all of the known windows to try to find the appropriate "title". Since "title" is not necessarily unique, this may have unexpected behavior. If you're having trouble figuring out the name of a window that you want to manipulate, look at the Selenium log messages which identify the names of windows created via window.open (and therefore intercepted by Selenium). You will see messages like the following for each window as it is opened: debug: window.open call intercepted; window ID (which you can use with selectWindow()) is "myNewWindow" In some cases, Selenium will be unable to intercept a call to window.open (if the call occurs during or before the "onLoad" event, for example). (This is bug SEL-339.) In those cases, you can force Selenium to notice the open window's name by using the Selenium openWindow command, using an empty (blank) url, like this: openWindow("", "myFunnyWindow").
- **windowFocus**
Gives focus to the currently selected window
- **windowFocusAndWait**
Gives focus to the currently selected window
- **windowMaximize**
Resize currently selected window to take up the entire screen
- **windowMaximizeAndWait**
Resize currently selected window to take up the entire screen
## Clicks
- **click**
(locator)
Clicks on a link, button, checkbox or radio button. If the click action causes a new page to load (like a link usually does), call waitForPageToLoad.
- **clickAndWait**
(locator)
Clicks on a link, button, checkbox or radio button. If the click action causes a new page to load (like a link usually does), call waitForPageToLoad.
- **clickAt**
(locator - coordString)
Clicks on a link, button, checkbox or radio button. If the click action causes a new page to load (like a link usually does), call waitForPageToLoad.
- **clickAtAndWait**
(locator - coordString)
Clicks on a link, button, checkbox or radio button. If the click action causes a new page to load (like a link usually does), call waitForPageToLoad.
- **doubleClick**
(locator)
Double clicks on a link, button, checkbox or radio button. If the double click action causes a new page to load (like a link usually does), call waitForPageToLoad.
- **doubleClickAndWait**
(locator)
Double clicks on a link, button, checkbox or radio button. If the double click action causes a new page to load (like a link usually does), call waitForPageToLoad.
- **doubleClickAt**
(locator - coordString)
Doubleclicks on a link, button, checkbox or radio button. If the action causes a new page to load (like a link usually does), call waitForPageToLoad.
- **doubleClickAtAndWait**
(locator - coordString)
Doubleclicks on a link, button, checkbox or radio button. If the action causes a new page to load (like a link usually does), call waitForPageToLoad.
- **dragAndDrop**
(locator - movementsString)
Drags an element a certain distance and then drops it
- **dragAndDropAndWait**
(locator - movementsString)
Drags an element a certain distance and then drops it
- **dragAndDropToObject**
(locatorOfObjectToBeDragged - locatorOfDragDestinationObject)
Drags an element and drops it on another element
- **dragAndDropToObjectAndWait**
(locatorOfObjectToBeDragged - locatorOfDragDestinationObject)
Drags an element and drops it on another element
## Form Fields
- **addSelection**
(locator - optionLocator)
Add a selection to the set of selected options in a multi-select element using an option locator. @see #doSelect for details of option locators
- **addSelectionAndWait**
(locator - optionLocator)
Add a selection to the set of selected options in a multi-select element using an option locator. @see #doSelect for details of option locators
- **check**
(locator)
Check a toggle-button (checkbox/radio)
- **checkAndWait**
(locator)
Check a toggle-button (checkbox/radio)
- **focus**
(locator)
Move the focus to the specified element; for example, if the element is an input field, move the cursor to that field.
- **focusAndWait**
(locator)
Move the focus to the specified element; for example, if the element is an input field, move the cursor to that field.
- **removeAllSelections**
(locator)
Unselects all of the selected options in a multi-select element.
- **removeAllSelectionsAndWait**
(locator)
Unselects all of the selected options in a multi-select element.
- **removeSelection**
(locator - optionLocator)
Remove a selection from the set of selected options in a multi-select element using an option locator. @see #doSelect for details of option locators
- **removeSelectionAndWait**
(locator - optionLocator)
Remove a selection from the set of selected options in a multi-select element using an option locator. @see #doSelect for details of option locators
- **select**
(selectLocator - optionLocator)
Select an option from a drop-down using an option locator. Option locators provide different ways of specifying options of an HTML Select element (e.g. for selecting a specific option, or for asserting that the selected option satisfies a specification). There are several forms of Select Option Locator. label=labelPattern: matches options based on their labels, i.e. the visible text. (This is the default.) label=regexp:^[Oo]ther value=valuePattern: matches options based on their values. value=other id=id: matches options based on their ids. id=option1 index=index: matches an option based on its index (offset from zero). index=2 If no option locator prefix is provided, the default behaviour is to match on label.
- **selectAndWait**
(selectLocator - optionLocator)
Select an option from a drop-down using an option locator. Option locators provide different ways of specifying options of an HTML Select element (e.g. for selecting a specific option, or for asserting that the selected option satisfies a specification). There are several forms of Select Option Locator. label=labelPattern: matches options based on their labels, i.e. the visible text. (This is the default.) label=regexp:^[Oo]ther value=valuePattern: matches options based on their values. value=other id=id: matches options based on their ids. id=option1 index=index: matches an option based on its index (offset from zero). index=2 If no option locator prefix is provided, the default behaviour is to match on label.
- **setCursorPosition**
(locator - position)
Moves the text cursor to the specified position in the given input element or textarea. This method will fail if the specified element isn't an input element or textarea.
- **setCursorPositionAndWait**
(locator - position)
Moves the text cursor to the specified position in the given input element or textarea. This method will fail if the specified element isn't an input element or textarea.
- **submit**
(formLocator)
Submit the specified form. This is particularly useful for forms without submit buttons, e.g. single-input "Search" forms.
- **submitAndWait**
(formLocator)
Submit the specified form. This is particularly useful for forms without submit buttons, e.g. single-input "Search" forms.
- **uncheck**
(locator)
Uncheck a toggle-button (checkbox/radio)
- **uncheckAndWait**
(locator)
Uncheck a toggle-button (checkbox/radio)
- **getSelectedLabels**
(selectLocator)
Gets all option labels (visible text) for selected options in the specified select or multi-select element.
- **getSelectedLabel**
(selectLocator)
Gets option label (visible text) for selected option in the specified select element.
- **getSelectedValues**
(selectLocator)
Gets all option values (value attributes) for selected options in the specified select or multi-select element.
- **getSelectedValue**
(selectLocator)
Gets option value (value attribute) for selected option in the specified select element.
- **getSelectedIndexes**
(selectLocator)
Gets all option indexes (option number, starting at 0) for selected options in the specified select or multi-select element.
- **getSelectedIndex**
(selectLocator)
Gets option index (option number, starting at 0) for selected option in the specified select element.
- **getSelectedIds**
(selectLocator)
Gets all option element IDs for selected options in the specified select or multi-select element.
- **getSelectedId**
(selectLocator)
Gets option element ID for selected option in the specified select element.
- **verifyAllButtons**
(equalTo)
- **verifyAllFields**
(equalTo)
- **verifyChecked**
(locator)
- **isChecked**
(locator)
Gets whether a toggle-button (checkbox/radio) is checked. Fails if the specified element doesn't exist or isn't a toggle-button.
- **verifyCursorPosition**
(locator - equalTo)
- **verifyEditable**
(locator)
- **waitForElementEditable**
(locator)
- **verifyNotAllButtons**
(equalTo)
- **verifyNotAllFields**
(equalTo)
- **verifyNotChecked**
(locator)
- **verifyNotCursorPosition**
(locator - equalTo)
- **verifyNotEditable**
(locator)
- **verifyNotSelectedIds**
(selectLocator - equalTo)
- **verifyNotSelectedIndexes**
(selectLocator - equalTo)
- **verifyNotSelectedLabels**
(selectLocator - equalTo)
- **verifyNotSelectedValues**
(selectLocator - equalTo)
- **verifyNotSelectedOptions**
(selectLocator - equalTo)
- **verifyNotSomethingSelected**
(selectLocator)
- **verifyNotValue**
(locator - equalTo)
- **verifySelectedIds**
(selectLocator - equalTo)
- **verifySelectedIndexes**
(selectLocator - equalTo)
- **verifySelectedLabels**
(selectLocator - equalTo)
- **verifySelectedValues**
(selectLocator - equalTo)
- **verifySelectOptions**
(selectLocator - equalTo)
- **verifySomethingSelected**
(selectLocator)
- **isSomethingSelected**
(selectLocator)
Determines whether some option in a drop-down menu is selected.
- **verifyValue**
(locator - equalTo)
- **assertAllButtons**
(equalTo)
- **assertAllFields**
(equalTo)
- **assertChecked**
(locator)
- **assertCursorPosition**
(locator - equalTo)
- **assertEditable**
(locator)
- **assertNotAllButtons**
- **assertNotAllFields**
- **assertNotChecked**
(locator)
- **assertNotCursorPosition**
(locator - equalTo)
- **assertNotEditable**
(locator)
- **assertNotSelectedIds**
(selectLocator - equalTo)
- **assertNotSelectedIndexes**
(selectLocator - equalTo)
- **assertNotSelectedLabels**
(selectLocator - equalTo)
- **assertNotSelectedValues**
(selectLocator - equalTo)
- **assertNotSelectOptions**
(selectLocator - equalTo)
- **assertNotSomethingSelected**
(selectLocator)
- **assertNotValue**
(locator - equalTo)
- **assertSelectedIds**
(selectLocator - equalTo)
- **assertSelectedIndexes**
(selectLocator - equalTo)
- **assertSelectedLabels**
(selectLocator - equalTo)
- **assertSelectedValues**
(selectLocator - equalTo)
- **assertSelectOptions**
(selectLocator - equalTo)
- **assertSomethingSelected**
(selectLocator)
- **assertValue**
(locator - equalTo)
- **waitForAllWindowIds**
(equalTo)
- **waitForAllWindowNames**
(equalTo)
- **waitForAllWindowTitles**
(equalTo)
- **waitForAttributeFromAllWindows**
(attributeName - equalTo)
- **waitForLocation**
(equalTo)
- **waitForNotAllWindowIds**
(equalTo)
- **waitForNotAllWindowNames**
(equalTo)
- **waitForNotAllWindowTitles**
(equalTo)
- **waitForNotAttributeFromAllWindows**
(attributeName - equalTo)
- **WaitForNotLocation**
(equalTo)
- **waitForNotTitle**
(equalTo)
- **waitForTitle**
(equalTo)
- **storeAllButtons**
- **storeAllFields**
- **storeChecked**
(locator - variable)
- **storeCursorPosition**
(locator - variable)
- **storeEditable**
(locator - variable)
- **storeSelectedIds**
(selectLocator - variable)
- **storeSelectedIndexes**
(selectLocator - variable)
- **storeSelectedLabels**
(selectLocator - variable)
- **storeSelectedValues**
(selectLocator - variable)
- **storeSelectOptions**
(selectLocator - variable)
- **storeSomethingSelected**
(selectLocator - variable)
- **storeValue**
(locator - variable)
## Keyboard Events
- **keyDown**
(locator - keySequence)
Simulates a user pressing a key (without releasing it yet).
- **keyDownAndWait**
(locator - keySequence)
Simulates a user pressing a key (without releasing it yet).
- **keyPress**
(locator - keySequence)
Simulates a user pressing and releasing a key.
- **keyPressAndWait**
(locator - keySequence)
Simulates a user pressing and releasing a key.
- **keyUp**
(locator - keySequence)
Simulates a user releasing a key.
- **keyUpAndWait**
(locator - keySequence)
Simulates a user releasing a key.
- **type**
(locator - value)
Sets the value of an input field, as though you typed it in. Can also be used to set the value of combo boxes, check boxes, etc. In these cases, value should be the value of the option selected, not the visible text.
- **typeAndWait**
(locator - value)
Sets the value of an input field, as though you typed it in. Can also be used to set the value of combo boxes, check boxes, etc. In these cases, value should be the value of the option selected, not the visible text.
- **typeKeys**
(locator - value)
Simulates keystroke events on the specified element, as though you typed the value key-by-key. This is a convenience method for calling keyDown, keyUp, keyPress for every character in the specified string; this is useful for dynamic UI widgets (like auto-completing combo boxes) that require explicit key events. Unlike the simple "type" command, which forces the specified value into the page directly, this command may or may not have any visible effect, even in cases where typing keys would normally have a visible effect. For example, if you use "typeKeys" on a form element, you may or may not see the results of what you typed in the field. In some cases, you may need to use the simple "type" command to set the value of the field and then the "typeKeys" command to send the keystroke events corresponding to what you just typed.
- **sendKeys**
(locator - value)
- **typeKeysAndWait**
(locator - value)
Simulates keystroke events on the specified element, as though you typed the value key-by-key. This is a convenience method for calling keyDown, keyUp, keyPress for every character in the specified string; this is useful for dynamic UI widgets (like auto-completing combo boxes) that require explicit key events. Unlike the simple "type" command, which forces the specified value into the page directly, this command may or may not have any visible effect, even in cases where typing keys would normally have a visible effect. For example, if you use "typeKeys" on a form element, you may or may not see the results of what you typed in the field. In some cases, you may need to use the simple "type" command to set the value of the field and then the "typeKeys" command to send the keystroke events corresponding to what you just typed.
## Keyboard Modifiers
- **altKeyDown**
Press the alt key and hold it down until doAltUp() is called or a new page is loaded.
- **altKeyDownAndWait**
Press the alt key and hold it down until doAltUp() is called or a new page is loaded.
- **altKeyUp**
Release the alt key.
- **altKeyUpAndWait**
Release the alt key.
- **controlKeyDown**
Press the control key and hold it down until doControlUp() is called or a new page is loaded.
- **controlKeyDownAndWait**
Press the control key and hold it down until doControlUp() is called or a new page is loaded.
- **controlKeyUp**
Release the control key.
- **controlKeyUpAndWait**
Release the control key.
- **metaKeyDown**
Press the meta key and hold it down until doMetaUp() is called or a new page is loaded.
- **metaKeyUp**
Release the meta key.
- **metaKeyUpAndWait**
Release the meta key.
- **shiftKeyDown**
Press the shift key and hold it down until doShiftUp() is called or a new page is loaded.
- **shiftKeyDownAndWait**
Press the shift key and hold it down until doShiftUp() is called or a new page is loaded.
- **shiftKeyUp**
Release the shift key.
- **shiftKeyUpAndWait**
Release the shift key.
## Mouse Events
- **mouseDown**
(locator)
Simulates a user pressing the left mouse button (without releasing it yet) on the specified element.
- **mouseDownAndWait**
(locator)
Simulates a user pressing the left mouse button (without releasing it yet) on the specified element.
- **mouseDownAt**
(locator - coordString)
Simulates a user pressing the left mouse button (without releasing it yet) at the specified location.
- **mouseDownAtAndWait**
(locator - coordString)
Simulates a user pressing the left mouse button (without releasing it yet) at the specified location.
- **mouseDownRight**
(locator)
Simulates a user pressing the right mouse button (without releasing it yet) on the specified element.
- **mouseDownRightAndWait**
(locator)
Simulates a user pressing the right mouse button (without releasing it yet) on the specified element.
- **mouseDownRightAt**
(locator - coordString)
Simulates a user pressing the right mouse button (without releasing it yet) at the specified location.
- **mouseDownRightAtAndWait**
(locator - coordString)
Simulates a user pressing the right mouse button (without releasing it yet) at the specified location.
- **mouseMove**
(locator)
Simulates a user pressing the mouse button (without releasing it yet) on the specified element.
- **mouseMoveAndWait**
(locator)
Simulates a user pressing the mouse button (without releasing it yet) on the specified element.
- **mouseMoveAt**
(locator - coordString)
Simulates a user pressing the mouse button (without releasing it yet) on the specified element.
- **mouseMoveAtAndWait**
(locator - coordString)
Simulates a user pressing the mouse button (without releasing it yet) on the specified element.
- **mouseOut**
(locator)
Simulates a user moving the mouse pointer away from the specified element.
- **mouseOutAndWait**
(locator)
Simulates a user moving the mouse pointer away from the specified element.
- **mouseOver**
(locator)
Simulates a user hovering a mouse over the specified element.
- **mouseOverAndWait**
(locator)
Simulates a user hovering a mouse over the specified element.
- **mouseUp**
(locator)
Simulates the event that occurs when the user releases the mouse button (i.e., stops holding the button down) on the specified element.
- **mouseUpAndWait**
(locator)
Simulates the event that occurs when the user releases the mouse button (i.e., stops holding the button down) on the specified element.
- **mouseUpAt**
(locator - coordString)
Simulates the event that occurs when the user releases the mouse button (i.e., stops holding the button down) at the specified location.
- **mouseUpAtAndWait**
(locator - coordString)
Simulates the event that occurs when the user releases the mouse button (i.e., stops holding the button down) at the specified location.
- **mouseUpRight**
(locator)
Simulates the event that occurs when the user releases the right mouse button (i.e., stops holding the button down) on the specified element.
- **mouseUpRightAndWait**
(locator)
Simulates the event that occurs when the user releases the right mouse button (i.e., stops holding the button down) on the specified element.
- **mouseUpRightAt**
(locator - coordString)
Simulates the event that occurs when the user releases the right mouse button (i.e., stops holding the button down) at the specified location.
- **mouseUpRightAtAndWait**
(locator - coordString)
Simulates the event that occurs when the user releases the right mouse button (i.e., stops holding the button down) at the specified location.
## Popups and Menus
- **answerOnNextPrompt**
(answer)
Instructs Selenium to return the specified answer string in response to the next JavaScript prompt [window.prompt()].
- **answerOnNextPromptAndWait**
(answer)
Instructs Selenium to return the specified answer string in response to the next JavaScript prompt [window.prompt()].
- **chooseCancelOnNextConfirmation**
By default, Selenium's overridden window.confirm() function will return true, as if the user had manually clicked OK; after running this command, the next call to confirm() will return false, as if the user had clicked Cancel. Selenium will then resume using the default behavior for future confirmations, automatically returning true (OK) unless/until you explicitly call this command for each confirmation. Take note - every time a confirmation comes up, you must consume it with a corresponding getConfirmation, or else the next selenium operation will fail.
- **chooseCancelOnNextConfirmationAndWait**
By default, Selenium's overridden window.confirm() function will return true, as if the user had manually clicked OK; after running this command, the next call to confirm() will return false, as if the user had clicked Cancel. Selenium will then resume using the default behavior for future confirmations, automatically returning true (OK) unless/until you explicitly call this command for each confirmation. Take note - every time a confirmation comes up, you must consume it with a corresponding getConfirmation, or else the next selenium operation will fail.
- **chooseOkOnNextConfirmation**
Undo the effect of calling chooseCancelOnNextConfirmation. Note that Selenium's overridden window.confirm() function will normally automatically return true, as if the user had manually clicked OK, so you shouldn't need to use this command unless for some reason you need to change your mind prior to the next confirmation. After any confirmation, Selenium will resume using the default behavior for future confirmations, automatically returning true (OK) unless/until you explicitly call chooseCancelOnNextConfirmation for each confirmation. Take note - every time a confirmation comes up, you must consume it with a corresponding getConfirmation, or else the next selenium operation will fail.
- **chooseOkOnNextConfirmationAndWait**
Undo the effect of calling chooseCancelOnNextConfirmation. Note that Selenium's overridden window.confirm() function will normally automatically return true, as if the user had manually clicked OK, so you shouldn't need to use this command unless for some reason you need to change your mind prior to the next confirmation. After any confirmation, Selenium will resume using the default behavior for future confirmations, automatically returning true (OK) unless/until you explicitly call chooseCancelOnNextConfirmation for each confirmation. Take note - every time a confirmation comes up, you must consume it with a corresponding getConfirmation, or else the next selenium operation will fail.
- **contextMenu**
(locator)
Simulates opening the context menu for the specified element (as might happen if the user "right-clicked" on the element).
- **contextMenuAndWait**
(locator)
Simulates opening the context menu for the specified element (as might happen if the user "right-clicked" on the element).
- **contextMenuAt**
(locator - coordString)
Simulates opening the context menu for the specified element (as might happen if the user "right-clicked" on the element).
- **contextMenuAtAndWait**
(locator - coordString)
Simulates opening the context menu for the specified element (as might happen if the user "right-clicked" on the element).
- **selectPopUp**
(windowID)
Simplifies the process of selecting a popup window (and does not offer functionality beyond what selectWindow() already provides). If windowID is either not specified, or specified as "null", the first non-top window is selected. The top window is the one that would be selected by selectWindow() without providing a windowID . This should not be used when more than one popup window is in play. Otherwise, the window will be looked up considering windowID as the following in order: 1) the "name" of the window, as specified to window.open(); 2) a javascript variable which is a reference to a window; and 3) the title of the window. This is the same ordered lookup performed by selectWindow .
- **deselectPopUp**
Selects the main window. Functionally equivalent to using selectWindow() and specifying no value for windowID.
- **getAlert**
Retrieves the message of a JavaScript alert generated during the previous action, or fail if there were no alerts. Getting an alert has the same effect as manually clicking OK. If an alert is generated but you do not consume it with getAlert, the next Selenium action will fail. Under Selenium, JavaScript alerts will NOT pop up a visible alert dialog. Selenium does NOT support JavaScript alerts that are generated in a page's onload() event handler. In this case a visible dialog WILL be generated and Selenium will hang until someone manually clicks OK.
- **getConfirmation**
Retrieves the message of a JavaScript confirmation dialog generated during the previous action. By default, the confirm function will return true, having the same effect as manually clicking OK. This can be changed by prior execution of the chooseCancelOnNextConfirmation command. If an confirmation is generated but you do not consume it with getConfirmation, the next Selenium action will fail. NOTE: under Selenium, JavaScript confirmations will NOT pop up a visible dialog. NOTE: Selenium does NOT support JavaScript confirmations that are generated in a page's onload() event handler. In this case a visible dialog WILL be generated and Selenium will hang until you manually click OK.
- **getPrompt**
Retrieves the message of a JavaScript question prompt dialog generated during the previous action. Successful handling of the prompt requires prior execution of the answerOnNextPrompt command. If a prompt is generated but you do not get/verify it, the next Selenium action will fail. NOTE: under Selenium, JavaScript prompts will NOT pop up a visible dialog. NOTE: Selenium does NOT support JavaScript prompts that are generated in a page's onload() event handler. In this case a visible dialog WILL be generated and Selenium will hang until someone manually clicks OK.
- **getWhetherThisFrameMatchFrameExpression**
(currentFrameString - target)
Determine whether current/locator identify the frame containing this running code. This is useful in proxy injection mode, where this code runs in every browser frame and window, and sometimes the selenium server needs to identify the "current" frame. In this case, when the test calls selectFrame, this routine is called for each frame to figure out which one has been selected. The selected frame will return true, while all others will return false.
- **getWhetherThisWindowMatchWindowExpression**
(currentWindowString - target)
Determine whether currentWindowString plus target identify the window containing this running code. This is useful in proxy injection mode, where this code runs in every browser frame and window, and sometimes the selenium server needs to identify the "current" window. In this case, when the test calls selectWindow, this routine is called for each window to figure out which one has been selected. The selected window will return true, while all others will return false.
## Browser Window
- **verifyAllWindowIds**
(equalTo)
- **verifyAllWindowNames**
(equalTo)
- **verifyAllWindowTitles**
(equalTo)
- **verifyAttributeFromAllWindows**
(attributeName - equalTo)
- **verifyLocation**
(equalTo)
- **verifyNotAllWindowIds**
(equalTo)
- **verifyNotAllWindowNames**
(equalTo)
- **verifyNotAttributeFromAllWindows**
(attributeName - equalTo)
- **verifyNotLocation**
(equalTo)
- **verifyNotTitle**
(equalTo)
- **verifyTitle**
(equalTo)
- **assertAllWindowIds**
(equalTo)
- **assertAllWindowNames**
(equalTo)
- **assertAllWindowTitles**
(equalTo)
- **assertAttributeFromAllWindows**
(attributeName - equalTo)
- **assertLocation**
(equalTo)
- **assertNotAllWindowIds**
(equalTo)
- **assertNotAllWindowNames**
(equalTo)
- **assertNotAllWindowTitles**
(equalTo)
- **assertNotAttributeFromAllWindows**
(attributeName)
- **assertNotLocation**
(equalTo)
- **assertNotTitle**
(equalTo)
- **assertTitle**
(equalTo)
- **assert**
Check that a variable is an expected value. The variable's value will be converted to a string for comparison. The test will stop if the assert fails.
- **validateXml**
Custom command by TestingBot. Validates if the document contents is valid XML.
- **waitForAlert**
(equalTo)
- **waitForAlertNotPresent**
- **waitForAlertPresent**
- **waitForConfirmation**
(equalTo)
- **waitForConfirmationNotPresent**
- **waitForConfirmationPresent**
- **waitForNotAlert**
(equalTo)
- **waitForNotConfirmation**
(equalTo)
- **waitForNotPrompt**
(equalTo)
- **waitForPrompt**
(equalTo)
- **waitForPromptNotPresent**
- **waitForPromptPresent**
- **storeAllWindowIds**
- **storeAllWindowNames**
- **storeAllWindowTitles**
- **storeAttributeFromAllWindows**
(attributeName - variable)
- **storeLocation**
- **storeTitle**
- **storeAllWindowHandles**
- **storeNewWindowHandle**
- **storeWindowHandle**
- **setWindowSize**
(size)
Sets the current window size.
## Cookies
- **verifyCookie**
(equalTo)
- **verifyCookieByName**
(name - equalTo)
- **verifyCookieNotPresent**
(name)
- **verifyCookiePresent**
(name)
- **verifyNotCookie**
(equalTo)
- **verifyNotCookieByName**
(name - equalTo)
- **assertCookie**
(equalTo)
- **assertCookieByName**
(name - equalTo)
- **assertCookieNotPresent**
(name)
- **assertCookiePresent**
(name)
- **assertNotCookie**
(equalTo)
- **assertNotCookieByName**
(name - equalTo)
- **createCookie**
(nameValuePair - optionsString)
Create a new cookie whose path and domain are same with those of current page under test, unless you specified a path for this cookie explicitly.
- **deleteAllVisibleCookies**
Calls deleteCookie with recurse=true on all cookies visible to the current page. As noted on the documentation for deleteCookie, recurse=true can be much slower than simply deleting the cookies using a known domain/path.
- **deleteCookie**
(name - optionsString)
Delete a named cookie with specified path and domain. Be careful; to delete a cookie, you need to delete it using the exact same path and domain that were used to create the cookie. If the path is wrong, or the domain is wrong, the cookie simply won't be deleted. Also note that specifying a domain that isn't a subset of the current domain will usually fail. Since there's no way to discover at runtime the original path and domain of a given cookie, we've added an option called 'recurse' to try all sub-domains of the current domain with all paths that are a subset of the current path. Beware; this option can be slow. In big-O notation, it operates in O(n\*m) time, where n is the number of dots in the domain name and m is the number of slashes in the path.
- **storeCookie**
- **storeCookieByName**
(name - variable)
- **storeCookiePresent**
(name - variable)
## Page Content
- **verifyAllLinks**
(equalTo)
- **verifyAttribute**
(attributeLocator - equalTo)
- **verifyBodyText**
(equalTo)
- **verifyElementNotPresent**
(locator)
- **verifyElementPresent**
(locator)
- **verifyHtmlSource**
(equalTo)
- **verifyNotAllLinks**
(equalTo)
- **verifyNotAttribute**
(attributeLocator)
- **verifyNotBodyText**
(equalTo)
- **verifyNotHtmlSource**
(equalTo)
- **verifyNotOrdered**
(locator1 - locator2)
- **verifyNotTable**
(tableCellAddress - equalTo)
- **verifyNotText**
(locator - equalTo)
- **verifyNotVisible**
(locator)
- **verifyOrdered**
(locator1 - locator2)
- **verifyTable**
(tableCellAddress - equalTo)
- **verifyText**
(locator - equalTo)
- **verifyTextNotPresent**
(pattern)
- **verifyTextPresent**
(pattern)
- **verifyVisible**
(locator)
- **assertAllLinks**
(equalTo)
- **assertAttribute**
(attributeLocator - equalTo)
- **assertBodyText**
(equalTo)
- **assertElementNotPresent**
(locator)
- **assertElementPresent**
(locator)
- **assertHtmlSource**
(equalTo)
- **assertNotAllLinks**
(equalTo)
- **assertNotAttribute**
(attributeLocator - equalTo)
- **assertNotBodyText**
(equalTo)
- **assertNotHtmlSource**
(equalTo)
- **assertNotOrdered**
(locator1 - locator2)
- **assertNotTable**
(tableCellAddress - equalTo)
- **assertNotText**
(locator - equalTo)
- **assertNotVisible**
(locator)
- **assertOrdered**
(locator1 - locator2)
- **assertTable**
(tableCellAddress - equalTo)
- **assertText**
(locator - equalTo)
- **assertTextNotPresent**
(pattern)
- **assertTextPresent**
(pattern)
- **assertVisible**
(locator)
- **waitForAllLinks**
(equalTo)
- **waitForAttribute**
(attributeLocator - equalTo)
- **waitForBodyText**
(equalTo)
- **waitForElementPresent**
(locator)
- **waitForHtmlSource**
(equalTo)
- **waitForNotAllLinks**
(equalTo)
- **waitForNotAttribute**
(attributeLocator - equalTo)
- **waitForNotBodyText**
(equalTo)
- **waitForElementNotPresent**
(locator)
- **waitForElementNotVisible**
(locator)
- **waitForElementVisible**
(locator)
- **waitForNotHtmlSource**
(equalTo)
- **waitForNotOrdered**
(locator1 - locator2)
- **waitForNotTable**
(tableCellAddress - equalTo)
- **waitForNotText**
(locator - equalTo)
- **waitForNotVisible**
(locator)
- **waitForOrdered**
(locator1 - locator2)
- **waitForTable**
(tableCellAddress - equalTo)
- **waitForText**
(locator - equalTo)
- **waitForTextNotPresent**
(pattern)
- **waitForTextPresent**
(pattern)
- **waitForVisible**
(locator)
- **storeAllLinks**
- **storeAttribute**
(attributeLocator - variable)
- **storeBodyText**
- **storeElementPresent**
(locator - variable)
- **storeHtmlSource**
- **storeTable**
(tableCellAddress - variable)
- **storeText**
(locator - variable)
- **storeTextPresent**
(pattern - variable)
- **storeVisible**
(locator - variable)
## Page Positioning
- **verifyElementHeight**
(locator - equalTo)
- **verifyElementIndex**
(locator - equalTo)
- **verifyElementPositionLeft**
(locator - equalTo)
- **verifyElementPositionTop**
(locator - equalTo)
- **verifyElementWidth**
(locator - equalTo)
- **verifyNotElementHeight**
(locator - equalTo)
- **verifyNotElementIndex**
(locator - equalTo)
- **verifyNotElementPositionLeft**
(locator - equalTo)
- **verifyNotElementPositionTop**
(locator - equalTo)
- **verifyNotElementWidth**
(locator - equalTo)
- **assertElementHeight**
(locator - equalTo)
- **assertElementIndex**
(locator - equalTo)
- **assertElementPositionLeft**
(locator - equalTo)
- **assertElementPositionTop**
(locator - equalTo)
- **assertElementWidth**
(locator - equalTo)
- **assertNotElementHeight**
(locator - equalTo)
- **assertNotElementIndex**
(locator - equalTo)
- **assertNotElementPositionLeft**
(locator - equalTo)
- **assertNotElementPositionTop**
(locator - equalTo)
- **assertNotElementWidth**
- **storeElementHeight**
(locator - variable)
- **storeElementIndex**
(locator - variable)
- **storeElementPositionLeft**
(locator - variable)
- **storeElementPositionTop**
(locator - variable)
- **storeElementWidth**
(locator - variable)
## Popups
- **verifyAlert**
(equalTo)
- **verifyAlertNotPresent**
- **verifyAlertPresent**
- **isAlertPresent**
Has an alert occurred? This function never throws an exception
- **verifyConfirmation**
(equalTo)
- **verifyConfirmationNotPresent**
- **verifyConfirmationPresent**
- **isConfirmationPresent**
Has confirm() been called? This function never throws an exception
- **verifyNotAlert**
(equalTo)
- **verifyNotConfirmation**
(equalTo)
- **verifyNotPrompt**
(equalTo)
- **verifyPrompt**
(equalTo)
- **verifyPromptNotPresent**
- **verifyPromptPresent**
- **isPromptPresent**
Has a prompt occurred? This function never throws an exception
- **assertAlert**
(equalTo)
- **assertAlertNotPresent**
- **assertAlertPresent**
- **assertConfirmation**
(equalTo)
- **assertConfirmationNotPresent**
- **assertConfirmationPresent**
- **assertNotAlert**
(equalTo)
- **assertNotConfirmation**
(equalTo)
- **assertNotPrompt**
(equalTo)
- **assertPrompt**
(equalTo)
- **assertPromptNotPresent**
- **assertPromptPresent**
- **waitForElementHeight**
(locator - equalTo)
- **waitForElementIndex**
(locator - equalTo)
- **waitForElementPositionLeft**
(locator - equalTo)
- **waitForElementPositionTop**
(locator - equalTo)
- **waitForElementWidth**
(locator - equalTo)
- **waitForNotElementHeight**
(locator - equalTo)
- **waitForNotElementIndex**
(locator - equalTo)
- **waitForNotElementPositionLeft**
(locator - equalTo)
- **waitForNotElementPositionTop**
(locator - equalTo)
- **waitForNotElementWidth**
(locator - equalTo)
- **waitForPopUp**
(windowID - timeout)
Waits for a popup window to appear and load up.
- **storeAlert**
- **storeAlertPresent**
- **storeConfirmation**
- **storeConfirmationPresent**
- **storePrompt**
- **storePromptPresent**
## Selenium
- **verifyEval**
(script - equalTo)
- **verifyExpression**
(expression - equalTo)
- **verifyMouseSpeed**
- **verifyNotEval**
(script - equalTo)
- **verifyNotExpression**
(expression - equalTo)
- **verifyNotMouseSpeed**
- **verifyNotSpeed**
- **verifyNotXpathCount**
(xpath - equalTo)
- **verifySpeed**
- **verifyXpathCount**
(xpath - equalTo)
- **assertEval**
(script - equalTo)
- **assertExpression**
(expression - equalTo)
- **assertMouseSpeed**
(equalTo)
- **assertNotEval**
(script - equalTo)
- **assertNotExpression**
(expression - equalTo)
- **assertNotMouseSpeed**
(equalTo)
- **assertNotSpeed**
(equalTo)
- **assertNotXpathCount**
(xpath - equalTo)
- **assertSpeed**
(equalTo)
- **assertXpathCount**
(xpath - equalTo)
- **waitForAllButtons**
(equalTo)
- **waitForAllFields**
(equalTo)
- **waitForChecked**
(locator)
- **waitForCursorPosition**
(locator - equalTo)
- **waitForCondition**
(script - timeout)
Runs the specified JavaScript snippet repeatedly until it evaluates to "true". The snippet may have multiple lines, but only the result of the last line will be considered. Note that, by default, the snippet will be run in the runner's test window, not in the window of your application. To get the window of your application, you can use the JavaScript snippet selenium.browserbot.getCurrentWindow(), and then run your JavaScript in there
- **waitForEditable**
(locator)
- **waitForEval**
(script - equalTo)
- **waitForExpression**
(expression - equalTo)
- **waitForMouseSpeed**
(equalTo)
- **waitForNotAllButtons**
(equalTo)
- **waitForNotAllFields**
(equalTo)
- **waitForNotChecked**
(locator)
- **waitForNotCursorPosition**
(locator - equalTo)
- **waitForNotEditable**
(locator)
- **waitForNotEval**
(script - equalTo)
- **waitForNotExpression**
(expression - equalTo)
- **waitForNotMouseSpeed**
(equalTo)
- **waitForNotSelectedIds**
(selectLocator - equalTo)
- **waitForNotSelectedIndexes**
(selectLocator - equalTo)
- **waitForNotSelectedLabels**
(selectLocator - equalTo)
- **waitForNotSelectedValues**
(selectLocator - equalTo)
- **waitForNotSelectOptions**
(selectLocator - equalTo)
- **waitForNotSomethingSelected**
(selectLocator)
- **waitForNotSpeed**
(equalTo)
- **waitForNotValue**
(locator - equalTo)
- **waitForNotXpathCount**
(xpath - equalTo)
- **waitForSelectedIds**
(selectLocator - equalTo)
- **waitForSelectedIndexes**
(selectLocator - equalTo)
- **waitForSelectedLabels**
(selectLocator - equalTo)
- **waitForSelectedValues**
(selectLocator - equalTo)
- **waitForSelectOptions**
(selectLocator - equalTo)
- **waitForSomethingSelected**
(selectLocator)
- **waitForSpeed**
(equalTo)
- **waitForValue**
(locator - equalTo)
- **waitForXpathCount**
(xpath - equalTo)
- **storeEval**
(script - variable)
- **storeExpression**
(expression - variable)
- **store**
(expression - variable)
This command is a synonym for storeExpression.
- **storeMouseSpeed**
- **storeSpeed**
- **storeXpathCount**
(xpath - variable)
## Screenshots
- **captureEntirePageScreenshot**
(filename - kwargs)
Saves the entire contents of the current window canvas to a PNG file. Contrast this with the captureScreenshot command, which captures the contents of the OS viewport (i.e. whatever is currently being displayed on the monitor), and is implemented in the RC only. Currently this only works in Firefox when running in chrome mode, and in IE non-HTA using the EXPERIMENTAL "Snapsie" utility. The Firefox implementation is mostly borrowed from the Screengrab! Firefox extension. Please see http://www.screengrab.org and http://snapsie.sourceforge.net/ for details.
## Selenium Settings
- **waitForCookie**
(equalTo)
- **waitForCookieByName**
(name - equalTo)
- **waitForCookieNotPresent**
(name)
- **waitForCookiePresent**
(name)
- **waitForNotCookie**
(equalTo)
- **waitForNotCookieByName**
(name - equalTo)
- **addLocationStrategy**
(strategyName - functionDefinition)
Defines a new function for Selenium to locate elements on the page. For example, if you define the strategy "foo", and someone runs click("foo=blah"), we'll run your function, passing you the string "blah", and click on the element that your function returns, or throw an "Element not found" error if your function returns null. We'll pass three arguments to your function: locator: the string the user passed in inWindow: the currently selected window inDocument: the currently selected document The function must return null if the element can't be found.
- **allowNativeXpath**
(allow)
Specifies whether Selenium should use the native in-browser implementation of XPath (if any native version is available); if you pass "false" to this function, we will always use our pure-JavaScript xpath library. Using the pure-JS xpath library can improve the consistency of xpath element locators between different browser vendors, but the pure-JS version is much slower than the native implementations.
- **ignoreAttributesWithoutValue**
(ignore)
Specifies whether Selenium will ignore xpath attributes that have no value, i.e. are the empty string, when using the non-native xpath evaluation engine. You'd want to do this for performance reasons in IE. However, this could break certain xpaths, for example an xpath that looks for an attribute whose value is NOT the empty string. The hope is that such xpaths are relatively rare, but the user should have the option of using them. Note that this only influences xpath evaluation when using the ajaxslt engine (i.e. not "javascript-xpath").
- **setBrowserLogLevel**
(logLevel)
Sets the threshold for browser-side logging messages; log messages beneath this threshold will be discarded. Valid logLevel strings are: "debug", "info", "warn", "error" or "off". To see the browser logs, you need to either show the log window in GUI mode, or enable browser-side logging in Selenium RC.
- **setMouseSpeed**
(pixels)
Configure the number of pixels between "mousemove" events during dragAndDrop commands (default=10). Setting this value to 0 means that we'll send a "mousemove" event to every single pixel in between the start location and the end location; that can be very slow, and may cause some browsers to force the JavaScript to timeout. If the mouse speed is greater than the distance between the two dragged objects, we'll just send one "mousemove" at the start location and then one final one at the end location.
- **setSpeed**
(value)
Set execution speed (i.e., set the millisecond length of a delay which will follow each selenium operation). By default, there is no such delay, i.e., the delay is 0 milliseconds.
- **getSpeed**
Get execution speed (i.e., get the millisecond length of the delay following each selenium operation). By default, there is no such delay, i.e., the delay is 0 milliseconds. See also setSpeed.
- **setTimeout**
(timeout)
Specifies the amount of time that Selenium will wait for actions to complete. Actions that require waiting include "open" and the "waitFor\*" actions. The default timeout is 30 seconds.
- **useXpathLibrary**
(libraryName)
Allows choice of one of the available libraries.
## Special
- **addScript**
(scriptContent - scriptTagId)
Loads script content into a new script tag in the Selenium document. This differs from the runScript command in that runScript adds the script tag to the document of the AUT, not the Selenium document. The following entities in the script content are replaced by the characters they represent: \< \> & The corresponding remove command is removeScript.
- **assignId**
(locator - identifier)
Temporarily sets the "id" attribute of the specified element, so you can locate it in the future using its ID rather than a slow/complicated XPath. This ID will disappear once the page is reloaded.
- **echo**
(message)
Prints the specified message into the third table cell in your Selenese tables. Useful for debugging.
- **fireEvent**
(locator - eventName)
Explicitly simulate an event, to trigger the corresponding "onevent" handler.
- **highlight**
(locator)
Briefly changes the backgroundColor of the specified element yellow. Useful for debugging.
- **rollup**
(rollupName - kwargs)
Executes a command rollup, which is a series of commands with a unique name, and optionally arguments that control the generation of the set of commands. If any one of the rolled-up commands fails, the rollup is considered to have failed. Rollups may also contain nested rollups.
- **runScript**
(script)
Creates a new "script" tag in the body of the current test window, and adds the specified text into the body of the command. Scripts run in this way can often be debugged more easily than scripts executed using Selenium's "getEval" command. Beware that JS exceptions thrown in these script tags aren't managed by Selenium, so you should probably wrap your script in try/catch blocks if there is any chance that the script will throw an exception.
- **getEval**
(command)
Gets the result of evaluating the specified JavaScript snippet. The snippet may have multiple lines, but only the result of the last line will be returned. Note that, by default, the snippet will run in the context of the "selenium" object itself, so this will refer to the Selenium object. Use window to refer to the window of your application, e.g. window.document.getElementById('foo') If you need to use a locator to refer to a single element in your application page, you can use this.browserbot.findElement("id=foo") where "id=foo" is your locator.
- **getLocation**
Gets the absolute URL of the current page.
- **getTitle**
Gets the title of the current page.
- **getBodyText**
Gets the entire text of the page.
- **getValue**
(locator)
Gets the (whitespace-trimmed) value of an input field (or anything else with a value parameter). For checkbox/radio elements, the value will be "on" or "off" depending on whether the element is checked or not.
- **getText**
(locator)
Gets the text of an element. This works for any element that contains text. This command uses either the textContent (Mozilla-like browsers) or the innerText (IE-like browsers) of the element, which is the rendered text shown to the user.
- **getTable**
(tableCellAddress)
Gets the text from a cell of a table. The cellAddress syntax tableLocator.row.column, where row and column start at 0.
- **pause**
(time)
Wait for the specified amount of time (in milliseconds)
- **comment**
(text)
Custom command by TestingBot. Add a comment between your test steps. This will show up in the test results.
- **getHtmlSource**
Returns the entire HTML source between the opening and closing "html" tags.
- **stopTest**
(condition)
Custom command by TestingBot. Will stop the test if the supplied condition is met. Other steps, which resulted in an error, after a successful stopTest, will no longer indicate a test as failed.
## Common
- **waitForPageToLoad**
(timeout)
Waits for a new page to load. You can use this command instead of the "AndWait" suffixes, "clickAndWait", "selectAndWait", "typeAndWait" etc. (which are only available in the JS API). Selenium constantly keeps track of new pages loading, and sets a "newPageLoaded" flag when it first notices a page load. Running any other Selenium command after turns the flag to false. Hence, if you want to wait for a page to load, you must wait immediately after a Selenium command that caused a page-load.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/web-automate/codeless-automation/faq
# Selenium IDE FAQ
TestingBot supports importing and running Selenium IDE tests on all our browsers and devices.
Please find some questions and answers below.
## FAQ
- **How do I use `selenium-side-runner` with TestingBot?**
You can easily run your tests with `selenium-side-runner`, try this command for example
```bash
selenium-side-runner --server http://key:secret@hub.testingbot.com/wd/hub -c "browserName='internet explorer' version='11.0' platform='Windows 8.1'" my-test-file.side
```
This will run the test on our Grid on IE11 - the test results will appear in the TestingBot dashboard.
- **Do you support the old Selenium IDE format? What about the .SIDE format?**
We support both versions in our TestLab. You can upload the old format (in `HTML` format) or newly generated tests (in `SIDE` format).
- **Do you support control flow?**
We do not yet support control flow, like `if`, `else`, `while`, ...
If you need this feature, please [contact us](https://testingbot.com/contact/new).
- **Do you support mobile testing with Codeless Automation?**
We support running your tests on our iOS and Android simulators/emulators.
Please consider using `css selectors` as much as possible, as Appium (which we use to control iOS/Android) does not support XPath selectors.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/web-automate/codeless-automation/variables
# Storing Selenium IDE Variables
With Selenium IDE it is possible to `store` variables.
These variables can then be used in later commands during the same test flow.
## How to store a variable
To store a variable during a test, please use the `store` command:
```bash
store - javascript(return "test" + Math.floor(Math.random()*11111) + "@test.com") - emailAddress
```
This example will store a variable called `emailAddress` and its value will be a random `"test....@test.com"` string.
Storing a variable happens in the context of the page. You can use any kind of javascript and reference DOM elements or other functions available on the page. The return value is stored.
## How to use a variable
Once you have stored a variable, you can use it in subsequent steps like this: `${emailAddress}`
## Examples
Below are some examples on how to [store and use variables](https://www.selenium.dev/selenium-ide/docs/en/api/commands#store).
```bash
storeText("css=.container:nth-child(1) > span:nth-child(3)","myVar")
```
This will store the text from element with CSS selector `.container:nth-child(1) > span:nth-child(3)` in a variable called `myVar`.
```bash
type('id=inputField','${myVar}')
```
This will type the text stored in the `myVar` variable to an element with `ID=inputField`
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/ai
# AI Testing
TestingBot provides AI solutions for QA and developers to automate their tests and other tasks. Here are some of the AI solutions we are providing:
- [TestingBot AI Insights](https://testingbot.com/support/ai/insights)
Get an AI explanation of why a test failed: root cause, timeline, evidence and suggested fixes. Works with Selenium, Appium, Playwright, Espresso, XCUITest and Maestro.
- [TestingBot AI Chat](https://testingbot.com/support/ai/chat)
Chat with an AI operator and instruct a remote browser. Generate automated tests from your instructions.
- [Codeless Tests](https://testingbot.com/support/ai/codeless)
Instruct our AI agent what to test. The AI agent will run tests against your website on regular intervals and alert if a test fails.
- [TestingBot MCP Server](https://testingbot.com/support/ai/mcp)
Connect TestingBot with AI models, using the TestingBot MCP Server.
- [Testing WebMCP Tools](https://testingbot.com/support/ai/webmcp)
Test your website's WebMCP tools (navigator.modelContext) on real browsers with Selenium, Playwright and Puppeteer.
- [Other AI integrations such as Browser-Use, Stagehand and Magnitude](https://testingbot.com/support/ai/integrations)
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/ai/insights
# TestingBot AI Insights
TestingBot AI Insights is an AI-powered test failure analysis feature. When an automated test fails, whether it is a Selenium, Appium, Playwright, Puppeteer, Cypress, Espresso, XCUITest or Maestro test, AI Insights reads the test's logs and explains, in plain language, why it failed: the most likely root cause, a timeline of what happened, the supporting evidence from the logs, and a suggested fix, together with a confidence score.
Instead of scrolling through a long command log to work out what went wrong, you open the test and read a short, structured explanation.
_The AI Analysis tab explains a failed test: summary, root cause, likely owner, confidence, timeline, evidence and suggested fixes._
## How to enable it
AI Insights is off by default and must be enabled per account by the account owner. Because your test logs are sent to a third-party AI provider for analysis, enabling it records an explicit consent.
1. Sign in as the **account owner** and go to [Account Settings](https://testingbot.com/members/user/edit#ai-analysis).
2. In the **AI Analysis** section, read what is sent, then tick the consent checkbox and save.
3. The **AI Analysis** tab will now appear on your test detail pages.
Team members cannot enable the feature themselves: it is an account-level setting controlled by the owner. You can disable it again at any time from the same screen, which immediately stops any further data being sent.
## Using AI Insights
Open a test at **Tests → (your test)** and select the **AI Analysis** tab.
- **Failed tests** are analyzed automatically the first time you open the tab. The explanation streams in section by section.
- **Passed tests** are not analyzed automatically, but you can choose **Analyze anyway**.
- Results are **cached** , so reopening a test shows the previous analysis instantly with no extra cost.
- Use **Re-analyze** to run a fresh analysis, for example after you have changed the test.
## What you get
A completed analysis is presented as a set of sections:
- **Summary** : one or two sentences on what the test was doing and how it failed.
- **Root cause** : the single most likely reason for the failure.
- **Likely owner** : where the problem most likely sits, classified as _application_, _test script_, _environment_, _flaky_ or _unknown_.
- **Confidence** : a score from 0 to 100 (shown as low, medium or high). When confidence is low, the likely owner is shown as a tentative "best guess" rather than a firm verdict.
- **Timeline** : the key events leading up to the failure.
- **Evidence** : the specific log lines and commands that support the conclusion.
- **Suggested fixes** : one to three concrete, ranked suggestions.
AI Insights is an assistant, not an oracle. It can be wrong, especially on ambiguous failures such as flaky timing issues or visual differences, where it will tell you the confidence is low. Always verify before acting on a suggestion.
## What data is sent
AI Insights sends a small, curated, text-only slice of the test to the AI provider. No screenshots or video are sent. It is built from artifacts you already have on TestingBot:
- The failing step and the steps around it.
- A filtered, compressed tail of the most relevant driver or device log (for example the Selenium, Appium, Playwright, logcat or iOS log), focused on errors, warnings and the failure window.
- For native runners (Espresso, XCUITest, Maestro), the failing test results and their stacktraces or error messages.
- For Maestro, the flow definition (the YAML that lists the intended steps), which helps explain which step failed and why.
- The test's status, status message and termination reason.
- The environment (browser or device, version and OS).
- For codeless AI tests, the natural-language test intent.
## What gets stripped
Before anything leaves our servers, the curated text passes through an automated masking step that redacts detectable secrets and personal data and replaces them with placeholders such as ``:
- API keys and tokens (including JWTs, AWS, Stripe, GitHub, Google and similar key formats).
- Passwords, including values typed into password fields during the test.
- `Authorization`, `Cookie` and similar sensitive HTTP headers.
- Private keys and other high-entropy secrets.
- Email addresses and card-number-shaped values.
This is best-effort masking (pseudonymization), not anonymization. It significantly reduces what is shared, but it cannot guarantee that every piece of sensitive data is removed. Only enable AI Insights for tests you are authorized to share, and prefer keeping secrets out of test logs in the first place. See our guidance on [handling sensitive data](https://testingbot.com/support/web-automate/selenium/sensitive-data).
## Privacy and data handling
- **AI provider** : analysis is performed by **Anthropic** (United States), our AI sub-processor, under their standard Data Processing Addendum with EU Standard Contractual Clauses.
- **No training** : Anthropic does not use data sent through their commercial API to train their models.
- **Short retention at the provider** : inputs and outputs are deleted by Anthropic within 30 days by default.
- **Consent** : the feature is off until the account owner explicitly enables it, and it can be disabled at any time.
- **Visual tests** : [AI region names](https://testingbot.com/support/visual-testing/stabilize-screenshots#ai-region-names), which describe what changed in a failed visual run, are a separate setting with their own consent. They send small crops of the screenshot and baseline, which are not masked, and are not turned on by enabling AI Insights.
- **Retention on TestingBot** : a generated analysis is stored with the test and is removed when the test is pruned, in line with our standard 30-day retention of test logs and assets.
## Availability and limits
- There are per-account daily and monthly limits to keep usage and cost bounded.
- Tests older than 30 days cannot be analyzed because their logs have been pruned.
- Tests that are still running cannot be analyzed until they complete.
## Frequently Asked Questions
### Which frameworks does AI Insights support?
AI Insights works with Selenium, Appium, Playwright, Puppeteer, Cypress, Espresso, XCUITest and Maestro tests, as well as codeless AI tests. For web and Appium tests it reads the command log and driver logs; for native mobile runners it reads the test results and stacktraces, the device log, and (for Maestro) the flow definition. Open the failing test or run and select the AI Analysis tab.
### Will it analyze passing tests?
Not automatically. Passing tests show an "Analyze anyway" option if you want an explanation.
### How accurate is it?
It is reliable for clear-cut failures such as assertion errors, element-not-found, timeouts and HTTP errors, and less certain for ambiguous cases such as flaky timing or visual differences. The confidence score reflects this, and low-confidence verdicts are clearly marked.
### How do I turn it off?
The account owner can disable it from [Account Settings](https://testingbot.com/members/user/edit#ai-analysis). Disabling it immediately stops any further data being sent.
### Can I keep secrets out of the analysis entirely?
Yes. We mask detectable secrets automatically, but the best practice is to avoid logging sensitive values. See [handling sensitive data](https://testingbot.com/support/web-automate/selenium/sensitive-data).
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/ai/integrations
# TestingBot's Remote Browser Cloud
TestingBot offers a battle-tested remote browser cloud, available to run both automated tests and AI tasks.
When it comes to running AI-driven browser automation tasks, selecting the right remote browser grid is crucial. **TestingBot** offers a scalable, feature-rich alternative to platforms like Browserbase, Steel.dev, Browserless and Lightpanda.
At TestingBot we've been operating a browser grid since 2012, providing unmatched reliability, global coverage and enterprise-grade security.
## Comparison Table
| Feature | TestingBot | Browserbase | Steel.dev | Browserless | Lightpanda |
| --- | --- | --- | --- | --- | --- |
| Real Browsers & Devices | ✅ Yes | ❌ No | ❌ No | ❌ No | ❌ No |
| Headless Browsers | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes |
| Global Testing Locations | ✅ Yes | ✅ Yes | ❌ No | ❌ No | ❌ No |
| Parallel Sessions | ✅ Unlimited | ❌ Limited | ✅ Yes | ❌ Limited | ❌ Limited |
| Security & Compliance | ✅ Enterprise-Grade | ❌ Basic | ❌ Basic | ❌ Basic | ❌ Basic |
## AI Integrations
TestingBot is capable of connecting remote browsers to various AI frameworks and libraries, including the following popular options:
- [AgentKit](https://testingbot.com/support/ai/integrations/agentkit) - Create AI agents that can run browser automation tasks using TestingBot's remote browser grid and AgentKit.
- [Browser-Use](https://testingbot.com/support/ai/integrations/browser-use) - A Python library for browser automation using AI agents.
- [Braintrust](https://testingbot.com/support/ai/integrations/braintrust) - Create tools to leverage AI for testing and automation.
- [Stagehand](https://testingbot.com/support/ai/integrations/stagehand) - An AI Browser Automation Framework that allows you to create and manage browser automation tasks using AI.
- [Magnitude](https://testingbot.com/support/ai/integrations/magnitude) - An AI framework using Vision to perform automated browser automation with LLMs.
- [Notte](https://testingbot.com/support/ai/integrations/notte) - A reliable and fast AI browser agent.
## Scraping data with real browsers
Looking to obtain data from the world wide web with real browsers? You can connect your [Puppeteer](https://testingbot.com/support/web-automate/puppeteer) and [Playwright](https://testingbot.com/support/web-automate/playwright) scripts to the large TestingBot browser grid, or use functions to perform tasks without writing any code:
- [Take screenshots](https://testingbot.com/support/functions/screenshot)
- [Generate PDFs](https://testingbot.com/support/functions/pdf)
- [Scrape Webpages](https://testingbot.com/support/functions/scrape)
- [Serverless Functions](https://testingbot.com/support/functions/serverless)
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/ai/integrations/agentkit
# Using AgentKit with TestingBot
Create AI agents that can run browser automation tasks using TestingBot's remote browser grid and AgentKit (by inngest).
## Dependencies
To install all dependencies, please follow these steps:
[npm](https://testingbot.com#) [pnpm](https://testingbot.com#) [yarn](https://testingbot.com#)
```bash
npm install @inngest/agent-kit playwright-core
```
```bash
pnpm add @inngest/agent-kit playwright-core
```
```bash
yarn add @inngest/agent-kit playwright-core
```
## Create a TestingBot service
In the example below, we'll show an example tool that will use Algolia to search Hacker News (HN) posts. The TestingBot remote browser grid will be used to perform the actions and return the results.
The example below uses the `chromium.connect` method to connect to a remote browser in the TestingBot cloud.
There are a lot of [Playwright Options](https://testingbot.com/support/web-automate/playwright/options) you can use to customize the browser session.
Make sure to replace `key` and `secret` with your TestingBot API key and secret.
```javascript
import "dotenv/config";
import {
anthropic,
createAgent,
createNetwork,
createTool,
} from "@inngest/agent-kit";
import { createServer } from "@inngest/agent-kit/server";
import { z } from "zod";
import { chromium } from "playwright-core";
const testingBotKey = process.env.TESTINGBOT_KEY;
const testingBotSecret = process.env.TESTINGBOT_SECRET;
const searchHN = createTool({
name: "search_hn",
description: "Search HN posts and comments",
parameters: z.object({
query: z.string().describe("The search query for HN"),
}),
handler: async ({ query }, { step }) => {
return await step?.run("search-on-hn", async () => {
const browser = await chromium.connect({
wsEndpoint: `wss://cloud.testingbot.com/playwright?key=${testingBotKey}&secret=${testingBotSecret}&browserName=chrome&browserVersion=latest`,
});
try {
const page = await browser.newPage();
// Construct the search URL
const searchUrl = `https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=${query}&sort=byPopularity&type=story`;
await page.goto(searchUrl);
// Wait for results to load
await page.waitForSelector(".SearchResults", { timeout: 10000 });
// Extract search results
const results = await page.evaluate(() => {
const posts = document.querySelectorAll(".SearchResults .Story");
return Array.from(posts).map((post) => ({
title: post.querySelector(".Story_title span")?.textContent?.trim(),
}));
});
return results.slice(0, 5); // Return top 5 results
} finally {
await browser.close();
}
});
},
});
// Create the search agent
const searchAgent = createAgent({
name: "hn_searcher",
description: "An agent that searches HN for relevant information",
system:
"You are a helpful assistant that searches HN through algolia for relevant information.",
tools: [searchHN],
});
// Create the network
const hnSearchNetwork = createNetwork({
name: "hn-search-network",
description: "A network that searches HN using TestingBot",
agents: [searchAgent],
maxIter: 2,
defaultModel: anthropic({
model: "claude-sonnet-4-5",
defaultParameters: {
max_tokens: 4096,
},
}),
});
// Create and start the server
const server = createServer({
networks: [hnSearchNetwork],
});
server.listen(3010, () =>
console.log("HN Search Agent server is running on port 3010")
);
```
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/ai/integrations/braintrust
# Using BrainTrust with TestingBot
## Braintrust Organization
To get started, make sure you create [a new Braintrust organization](https://www.braintrust.dev/docs/guides/projects). Next, install the necessary dependencies:
## Install dependencies
To use Braintrust with TestingBot, you need to install the following dependencies:
```bash
npm install zod braintrust playwright-core
```
## Variables
To use TestingBot with Braintrust, you need to set the following environment variables: `TESTINGBOT_KEY` and `TESTINGBOT_SECRET`.
You will also need to set up the AI API Key that you are using. Braintrust supports various AI providers, such as OpenAI, Anthropic and others.
## Tools
Braintrust allows you to create [tools](https://www.braintrust.dev/docs/guides/functions/tools) that can be used by agents. In this example, we will create a tool that takes a URL parameter and will use Readability to convert the HTML.
```javascript
import * as braintrust from "braintrust";
import { z } from "zod";
import { chromium } from "playwright-core";
const testingBotKey = process.env.TESTINGBOT_KEY;
const testingBotSecret = process.env.TESTINGBOT_SECRET;
async function loadPage({ url }: { url: string }) {
const browser = await chromium.connectOverCDP(
`wss://cloud.testingbot.com/?key=${testingBotKey}&secret=${testingBotSecret}&browserName=chrome&browserVersion=latest`
);
const defaultContext = browser.contexts()[0];
const page = await defaultContext.newPage();
await page.goto(url);
const readable = await page.evaluate(async () => {
const readability = await import('https://cdn.skypack.dev/@mozilla/readability');
return new readability.Readability(document).parse();
});
const text = `${readable?.title ?? ''}\n${readable?.textContent ?? ''}`;
await browser.close();
return { page: text };
}
const project = braintrust.projects.create({ name: "TestingBot API Tool" });
project.tools.create({
handler: loadPage,
parameters: z.object({
url: z.string(),
}),
returns: z.object({
page: z.string(),
}),
name: "Load page",
slug: "load-page",
description: "Load a page from the internet",
ifExists: "replace",
});
```
## Deploy tool
You can now deploy the tool to Braintrust. You can do this by running the following command:
```bash
npx braintrust push testingbot.ts
```
Once the tool is deployed, you will see the tool listed in the Library's Tools section. Now you can specify a URL and use the "Load page" tool to use the TestingBot tool.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/ai/integrations/browser-use
# Browser Use Cloud API
Browser-Use is a Python library that allows you to create browser automation scripts using AI. You can use multiple models such as GPT-4o, DeepSeek, Gemini-2.0, Claude and more. Local models such as Qwen 2.5 are also supported. The library essentially uses a Playwright browser instance to execute tasks on a (remote) browser. You can perform tasks such as:
- Navigate to Google and search for TestingBot
- Go to Reddit and open an article
- Open a new tab and go to CNN
By using the TestingBot remote browser grid, you can run multiple AI agents in parallel without worrying about setting up and managing the browsers yourself.
Additionally, TestingBot records each session with a video, logs and generated metadata are saved as well.
You can run browsers from [various geographical locations](https://testingbot.com/support/web-automate/playwright/options#geo) and use different browser versions, on multiple operating systems.
To install browser use, please follow these steps:
- Install the Browser-Use library: `pip install browser-use`
- Install the Playwright library: `pip install playwright`
## Connecting to a remote browser
Please see the example below on how to connect your Browser-Use agent to a remote browser in the TestingBot cloud.
The `wss` URL points to the TestingBot cloud, where you can specify the browser and platform you want to use. You can also specify the name of the session.
Make sure to replace `key` and `secret` with your TestingBot API key and secret.
```python
import json
import httpx
import os
import asyncio
import logging
from browser_use.browser import BrowserSession
from browser_use.agent.service import Agent
from browser_use.llm import ChatOpenAI
capabilities = {
"tb:options": {
"name": "Browser Use Example",
"key": os.environ["TESTINGBOT_KEY"],
"secret": os.environ["TESTINGBOT_SECRET"],
},
"browserName": "chrome",
"browserVersion": "latest",
"platform": "WIN10"
}
async def create_testingbot_session():
try:
async with httpx.AsyncClient(timeout=600.0) as client:
response = await client.post(
"https://cloud.testingbot.com/session",
json={ "capabilities": capabilities },
)
response.raise_for_status()
session_data = response.json()
logging.info(
"TestingBot session created successfully: %s",
json.dumps(session_data, indent=2)
)
return session_data
except Exception as e:
logging.error("Failed to create TestingBot session: %s", e)
async def main():
session_data = await create_testingbot_session()
cdp_url = session_data["cdp_url"]
browser_session = BrowserSession(cdp_url=cdp_url)
agent = Agent(
task="Go to the TestingBot homepage and click the pricing menu item",
llm=ChatOpenAI(model="gpt-4o"),
browser_session=browser_session,
)
try:
await agent.run()
finally:
await browser_session.close()
if __name__ == " __main__":
asyncio.run(main())
```
When you run this example, the Browser-Use agent will connect to a remote browser in the TestingBot cloud, and execute the task specified in the `task` variable.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/ai/integrations/stagehand
# TestingBot & Stagehand
Stagehand is an AI Browser Automation Framework that allows you to create and manage browser automation tasks using AI. It is available in both NodeJS and Python.
By connecting Stagehand to TestingBot's remote browser grid, you can run multiple AI-driven browser automation tasks in parallel, without the need to manage the browsers yourself. Each session is video recorded and available for replay in the TestingBot dashboard. You can run browsers from [various geographical locations](https://testingbot.com/support/web-automate/playwright/options#geo) and use different browser versions, on multiple operating systems.
## Connecting to a remote browser
To get started, please install these packages:
- Install the Stagehand library: `@browserbasehq/stagehand`
- Install the TestingBot library: `npm install testingbot-api`
We can now run a simple example. This will connect Stagehand with a remote TestingBot browser session. Use the AI prompt to test a specific scenario, by describing what you want to test in your own natural language.
```javascript
const { Stagehand } = require("@browserbasehq/stagehand");
const TestingBot = require("testingbot-api");
(async () => {
const tb = new TestingBot({
api_key: process.env.TESTINGBOT_KEY,
api_secret: process.env.TESTINGBOT_SECRET
});
const options = {
capabilities: {
browserName: 'chrome',
browserVersion: 'latest',
platform: 'WIN11'
}
};
const session = await tb.createSession(options);
const stagehand = new Stagehand({
env: "LOCAL",
localBrowserLaunchOptions: {
cdpUrl: session['cdp_url']
}
});
await stagehand.init();
const page = stagehand.page;
await page.goto("https://testingbot.com/pricing");
const extractResult = await page.extract("Extract the pricing plans from this page");
console.log(`Extract result:\n`, extractResult);
})();
```
Make sure to replace `TESTINGBOT_KEY` and `TESTINGBOT_SECRET` with your actual TestingBot API key and secret.
In the example above, we create a new TestingBot session with the desired capabilities. We then pass the `cdpUrl` to Stagehand, which allows it to connect to the remote browser session on TestingBot.
You can now use Stagehand's AI capabilities to interact with the page, extract information, or perform other actions as needed.
For more information on using Stagehand, please refer to the [Stagehand documentation](https://docs.stagehand.dev/first-steps/quickstart).
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/ai/integrations/magnitude
# Magnitude AI
[Magnitude](https://github.com/magnitudedev/magnitude) is an AI browser automation framework that allows you to run browser automation scripts using AI models. It uses Vision AI to understand the webpage and interact with it, making it possible to automate tasks that require visual understanding.
You can use the framework to automate tasks on webpages, test functionality, extract data and verify any specific behavior on the webpage, just like a normal user would do.
## Running Magnitude AI tests on TestingBot
To run Magnitude AI tests on TestingBot, you can use the following steps and example code to get started:
- `npx create-magnitude-app`
- Next, modify the `src/index.tsx` file that was generated, add a `browser` option to the `startBrowserAgent` function.
```javascript
import { startBrowserAgent } from "magnitude-core";
import z from 'zod';
import dotenv from 'dotenv';
dotenv.config();
const capabilities = {
'tb:options': {
key: process.env.TESTINGBOT_KEY,
secret: process.env.TESTINGBOT_SECRET,
},
browserName: 'chrome',
browserVersion: 'latest'
}
async function main() {
const agent = await startBrowserAgent({
// Starting URL for agent
url: 'https://docs.magnitude.run/getting-started/quickstart',
// Show thoughts and actions
narrate: true,
browser: {
cdp: `wss://cloud.testingbot.com/puppeteer?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}`
}
});
// Intelligently extract data based on the DOM content matching a provided zod schema
const gettingStarted = await agent.extract('Extract how to get started with Magnitude', z.object({
// Agent can extract existing data or new insights
difficulty: z.enum(['easy', 'medium', 'hard']),
steps: z.array(z.string()),
}));
// Navigate to a new URL
await agent.nav('https://magnitasks.com');
// Magnitude can handle high-level tasks
await agent.act('Create a task', {
// Optionally pass data that the agent will use where appropriate
data: {
title: 'Get started with Magnitude',
description: gettingStarted.steps.map(step => `• ${step}`).join('\n')
}
});
// It can also handle low-level actions
await agent.act('Drag "Get started with Magnitude" to the top of the in progress column');
// Stop agent and browser
await agent.stop();
}
main();
```
When you run this example, the Magnitude AI agent will use a remote TestingBot browser to interact with. Results will appear on your machine.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/ai/integrations/notte
# Notte Browser Agent
[Notte](https://github.com/nottelabs/notte) is Browser AI agent that allows you to run browser automation scripts using AI models. It is a web agent framework built for speed, cost-efficiency, scale and reliability.
You can use the framework to automate tasks on webpages, test functionality, extract data and verify any specific behavior on the webpage, just like a normal user would do.
## Running Notte Browser Agent tasks with TestingBot
Notte allows you to specify a `CDP` endpoint, enabling it to connect to remote browser instances.
To run Notte Browser Agent tasks on TestingBot, you can use the following steps and example code to get started:
- `pip install notte 'litellm[proxy]' python-dotenv`
- Add your Gemini API credentials and TestingBot credentials to the `.env` file:
```bash
GEMINI_API_KEY="..."
TESTINGBOT_KEY="api_key"
TESTINGBOT_SECRET="api_secret"
```
```python
import os
import notte
from dotenv import load_dotenv
load_dotenv()
key = os.environ["TESTINGBOT_KEY"]
secret = os.environ["TESTINGBOT_SECRET"]
cdp_url = f"wss://cloud.testingbot.com/?key={key}&secret={secret}&browserName=chrome&browserVersion=latest"
with notte.Session(cdp_url=cdp_url) as session:
agent = notte.Agent(session=session, reasoning_model='gemini/gemini-2.5-flash', max_steps=30)
response = agent.run(task="extract pricing plans from https://testingbot.com")
```
You can find more CDP options on [this page](https://testingbot.com/support/web-automate/puppeteer/options).
When you run this example, a remote browser session will be started on TestingBot. Notte will use the Gemini LLM to interact with the remote browser and run the task you specified.
By using TestingBot as the CDP endpoint, you can run Notte Browser Agent tasks on any of the available browsers in the TestingBot cloud, with speed, scale and reliability.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/ai/chat
# What is TestingBot AI Chat?
TestingBot AI Chat is an innovative, codeless test automation tool that harnesses Generative AI and Large Language Models (LLMs) to let users interact with an AI agent in a chat-based interface. Instead of writing complex scripts or manually locating elements, simply type a prompt and the AI agent will operate a remote browser in real time.
Your prompts can be transformed into automated test cases, allowing you to reuse them for continuous testing of your website or web application. Additionally, the AI agent can generate Selenium WebDriver, Playwright or Puppeteer test scripts based on its recorded interactions with the remote browser.
To get started, upgrade your account and go to the [TestingBot member dashboard](https://testingbot.com/members). Click **AI Chat** in the left-hand menu to launch the AI Chat interface.
https://www.youtube.com/embed/dEu7sKL0pMo?si=vaAP1L51M37z244y&rel=0
## Example Prompts
Below are some example prompts that you can use with TestingBot AI Chat:
- ```
Navigate to https://cnn.com and click on the first article. Verify that it loads correctly. Generate a Playwright test for this.
```
- ```
Go to https://testingbot.com/pricing and click on the Enterprise plan. Verify that the pricing details are correct. Generate a Puppeteer test for this.
```
- ```
Open a new tab and go to https://github.com. Search for "TestingBot" and click on the first result. Verify that the page loads correctly. Generate a Selenium WebDriver test for this.
```
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/ai/codeless
# AI Codeless Web Testing
TestingBot provides AI solutions for QA and developers to automate their tests and other tasks. Simply instruct our AI agent what to test and the AI agent will create and run tests against your website on regular intervals, alerting you if a test fails.
Enter a URL and a prompt to instruct the AI agent what to test. The AI agent will run the test on a real browser in the TestingBot cloud.
When an error is detected, the AI agent will alert you.
Instead of generating test code from ChatGPT or Claude, you can instruct the AI agent to run tests based on natural language prompts.
Unlike traditional test scripts, which can break as soon as the website's UI changes, AI-driven tests dynamically interpret your instructions each time they run. This ensures your tests remain adaptable and up-to-date, even as the website evolves.
## Getting Started
To get started, please go to the [Codeless Testing](https://testingbot.com/members/lab) section and click the **Create a new AI test** button.
[Create a new AI test](https://testingbot.com/members/lab) 
### Create a new test
You will see the form below, where you can add a name for the test and the webpage you want to start the test with.
In the textarea you can write the test prompt in your own language. You can find example prompts in the next [prompts section](https://testingbot.com#prompts).
## Prompts
The AI agent will run tests based on the prompts you provide. A prompt is a set of instructions for the AI agent to follow.
We recommend keeping test prompts as concise as possible. For example, you can instruct the AI agent to:
- Click on a button
- Fill in a form
- Test a checkout flow
- Verify that an element is visible or not visible
- Verify that an element contains specific text
- Verify that an element does not contain specific text
- Verify that an element has or does not have a specific attribute
### Example Prompts
Below are some example prompts you could use to test a website:
#### User Authentication & Account Management
- Click on the "Sign In" button and verify that the login form appears.
- Try logging in with an incorrect password and confirm that an error message appears.
#### Forms & Input Validation
- Fill in the registration form with valid data and submit it.
- Try submitting the registration form without entering required fields and verify that validation errors appear.
- Enter an invalid email format and check that an error message is displayed.
#### Navigation & UI Elements
- Click on the "Home" link in the navigation menu and ensure it redirects correctly.
- Open the mobile menu and confirm that all expected links are visible.
- Scroll down the homepage and check that the footer loads correctly.
- Hover over the dropdown menu and verify that all submenu items appear.
#### E-Commerce & Payment Testing
- Add a product to the cart and verify that the cart updates.
- Enter invalid credit card details and ensure an error message appears.
#### Content & Accessibility
- Check that all images on the homepage load properly.
- Verify that all links on the website are clickable and lead to the correct pages.
- Use keyboard navigation to tab through elements and ensure accessibility compliance.
- Enable dark mode (if available) and confirm that the UI adjusts correctly.
- Resize the browser window and confirm that the website is responsive.
#### Performance & Security
- Try opening the website in an incognito window and verify that session-based content does not persist.
- Attempt to submit a form with script injection and confirm that it is sanitized.
- Check if the website forces HTTPS and does not allow unsecured connections.
#### Multilingual & Localization Testing
- Switch the website language to French and confirm that all text updates correctly.
- Change the currency on the pricing page and verify that the prices are updated.
## Browsers
You can select one or more browsers to run your tests on. The AI agent will run the tests on the selected browsers.
Run your tests on the latest versions of Chrome and Edge, on Windows and Linux.
## Schedule
Indicate when and at which interval, you would like to run the test.
You can choose to run the test once, on a daily basis, weekly, or repeat every xx minutes/hours.
## Alerting
When a test fails, you can choose to be alerted via e-mail, API call (webhook) or SMS text message.
## API
You can use the TestingBot REST-API to [create AI tests from prompts](https://testingbot.com/support/api/codeless#createtest). Create one or more tests with your prompt, assign browsers and set a schedule.
Use [webhooks](https://testingbot.com/support/integrations/webhooks) to get notified of the results.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/ai/mcp
# What is TestingBot MCP?
The TestingBot MCP server (Model Context Protocol) is a small program that runs locally on your machine and connects your AI agent or Large Language Model to TestingBot's testing infrastructure. Your AI client sends natural-language requests, and the server turns them into actions: it calls the TestingBot API to manage your account and tests, and it drives real desktop browsers, mobile browsers and native apps in the cloud.
It works with any MCP-compatible client (Claude, Cursor, VS Code, Windsurf and more) and is model-agnostic. You do not write any test code or call any API directly. You describe what you want, and the agent picks the right tools.
Looking to test the WebMCP tools your own website exposes to in-browser AI agents? That is a different feature. See [Testing WebMCP Tools](https://testingbot.com/support/ai/webmcp).
## What you can do
The TestingBot MCP server ships as two bundled packages that together expose a broad set of tools. They fall into three broad areas.
### Account & test management
Query and manage everything in your TestingBot account through conversation, without opening the dashboard:
- **Tests** : list recent tests, read full session details, update metadata, mark passed/failed, stop or delete tests.
- **Builds** : list builds, see the tests inside a build, and delete builds.
- **Browsers & devices** : list available desktop browsers, platforms and mobile devices (real and emulators/simulators).
- **Screenshots** : capture a URL across multiple browsers and retrieve the resulting images.
- **Storage** : upload native mobile apps (`.apk`/`.ipa`/`.zip`) for testing.
- **Team & user** : view account info, team settings and usage, and update your profile.
- **Tunnels & logs** : list/stop tunnels and fetch session failure logs for debugging.
- **Project scaffolding** : scan a project for test files and generate a ready-to-paste TestingBot config for your framework.
### App Automate: Maestro, Espresso and XCUITest
The agent can run a full native-app suite end to end, without you touching a CI config. It uploads the build, uploads (or writes) the tests, picks a device, starts the run and reports back:
- **[Maestro](https://testingbot.com/support/app-automate/maestro)**: upload an app and its flow YAML (from a directory, a zip, or written inline by the agent), run it on real devices, emulators or simulators, filter by tag, shard across devices, then pull per-flow results, video, screenshots and device logs. Failed flows can be fixed and retried individually.
- **[Espresso](https://testingbot.com/support/app-automate/espresso)**: run a pre-built instrumented Android suite, optionally narrowed to specific test classes, packages or annotations.
- **[XCUITest](https://testingbot.com/support/app-automate/xcuitest)**: run a zipped XCUITest runner bundle against your `.ipa` on iOS devices.
All three are asynchronous: the run starts, the agent polls it, and fetches the JUnit report and failure detail when it finishes. Nothing blocks while devices execute.
### Live browser & mobile automation
Beyond account management, the bundled automation package lets the agent drive real sessions step by step:
- **Desktop browsers** : open real Chrome, Firefox, Edge, Safari or IE on Windows, macOS and Linux, then navigate, click, type, read text, run JavaScript and take screenshots.
- **Mobile browsers** : drive Chrome/Safari on Android and iOS, on emulators/simulators (the cheaper default) or on physical real devices.
- **Native mobile apps** : launch and control uploaded iOS/Android apps with Appium.
- **Interactive live sessions** : start a manual live testing session, or watch and take over any automated session via its live-view URL.
- **CDP sessions** : get a remote browser with a Chrome DevTools Protocol endpoint to drive with Playwright or Puppeteer.
For the complete list of tools and their parameters, see the [Tool reference](https://testingbot.com/support/ai/mcp/tools). For end-to-end examples, see [Use cases & workflows](https://testingbot.com/support/ai/mcp/use-cases).
## How it works
The MCP server runs as a local process, started by your AI client over standard input/output. When you type a request, your AI client decides which tools to call and sends those calls to the server. The server then does one of three things:
1. For account and test operations, it makes authenticated calls to the TestingBot API and returns the results to your client in a readable format.
2. For automation, it spins up a real cloud session (desktop or mobile) and issues WebDriver/Appium commands against it, returning a `sessionId` and a live-view URL so you can follow along.
3. For App Automate runs (Maestro, Espresso, XCUITest), it uploads your app and tests, queues the run on TestingBot's device cloud and hands back a run id. The run executes on our infrastructure while your agent polls it, so a long suite never ties up the conversation.
The MCP server is distributed as `@testingbot/mcp-server`, which handles account and orchestration tools and bundles `@testingbot/automation-mcp` for live browser and mobile automation. You install just the one package, both sets of tools become available automatically.
Every automation session returns a live-view URL in the form `https://testingbot.com/tests//live?auth=` so you can watch the session run, or step in manually.
Your TestingBot credentials stay on your local machine. The MCP server talks directly to the TestingBot API, your keys are never sent to the LLM provider. Read more on the [Authentication](https://testingbot.com/support/ai/mcp/authentication) page.
## Supported clients
The TestingBot MCP server works with any MCP-compatible client through a standard stdio configuration. Tested clients include:
- Claude Desktop and Claude Code
- Cursor
- VS Code (Copilot / MCP)
- Windsurf
- Any other MCP-compatible client (generic stdio config)
1-click installers are available for VS Code and Cursor, and Claude can install the packaged `.mcpb` bundle. See the cards in the [Quick start](https://testingbot.com#quick-start) below, or the full [Setup & installation](https://testingbot.com/support/ai/mcp/setup) guide for per-client configuration.
## Quick start
Getting up and running takes three steps.
### 1. Install
Use one of the 1-click installers below, or add the server to your client's MCP configuration manually. The server runs through `npx`, so there is nothing to install globally:
```bash
npx -y @testingbot/mcp-server@latest
```
In your MCP client, that maps to the standard configuration block:
```json
{
"mcpServers": {
"testingbot": {
"command": "npx",
"args": ["-y", "@testingbot/mcp-server@latest"],
"env": {
"TESTINGBOT_KEY": "",
"TESTINGBOT_SECRET": ""
}
}
}
}
```
For 1-click setup, pick your editor below:
#### Visual Studio Code
Install the TestingBot MCP server for Visual Studio Code with a single click. This will automatically configure the MCP settings in your VS Code environment.
[Install for VS Code](https://testingbot.com/mcp/install?client=vscode)
#### Cursor
Install the TestingBot MCP server for Cursor with a single click. This will automatically configure the MCP settings in your Cursor editor.
[Install for Cursor](https://testingbot.com/mcp/install?client=cursor)
#### Claude
Install the `mcp-server.mcpb` file from the [MCP releases page](https://github.com/testingbot/mcp-server/releases). Claude will automatically set up the TestingBot MCP server.
For per-client configuration file locations and step-by-step instructions for every supported client, see the full [Setup & installation](https://testingbot.com/support/ai/mcp/setup) guide.
### 2. Authenticate
Once the server is configured, connect it to your TestingBot account. The recommended way is the `tb_login` tool, just ask your agent to log in, and a browser window opens for you to authorize. Nothing is pasted, and your credentials are written to `~/.testingbot/credentials` on your machine and used automatically afterwards.
Alternatively, set `TESTINGBOT_KEY` and `TESTINGBOT_SECRET` in the client's `env` block (shown in the config above), useful for CI and shared configs. Get your keys from your [account settings](https://testingbot.com/members/user/edit). See [Authentication](https://testingbot.com/support/ai/mcp/authentication) for browser login, device-code login for remote/headless machines, and credential storage details.
### 3. Ask your agent
That is it. Describe what you want in plain language and the agent will pick the right tools. For example:
> "Open Chrome on Windows 11, go to https://example.com, and tell me the page title."
> "List my last 10 TestingBot tests and show me the details of any that failed."
> "Run the Maestro flows in ./flows against ./app.apk on a real Pixel 8, then tell me which ones failed and why."
Browse many more end-to-end examples in [Use cases & workflows](https://testingbot.com/support/ai/mcp/use-cases).
## Documentation index
Dive deeper into any part of the TestingBot MCP server:
- [**Setup & installation**](https://testingbot.com/support/ai/mcp/setup): prerequisites, install methods and per-client configuration for Claude, Cursor, VS Code, Windsurf and more.
- [**Authentication**](https://testingbot.com/support/ai/mcp/authentication): browser login, device-code flow, API keys, credential storage and security.
- [**Tool reference**](https://testingbot.com/support/ai/mcp/tools): every tool, grouped by category, with full parameter tables.
- [**Use cases & workflows**](https://testingbot.com/support/ai/mcp/use-cases): real scenarios with example prompts and the tool sequences behind them.
- [**Troubleshooting**](https://testingbot.com/support/ai/mcp/troubleshooting): fixes for common issues with login, devices, screenshots and sessions.
- [**FAQ**](https://testingbot.com/support/ai/mcp/faq): answers to the most common questions about the MCP server.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/ai/mcp/setup
# Setup & installation
This guide walks you through installing the TestingBot MCP server and connecting it to your AI client. The server runs locally and is started on demand by your client through `npx`, so there is nothing to install globally and nothing to keep running yourself.
New to TestingBot MCP? Start with the [overview](https://testingbot.com/support/ai/mcp) to understand what the server can do. For everything about logging in, see the dedicated [Authentication](https://testingbot.com/support/ai/mcp/authentication) page.
## Prerequisites
Before you begin, make sure you have the following:
- **Node.js 18 or higher** (20+ recommended). The server is launched with `npx`, which ships with Node.js. Check your version with `node -v`.
- **An MCP-compatible client** such as Claude Desktop, Claude Code, Cursor, VS Code (Copilot/MCP), Windsurf, or any other client that supports the Model Context Protocol over stdio.
- **A TestingBot account**. You can [sign up for free](https://testingbot.com/users/sign_up) if you do not have one yet.
You do **not** need to paste an API key to get started. The recommended way to sign in is the `tb_login` tool, which opens a browser and captures your credentials without copy/paste. Authentication is covered in full on the [Authentication](https://testingbot.com/support/ai/mcp/authentication) page.
## Install
The TestingBot MCP server is distributed as the `@testingbot/mcp-server` NPM package. It bundles `@testingbot/automation-mcp`, which adds live browser and mobile automation tools, so a single install gives you both account management and automation capabilities.
You do not install the package by hand. Instead you add a small configuration entry to your MCP client that tells it to launch the server with `npx`. The command your client will run is:
```bash
npx -y @testingbot/mcp-server@latest
```
The `-y` flag lets `npx` download the package the first time without prompting, and `@latest` ensures you always start the most recent version. Keep `@latest` rather than pinning a version: newer releases add tools, and a pinned older version silently lacks them. App Automate (Maestro, Espresso and XCUITest) needs 1.1.0 or later, and a `.mcpb` bundle installed in Claude Desktop has to be re-downloaded to update.
### 1-click installers
For VS Code and Cursor you can skip the manual configuration with a 1-click installer, which writes the MCP config for you:
- [Install for VS Code](https://testingbot.com/mcp/install?client=vscode)
- [Install for Cursor](https://testingbot.com/mcp/install?client=cursor)
For Claude Desktop you can install the `.mcpb` bundle from the [MCP server releases page](https://github.com/testingbot/mcp-server/releases), Claude will configure the server automatically.
The standard configuration below is identical for every client, only the location of the config file changes. Pick your client in the tabs in the next section.
## Per-client configuration
Select your client below. Each tab shows where the configuration file lives and the JSON to add. Two variants are shown for each client:
- **Browser login (recommended)**: no key in your config. After the server starts, ask your agent to run `tb_login` and authorize in the browser.
- **API key** : set `TESTINGBOT_KEY` and `TESTINGBOT_SECRET` in the `env` block. Best for CI and shared configurations. Get your keys at [your account settings](https://testingbot.com/members/user/edit).
[Claude Desktop](https://testingbot.com#) [Claude Code](https://testingbot.com#) [Cursor](https://testingbot.com#) [VS Code](https://testingbot.com#) [Windsurf](https://testingbot.com#) [Other](https://testingbot.com#)
Edit Claude Desktop's `claude_desktop_config.json`. You can open it from **Settings → Developer → Edit Config** , or find it at:
- macOS: `~/Library/Application Support/Claude/claude_desktop_config.json`
- Windows: `%APPDATA%\Claude\claude_desktop_config.json`
**Browser login (recommended)**, no key, run `tb_login` afterwards:
```json
{
"mcpServers": {
"testingbot": {
"command": "npx",
"args": ["-y", "@testingbot/mcp-server@latest"]
}
}
}
```
**API key** variant:
```json
{
"mcpServers": {
"testingbot": {
"command": "npx",
"args": ["-y", "@testingbot/mcp-server@latest"],
"env": {
"TESTINGBOT_KEY": "",
"TESTINGBOT_SECRET": ""
}
}
}
}
```
Save the file and restart Claude Desktop. The TestingBot tools will appear in the tools menu.
Claude Code reads MCP servers from a `.mcp.json` file in your project root (or your user config). The quickest way is the CLI:
```bash
claude mcp add testingbot -- npx -y @testingbot/mcp-server@latest
```
To configure it by hand instead, create or edit `.mcp.json` in your project root.
**Browser login (recommended)**, no key, run `tb_login` afterwards:
```json
{
"mcpServers": {
"testingbot": {
"command": "npx",
"args": ["-y", "@testingbot/mcp-server@latest"]
}
}
}
```
**API key** variant:
```json
{
"mcpServers": {
"testingbot": {
"command": "npx",
"args": ["-y", "@testingbot/mcp-server@latest"],
"env": {
"TESTINGBOT_KEY": "",
"TESTINGBOT_SECRET": ""
}
}
}
}
```
Run `/mcp` inside Claude Code to confirm the `testingbot` server is connected.
Use the [1-click installer for Cursor](https://testingbot.com/mcp/install?client=cursor), or configure it manually by creating or opening `.cursor/mcp.json` in your project (or `~/.cursor/mcp.json` for a global config).
**Browser login (recommended)**, no key, run `tb_login` afterwards:
```json
{
"mcpServers": {
"testingbot": {
"command": "npx",
"args": ["-y", "@testingbot/mcp-server@latest"]
}
}
}
```
**API key** variant:
```json
{
"mcpServers": {
"testingbot": {
"command": "npx",
"args": ["-y", "@testingbot/mcp-server@latest"],
"env": {
"TESTINGBOT_KEY": "",
"TESTINGBOT_SECRET": ""
}
}
}
}
```
Restart Cursor. The TestingBot server will appear under **Settings → MCP**.
Use the [1-click installer for VS Code](https://testingbot.com/mcp/install?client=vscode), or configure it manually. Open the command palette (`Ctrl+Shift+P` / `Cmd+Shift+P`), run **MCP: Add Server** , and choose the NPM package method with `@testingbot/mcp-server`.
You can also edit the workspace file `.vscode/mcp.json` directly. Note that VS Code nests servers under a `servers` key:
**Browser login (recommended)**, no key, run `tb_login` afterwards:
```json
{
"servers": {
"testingbot": {
"command": "npx",
"args": ["-y", "@testingbot/mcp-server@latest"]
}
}
}
```
**API key** variant:
```json
{
"servers": {
"testingbot": {
"command": "npx",
"args": ["-y", "@testingbot/mcp-server@latest"],
"env": {
"TESTINGBOT_KEY": "",
"TESTINGBOT_SECRET": ""
}
}
}
}
```
Reload the window. The TestingBot tools become available to Copilot Chat in agent mode.
Windsurf reads MCP servers from `~/.codeium/windsurf/mcp_config.json`. You can also open it from **Settings → Cascade → MCP Servers → Manage MCPs**.
**Browser login (recommended)**, no key, run `tb_login` afterwards:
```json
{
"mcpServers": {
"testingbot": {
"command": "npx",
"args": ["-y", "@testingbot/mcp-server@latest"]
}
}
}
```
**API key** variant:
```json
{
"mcpServers": {
"testingbot": {
"command": "npx",
"args": ["-y", "@testingbot/mcp-server@latest"],
"env": {
"TESTINGBOT_KEY": "",
"TESTINGBOT_SECRET": ""
}
}
}
}
```
Refresh the MCP servers list in Windsurf to connect.
Any MCP-compatible client can launch the server over stdio. Add an entry that runs the standard command. Most clients use the `mcpServers` shape below:
**Browser login (recommended)**, no key, run `tb_login` afterwards:
```json
{
"mcpServers": {
"testingbot": {
"command": "npx",
"args": ["-y", "@testingbot/mcp-server@latest"]
}
}
}
```
**API key** variant:
```json
{
"mcpServers": {
"testingbot": {
"command": "npx",
"args": ["-y", "@testingbot/mcp-server@latest"],
"env": {
"TESTINGBOT_KEY": "",
"TESTINGBOT_SECRET": ""
}
}
}
}
```
If your client uses a different key (for example `servers` instead of `mcpServers`), keep the `command`, `args` and optional `env` values exactly as shown and adapt the wrapper to your client's format.
We recommend the browser-login variant: it keeps long-lived API secrets out of config files. After the server starts, ask your agent to run `tb_login`. See [Authentication](https://testingbot.com/support/ai/mcp/authentication) for the full flow, including the device-code option for SSH and headless environments.
If you use the API-key variant, treat your config file like a secret. Avoid committing files that contain `TESTINGBOT_SECRET` to source control.
## Verify your setup
Once your client has loaded the server, confirm everything works by asking your AI agent to perform a simple read-only action:
> "List my recent TestingBot tests"
> "Show my TestingBot account info"
Behind the scenes these run the `getTests` and `getUserInfo` tools. A successful response lists your tests, or returns account details such as your name, plan, company, country, available seconds and concurrency.
### What success looks like
- The TestingBot tools appear in your client's tool list (often shown as a tools or MCP icon).
- The agent returns real account data rather than an error.
- No restart is needed after authenticating, credentials are picked up automatically.
### If you see “Run tb\_login”
If a tool replies with a message like _“No TestingBot credentials configured. Run the tb\_login tool…”_, the server started successfully but is not yet authenticated (this is the expected state for the browser-login variant). Ask your agent to run `tb_login` and complete the sign-in:
> "Run tb\_login to authenticate with TestingBot"
On a desktop this opens your browser so you can click **Authorize**. On a remote or headless host, it prints a short code, open [https://testingbot.com/device](https://testingbot.com/device), enter the code, and authorize. Full details are on the [Authentication](https://testingbot.com/support/ai/mcp/authentication) page.
Run into a problem? The [Troubleshooting](https://testingbot.com/support/ai/mcp/troubleshooting) page covers common issues such as Node version errors, the server not appearing, and credential prompts.
## Next steps
Now that the server is installed and verified, continue with:
- [Authentication](https://testingbot.com/support/ai/mcp/authentication), browser login, device-code flow, API keys and where credentials are stored.
- [Tool reference](https://testingbot.com/support/ai/mcp/tools), every tool the server exposes, grouped by category.
- [Use cases & workflows](https://testingbot.com/support/ai/mcp/use-cases), end-to-end examples and the prompts that drive them.
- [FAQ](https://testingbot.com/support/ai/mcp/faq), quick answers to common questions.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/ai/mcp/authentication
# Authentication
Before the TestingBot MCP server can manage tests, drive real browsers or query your account, it needs your TestingBot credentials. There are three ways to authenticate, and the right one depends on where you run your MCP client, a desktop machine, an SSH session, a CI pipeline or a hosted web client.
This page covers all three methods, where credentials are stored, the full list of environment variables you can set, and how the server behaves when no credentials are present. If you have not installed the server yet, start with [Setup & installation](https://testingbot.com/support/ai/mcp/setup) first.
## Authentication methods
The TestingBot MCP server supports three authentication methods:
- **Browser login** , run the `tb_login` tool and authorize in a browser window on your local machine. No keys are ever pasted into your AI client. This is the recommended option for desktop use.
- **Device-code flow** , a variant of `tb_login` for SSH, remote, headless or hosted web clients. You get a short code, open [https://testingbot.com/device](https://testingbot.com/device) on any device and authorize there.
- **API key & secret** , set `TESTINGBOT_KEY` and `TESTINGBOT_SECRET` in your client's `env` block. Best for CI and shared, non-interactive configurations.
### Configuration precedence
When the server starts it resolves credentials in this fixed order:
1. **Environment variables** , if _both_ the key (`TESTINGBOT_KEY`) _and_ the secret (`TESTINGBOT_SECRET`) are set, they are used.
2. **Credentials file** , otherwise the server reads `~/.testingbot/credentials` (written by `tb_login`).
3. **Degraded mode** , if neither is available, every tool except `tb_login` replies that no credentials are configured. See [Degraded mode](https://testingbot.com#degraded) below.
Environment variables always win. If you set a key and secret in your client's `env` block, the server uses those and never touches the credentials file, so you do not need to run `tb_login`.
## Browser login with tb\_login (loopback)
The simplest way to authenticate on a desktop is the `tb_login` tool. It uses a loopback authorization flow (RFC 8252): the server opens your default browser to TestingBot, you click **Authorize** while signed in, and the credentials are captured on a local `127.0.0.1` callback. Nothing is ever copied, pasted or shown to your AI client.
### How it works, step by step
1. Ask your AI client to authenticate, for example:
> "Log in to TestingBot"
1. Your client invokes the `tb_login` tool.
2. The server opens your default browser to TestingBot and starts a temporary local callback listener.
3. If you are not signed in yet, log in to your TestingBot account, then click **Authorize**.
4. The browser redirects back to the local `127.0.0.1` callback and the credentials are captured.
5. The server writes them to `~/.testingbot/credentials` and starts using them immediately, no restart needed.
After this, all other tools (such as `getUserInfo`, `getTests` or `tb_openBrowser`) work automatically because the server reads the credentials file.
Loopback login is the default on a desktop machine. You do not need to specify a `mode`, `auto` picks loopback when a browser is available. See [The mode parameter](https://testingbot.com#mode).
Loopback login needs a local browser and the ability to open a `127.0.0.1` listener. On an SSH session, a container, a CI runner or a hosted web client there is no local browser to open, use the [device-code flow](https://testingbot.com#device-code) instead.
## Device-code flow
When there is no local browser (SSH sessions, remote or headless hosts, containers, or web-based MCP clients), use the device-code flow (RFC 8628). Instead of opening a browser locally, the server prints a short code that you enter on any device with a browser.
### How it works, step by step
1. Run `tb_login` with the device mode, for example:
> "Log in to TestingBot using the device code flow"
1. The server prints a short, human-readable code.
2. On any device, open [https://testingbot.com/device](https://testingbot.com/device).
3. Enter the code, sign in if needed, and click **Authorize**.
4. The server detects the approval, writes the credentials to `~/.testingbot/credentials` and starts using them.
The device flow is **resumable**. If the polling window closes before you finish authorizing, just run `tb_login` again and the server will continue where it left off. You can tune the polling and lifetime windows with the [device/loopback environment variables](https://testingbot.com#env-vars).
## The mode parameter
`tb_login` takes a single optional `mode` parameter that chooses the authorization flow. You can also set a default with the `TESTINGBOT_AUTH_MODE` environment variable.
| Value | Best for | Behavior |
| --- | --- | --- |
| `auto` | Most users (default) | Picks `loopback` on a desktop and falls back to `device` on a headless host. |
| `loopback` | Desktop machines | Opens the browser to TestingBot; you click Authorize; credentials are captured on a local `127.0.0.1` callback. Nothing pasted. |
| `device` | SSH / remote / headless / web clients | Prints a short code; open [https://testingbot.com/device](https://testingbot.com/device), enter it and Authorize. Resumable. |
For example, to force the device flow regardless of environment, ask your client to run `tb_login` with `mode` set to `device`, or set `TESTINGBOT_AUTH_MODE=device` in your client's `env` block.
## API key & secret
As an alternative to `tb_login`, you can supply your TestingBot API key and secret directly. This is the preferred option for CI pipelines and shared configurations where there is no interactive browser. Add them to your MCP client's `env` block:
```json
{
"mcpServers": {
"testingbot": {
"command": "npx",
"args": ["-y", "@testingbot/mcp-server@latest"],
"env": { "TESTINGBOT_KEY": "", "TESTINGBOT_SECRET": "" }
}
}
}
```
You can find your key and secret on your [TestingBot account page](https://testingbot.com/members/user/edit). Both must be set for the environment-variable path to take effect.
Prefer `tb_login` for day-to-day local use, it keeps your key and secret out of config files entirely. Reach for the `env` block when you need a non-interactive setup such as CI. See [Setup & installation](https://testingbot.com/support/ai/mcp/setup) for the full per-client configuration.
Treat your API secret like a password. Avoid committing config files that contain it to version control; use your CI provider's secret store and reference the values as environment variables instead.
## Where credentials live
When you authenticate with `tb_login`, the server stores your credentials in an INI file at `~/.testingbot/credentials`. The file is organized into profile sections so you can keep more than one account on the same machine:
```bash
[default]
key = your-api-key
secret = your-api-secret
```
By default the server reads the `[default]` profile. To use a different profile, set `TESTINGBOT_PROFILE`; to relocate the whole directory, set `TESTINGBOT_CONFIG_DIR`.
### File permissions
The credentials are written with restrictive permissions so other users on the machine cannot read them:
- The directory `~/.testingbot` is created with mode `0700` (owner only).
- The `credentials` file is written with mode `0600` (owner read/write only).
Credentials supplied through environment variables are never written to the credentials file. If you ever want to start fresh, delete `~/.testingbot/credentials` and run `tb_login` again.
## Environment variables
The following environment variables can be set in your MCP client's `env` block (or your shell) to control authentication and connection behavior. Several have aliases for compatibility with existing TestingBot tooling.
| Variable | Aliases | Default | Description |
| --- | --- | --- | --- |
| `TESTINGBOT_KEY` | `TB_KEY`, `TESTINGBOT_USERNAME` | – | Your TestingBot API key. |
| `TESTINGBOT_SECRET` | `TB_SECRET`, `TESTINGBOT_ACCESS_KEY` | – | Your TestingBot API secret. Both key and secret must be set for the env path to apply. |
| `TESTINGBOT_AUTH_MODE` | – | `auto` | Default `tb_login` mode: `auto`, `loopback` or `device`. |
| `TESTINGBOT_BASE_URL` | – | `https://testingbot.com` | Override the TestingBot API endpoint. |
| `TESTINGBOT_CONFIG_DIR` | – | `~/.testingbot` | Override the directory that holds the credentials file. |
| `TESTINGBOT_PROFILE` | – | `default` | Credentials profile (section) name to read from the credentials file. |
| `WDIO_LOG_LEVEL` | – | `silent` | WebDriver client log level. The server sets it to `silent` by default; rarely changed. |
| `TESTINGBOT_DEVICE_POLL_WINDOW_MS` | – | `25000` | Device-code polling window, in milliseconds. |
| `TESTINGBOT_LOOPBACK_WAIT_MS` | – | `60000` | How long the loopback flow waits for you to authorize, in milliseconds. |
| `TESTINGBOT_LOOPBACK_LIFETIME_MS` | – | `900000` | Maximum lifetime of the loopback authorization session, in milliseconds. |
## Degraded mode
If the server cannot find credentials (neither a key/secret pair in the environment nor a credentials file), it starts in **degraded mode**. In this state every tool except `tb_login` returns a message like:
```bash
No TestingBot credentials configured. Run the tb_login tool to authenticate.
```
This is not an error in your client configuration, it simply means you have not authenticated yet. To fix it:
- Run the `tb_login` tool (ask your client to "log in to TestingBot"), or
- Set `TESTINGBOT_KEY` and `TESTINGBOT_SECRET` in your client's `env` block and reload the server.
Seeing the "Run tb\_login" message repeatedly after authenticating? Check the [Troubleshooting](https://testingbot.com/support/ai/mcp/troubleshooting) page, it is usually a profile or config-directory mismatch.
## Security notes
The TestingBot MCP server is designed so your credentials stay under your control and never leak into your AI conversation.
- **Credentials stay local.** Whether captured by `tb_login` or read from environment variables, they live on your machine (in `~/.testingbot/credentials` or the process environment) and are never sent to your LLM provider.
- **Direct API calls.** The MCP server talks directly to the TestingBot API. Your AI client sends natural-language requests to the local server; the server performs the work and returns results.
- **No key pasting.** The browser and device-code flows capture credentials out of band, so your key and secret never pass through the chat transcript.
- **Restrictive permissions.** The credentials file is written `0600` inside a `0700` directory, readable only by your user account.
Next steps: review the [Tool reference](https://testingbot.com/support/ai/mcp/tools) to see what you can do once authenticated, browse common [use cases & workflows](https://testingbot.com/support/ai/mcp/use-cases), or return to the [TestingBot MCP overview](https://testingbot.com/support/ai/mcp).
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/ai/mcp/tools
# Tool reference
This page documents every tool exposed by the TestingBot MCP server. You never call these tools by hand, your AI client (Claude, Cursor, VS Code, Windsurf, and others) calls them on your behalf when you describe what you want in plain language. The names and parameters below are useful when you want to know exactly what an agent can do, or to phrase a prompt precisely.
The tools come from two bundled packages. `@testingbot/mcp-server` handles your account and orchestration (tests, builds, storage, App Automate runs for Maestro, Espresso and XCUITest, screenshots, live sessions, team, tunnels, logs and project scaffolding). `@testingbot/automation-mcp`, which is bundled automatically, drives live browser and mobile sessions over WebDriver and proxies native app testing to Appium. See [Setup & installation](https://testingbot.com/support/ai/mcp/setup) to connect the server, and [Use cases & workflows](https://testingbot.com/support/ai/mcp/use-cases) for end-to-end examples that chain these tools together.
Every tool below carries an example prompt. You do not have to phrase it that way, it is there to show the kind of request that resolves to that tool. Parameters are passed by your AI client, not typed by you. If a tool replies with _“No TestingBot credentials configured. Run the tb\_login tool…”_, authenticate first, see [Authentication](https://testingbot.com/support/ai/mcp/authentication).
## Authentication
Logs you in without pasting an API key. After a successful login, credentials are written to `~/.testingbot/credentials` and used automatically, no restart needed. Read more on the [Authentication](https://testingbot.com/support/ai/mcp/authentication) page.
| Tool | Description |
| --- | --- |
| `tb_login` | Browser or device-code login, no key paste required. |
### tb\_login
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `mode` | string | No | `auto` | Login flow: `auto` (loopback on a desktop, device on headless), `loopback` (RFC 8252 local callback), or `device` (RFC 8628 short code at [https://testingbot.com/device](https://testingbot.com/device)). |
**Example prompt:** _“Log in to TestingBot”_
## Browser & device
Discover which browsers, platforms and mobile devices are available before you start a session.
### getBrowsers
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `type` | string | No | – | Filter the list: `web` or `mobile`. Omit for all. |
**Example prompt:** _“Which Chrome versions can I run on Windows 11?”_
### getDevices
Lists mobile devices (real devices plus emulators/simulators) with their current availability. Takes no parameters.
**Example prompt:** _“Which real iPhones are free right now?”_
## Tests
Browse, inspect and manage the test sessions in your account.
### getTests
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `offset` | number | No | `0` | Pagination offset. |
| `limit` | number | No | `10` | Number of tests to return (max 100). |
**Example prompt:** _“List my last 10 TestingBot tests”_
### getTestDetails
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `sessionId` | string | Yes | – | The session to fetch full details for. |
**Example prompt:** _“Show me the full details of my most recent failed test, with the video and logs”_
### updateTest
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `sessionId` | string | Yes | – | The session to update. |
| `name` | string | No | – | New test name. |
| `status` | string | No | – | Mark the test as `passed` or `failed`. |
| `build` | string | No | – | Associate the test with a build. |
| `extra` | string | No | – | Additional metadata as a JSON string. |
**Example prompt:** _“Mark that session as failed and rename it to Checkout regression”_
### deleteTest
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `sessionId` | string | Yes | – | The session to delete. |
**Example prompt:** _“Delete the session I just started by mistake”_
### stopTest
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `sessionId` | string | Yes | – | The running session to stop. |
**Example prompt:** _“Stop the session that is still running”_
## Builds
Group related tests into builds and manage them.
### getBuilds
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `offset` | number | No | `0` | Pagination offset. |
| `limit` | number | No | `10` | Number of builds to return (max 100). |
**Example prompt:** _“List my last 5 builds”_
### getTestsForBuild
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `buildId` | string | Yes | – | The build whose tests you want to list. |
**Example prompt:** _“Which tests are in the build called nightly?”_
### deleteBuild
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `buildId` | string | Yes | – | Deletes the build and all of its tests. |
**Example prompt:** _“Delete the build called scratch-run and everything in it”_
## Storage
Upload and manage native mobile app binaries (`.apk`, `.ipa`, `.zip`) so they can be installed on devices for native app testing.
### uploadFile
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `localFilePath` | string | Yes | – | Absolute path to a local `.apk`/`.ipa`/`.zip`. |
**Example prompt:** _“Upload /Users/me/builds/app-release.apk to TestingBot storage”_
### uploadRemoteFile
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `remoteUrl` | string | Yes | – | An http/https URL to fetch and store (SSRF-guarded). |
**Example prompt:** _“Upload the build at https://ci.example.com/artifacts/app.ipa to my TestingBot storage”_
### getStorageFiles
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `offset` | number | No | `0` | Pagination offset. |
| `limit` | number | No | `10` | Number of files to return (max 100). |
**Example prompt:** _“Which app builds do I have in storage?”_
### deleteStorageFile
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `appUrl` | string | Yes | – | The stored app reference to delete. |
**Example prompt:** _“Delete tb://app123 from storage”_
## App Automate: Maestro
Run [Maestro](https://testingbot.com/support/app-automate/maestro) flows on TestingBot real devices, emulators and simulators. The agent uploads your app, uploads (or writes) the flow YAML, starts a run, and polls until it finishes.
Maestro runs are **asynchronous** : `runMaestroTest` returns a run ID immediately and does not block while devices execute. The agent then polls `getMaestroRunStatus` and fetches `getMaestroRunResults` once the run completes.
| Tool | Description |
| --- | --- |
| `uploadMaestroApp` | Upload the app under test. Returns a project id. |
| `uploadMaestroFlows` | Upload the flow YAML for a project, from disk or inline. |
| `runMaestroTest` | Start a run. Returns a run id immediately. |
| `getMaestroRunStatus` | Poll per-flow progress. |
| `getMaestroRunResults` | Final outcomes plus the JUnit XML report. |
| `getMaestroFlowDetails` | Errors, step-level report, video, screenshots and device logs for one flow. |
| `cancelMaestroRun` | Cancel a running run. |
| `retryMaestroRun` | Retry a whole run, or a single failed flow. |
| `uploadMaestroCompanionApp` | Upload a companion app to install alongside the app under test. |
| `listMaestroProjects` | Find existing projects instead of re-uploading. |
| `listMaestroRuns` | List runs, or look them up by run name. |
| `maestroCheatSheet` | Maestro YAML syntax reference for the agent. |
### uploadMaestroApp
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `localFilePath` | string | Yes | – | Local path to the `.apk`, `.aab`, `.ipa` or zipped `.app`. |
Returns a **project id** that every other Maestro tool takes. Uploads are deduplicated by checksum, so re-running with an unchanged binary reuses the existing project instead of uploading again.
**Example prompt:** _“Upload ./build/app.apk for Maestro testing”_
### uploadMaestroFlows
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `projectId` | number | Yes | – | Project id from `uploadMaestroApp`. |
| `flowsPath` | string | No | – | Directory of `.yaml`/`.yml` flows (structure preserved, so `runFlow` subflows keep resolving), or a pre-built `.zip`. |
| `flows` | array | No | – | Inline flows as `{ fileName, content }` objects, for flows the agent writes itself. |
Each upload replaces the flows previously uploaded for that project.
**Example prompt:** _“Upload the flows in ./flows to that project”_
### runMaestroTest
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `projectId` | number | Yes | – | Project id from `uploadMaestroApp`. |
| `platformName` | string | Yes | – | `Android` or `iOS`. |
| `deviceName` | string | No | any device | Device name, wildcards and regex supported, e.g. `Pixel 8`, `iPhone 1[4-6]`, `*`. |
| `version` | string | No | – | OS version, e.g. `14`. |
| `realDevice` | boolean | No | `false` | Run on a physical device instead of an emulator/simulator. Required for `.ipa` apps. |
| `name` | string | No | – | Run name shown in the dashboard, and the value `listMaestroRuns` looks up. |
| `env` | object | No | – | Environment variables passed to the flows. |
| `includeTags` | array | No | – | Only run flows carrying these Maestro tags. |
| `excludeTags` | array | No | – | Skip flows carrying these Maestro tags. |
| `shardSplit` | number | No | – | Split the flows across this many parallel devices (1–10). |
| `otherApps` | array | No | – | Up to 4 companion app URLs (`tb://` or `http(s)://`) installed alongside the app under test. |
**Example prompt:** _“Run my Maestro flows on a real Pixel 8 with Android 14”_
### getMaestroRunStatus
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `projectId` | number | Yes | – | The project to inspect. |
| `runId` | number | No | – | A specific run. Omit it to list the runs of the latest execution. |
Per-flow statuses are `WAITING`, `READY` (running), `DONE`, `FAILED` and `CANCELLED`, with failures listed first. The flow ids in this output are what `getMaestroFlowDetails` and `retryMaestroRun` take.
**Example prompt:** _“How is that Maestro run doing?”_
### getMaestroRunResults
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `projectId` | number | Yes | – | The project. |
| `runId` | number | Yes | – | The completed run. |
Returns per-flow outcomes, failures first with error messages and video links, plus the JUnit XML report.
**Example prompt:** _“Give me the results of the Maestro run now that it has finished”_
### getMaestroFlowDetails
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `projectId` | number | Yes | – | The project. |
| `runId` | number | Yes | – | The run. |
| `flowId` | number | Yes | – | The flow, taken from `getMaestroRunStatus`. |
This is the debugging tool: error messages, the step-level JUnit report and links to the video, screenshots and device logs. Fetch it for a failed flow, fix the YAML, upload the flows again, then retry.
**Example prompt:** _“Why did the checkout flow fail? Show me the error and the video”_
### cancelMaestroRun
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `projectId` | number | Yes | – | The project. |
| `runId` | number | Yes | – | The run to cancel. |
**Example prompt:** _“Cancel the Maestro run I just started”_
### retryMaestroRun
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `projectId` | number | Yes | – | The project. |
| `runId` | number | Yes | – | The run to retry. |
| `flowId` | number | No | – | Retry only this flow instead of the whole run. |
**Example prompt:** _“Fix the login flow and re-run just that one”_
### uploadMaestroCompanionApp
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `localFilePath` | string | Yes | – | Local path to the companion `.apk`, `.aab`, `.ipa` or `.zip`. |
Returns a `tb://` app URL to pass in the `otherApps` parameter of `runMaestroTest` (max 4 per run).
**Example prompt:** _“Upload ./build/mock-backend.apk as a companion app for the run”_
### listMaestroProjects
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `offset` | number | No | `0` | Pagination offset. |
| `count` | number | No | `10` | Number of projects to return (max 100). |
Newest first, with app details, flow names and run ids, so the agent can reuse a project id rather than re-uploading an app.
**Example prompt:** _“What Maestro projects do I already have on my account?”_
### listMaestroRuns
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `buildName` | string | No | – | Look up runs with this exact run name instead of listing all runs. |
| `offset` | number | No | `0` | Pagination offset. Ignored when `buildName` is set. |
| `count` | number | No | `10` | Number of runs to return (max 100). Ignored when `buildName` is set. |
Returns `projectId` and `runId` pairs. The `buildName` lookup is what makes prompts like _“rerun whatever failed in last night's nightly build”_ work.
**Example prompt:** _“Find the Maestro run named nightly and tell me what failed in it”_
### maestroCheatSheet
Returns a Maestro YAML syntax reference: file structure, commands, selectors and the TestingBot run workflow. Takes no parameters. Agents call this before writing or fixing flows, so the YAML they produce matches what Maestro actually accepts.
**Example prompt:** _“Write me a Maestro flow that logs in and adds an item to the cart”_
Maestro tooling requires `@testingbot/mcp-server` 1.1.0 or later. If your agent reports that these tools do not exist, update the server, see [Setup & installation](https://testingbot.com/support/ai/mcp/setup).
## App Automate: Espresso & XCUITest
Run pre-built [Espresso](https://testingbot.com/support/app-automate/espresso) (Android) and [XCUITest](https://testingbot.com/support/app-automate/xcuitest) (iOS) suites on TestingBot devices. Every tool takes a `framework` parameter, `espresso` or `xcuitest`, and follows the same asynchronous start, poll, fetch pattern as Maestro.
| Tool | Description |
| --- | --- |
| `uploadAppAutomateApp` | Upload the app under test. Returns a project id. |
| `uploadAppAutomateTests` | Upload the test suite for that project. |
| `runAppAutomateTest` | Start a run. Returns a run id immediately. |
| `getAppAutomateRunStatus` | Poll run progress. |
| `getAppAutomateRunResults` | Per-run outcomes plus the JUnit XML report. |
### uploadAppAutomateApp
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `framework` | string | Yes | – | `espresso` or `xcuitest`. |
| `localFilePath` | string | Yes | – | The app under test: `.apk`/`.aab` for Espresso, `.ipa`/`.zip` for XCUITest. |
Returns the project id used by the other two App Automate tools. For Maestro flows, use `uploadMaestroApp` instead.
**Example prompt:** _“Upload ./app/build/outputs/apk/debug/app-debug.apk for an Espresso run”_
### uploadAppAutomateTests
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `framework` | string | Yes | – | `espresso` or `xcuitest`. |
| `projectId` | number | Yes | – | Project id from `uploadAppAutomateApp`. |
| `localFilePath` | string | Yes | – | The instrumented test `.apk` for Espresso, or the zipped XCUITest runner bundle (`.zip`). |
**Example prompt:** _“Upload the instrumented test APK for that project”_
### runAppAutomateTest
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `framework` | string | Yes | – | `espresso` or `xcuitest`. |
| `projectId` | number | Yes | – | Project id from `uploadAppAutomateApp`. |
| `deviceName` | string | No | any device | Device name, wildcards and regex supported, e.g. `Galaxy S23`, `iPhone 1[4-6]`, `*`. |
| `version` | string | No | – | OS version, e.g. `14`. |
| `realDevice` | boolean | No | `false` | Run on a physical device instead of an emulator/simulator. |
| `name` | string | No | – | Run name shown in the dashboard. |
| `build` | string | No | – | Build identifier used to group runs. |
| `locale` | string | No | – | Locale for the run, e.g. `fr_FR`. |
| `timeZone` | string | No | – | Time zone for the run, e.g. `Europe/Brussels`. |
| `testClasses` | array | No | – | Espresso only: run only these fully qualified test classes. |
| `skipTestClasses` | array | No | – | Espresso only: skip these test classes. |
| `packages` | array | No | – | Espresso only: run only these packages. |
| `annotations` | array | No | – | Espresso only: run only tests carrying these annotations. |
| `testRunner` | string | No | – | Espresso only: custom instrumentation runner. |
**Example prompt:** _“Run the Espresso suite on a real Galaxy S23, only the com.example.LoginTest class”_
### getAppAutomateRunStatus
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `framework` | string | Yes | – | `espresso` or `xcuitest`. |
| `projectId` | number | Yes | – | The project to inspect. |
| `runId` | number | No | – | A specific run. Omit it to list every run in the project. |
Statuses are `WAITING`, `READY` (running), `DONE` and `FAILED`. Each run also reports its Appium session id.
**Example prompt:** _“Is my XCUITest run finished yet?”_
### getAppAutomateRunResults
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `framework` | string | Yes | – | `espresso` or `xcuitest`. |
| `projectId` | number | Yes | – | The completed project. |
| `runId` | number | No | – | XCUITest only: fetch the JUnit report of this single run. |
**Example prompt:** _“Show me the JUnit results for that Espresso project”_
Because each Espresso and XCUITest run surfaces its Appium session id, the session tools work on them too: pass that id to [`getTestDetails`](https://testingbot.com#tests) for the video and screenshots, or to [`getFailureLogs`](https://testingbot.com#logs) for the device and Appium logs.
## Screenshots
Capture a URL across multiple browsers and platforms for cross-browser visual checks.
### takeScreenshot
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `url` | string | Yes | – | The page to capture. |
| `browsers` | array | Yes | – | Array of `{ browserName, version?, os }` targets. |
| `resolution` | string | No | `1920x1080` | Screen resolution to render at. |
| `waitTime` | number | No | `5` | Seconds to wait before capture (0–60). |
| `fullPage` | boolean | No | `false` | Capture the full scrollable page. |
Returns a job id that you pass to `retrieveScreenshots`.
**Example prompt:** _“Take full-page screenshots of testingbot.com in Chrome on Windows 11 and Safari on macOS”_
### retrieveScreenshots
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `screenshotId` | string | Yes | – | The job id returned by `takeScreenshot`. |
**Example prompt:** _“Are those screenshots ready? Give me the image links”_
### getScreenshotList
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `offset` | number | No | `0` | Pagination offset. |
| `limit` | number | No | `10` | Number of jobs to return (max 100). |
**Example prompt:** _“List my recent screenshot jobs”_
## User
Read and update your account profile.
### getUserInfo
Returns account information, name, plan, company, country, available seconds and concurrency. Takes no parameters. Note: `getUserInfo` does not return an email field.
**Example prompt:** _“How many testing minutes do I have left?”_
### updateUserInfo
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `firstName` | string | No | – | New first name. |
| `lastName` | string | No | – | New last name. |
| `email` | string | No | – | New email address. |
**Example prompt:** _“Update the last name on my profile to De Vries”_
## Live testing
Start an interactive, manual live session you can watch and control. Each session returns a live-view URL of the form `https://testingbot.com/tests//live?auth=`.
### startLiveSession
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `platformType` | string | Yes | – | `desktop` or `mobile`. |
| `desiredURL` | string | Yes | – | The URL to open in the session. |
| `desiredOS` | string | Yes | – | Operating system to run on. |
| `desiredOSVersion` | string | Yes | – | OS version. |
| `desiredBrowser` | string | No | – | Browser to launch (desktop). |
| `desiredBrowserVersion` | string | No | – | Browser version (desktop). |
| `desiredDevice` | string | No | – | Device name (mobile). |
**Example prompt:** _“Start a live session on Windows 11 in Chrome and open my staging site”_
### startDesktopLiveSession
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `desiredURL` | string | Yes | – | The URL to open. |
| `desiredOS` | string | Yes | – | `Windows`, `Mac` or `Linux`. |
| `desiredOSVersion` | string | Yes | – | OS version. |
| `desiredBrowser` | string | Yes | – | `chrome`, `firefox`, `safari`, `edge` or `ie`. |
| `desiredBrowserVersion` | string | No | `latest` | Browser version. |
**Example prompt:** _“Give me a manual Safari session on macOS Ventura pointed at example.com”_
### startMobileLiveSession
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `desiredURL` | string | Yes | – | The URL to open. |
| `desiredOS` | string | Yes | – | `android` or `ios`. |
| `desiredOSVersion` | string | Yes | – | OS version. |
| `desiredDevice` | string | Yes | – | The device to run on. |
**Example prompt:** _“Open a live session on an iPhone 15 Pro and load my checkout page”_
## Team
Inspect your team settings, usage and members.
| Tool | Param | Required | Description |
| --- | --- | --- | --- |
| `getTeam` | – | – | Team settings and usage. No parameters. |
| `getUsersInTeam` | – | – | List all team members. No parameters. |
| `getUserFromTeam` | `userId` | Yes | Fetch a single team member by id. |
**Example prompt:** _“Show my team's usage this month and list everyone in it”_
## CDP
Spin up a remote browser exposing a Chrome DevTools Protocol endpoint, so you can drive it with Puppeteer or Playwright. See the Playwright/Puppeteer example in [Use cases & workflows](https://testingbot.com/support/ai/mcp/use-cases).
### createCdpSession
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `browserName` | string | Yes | – | Browser to launch. |
| `browserVersion` | string | No | `latest` | Browser version. |
| `platform` | string | Yes | – | Platform to run on. |
| `screenResolution` | string | No | – | Screen resolution. |
| `timeZone` | string | No | – | Time zone for the session. |
| `name` | string | No | – | Session name. |
| `build` | string | No | – | Build to associate the session with. |
| `extraCapabilities` | object | No | – | Additional WebDriver capabilities. |
**Example prompt:** _“Start a CDP-enabled Chrome session on Windows 11 and give me the WebSocket URL”_
## Tunnels
Manage TestingBot Tunnels for testing local or firewalled sites.
| Tool | Param | Required | Description |
| --- | --- | --- | --- |
| `getTunnelList` | – | – | List active tunnels. No parameters. |
| `deleteTunnel` | `tunnelId` | Yes | Stop an active tunnel by id. |
**Example prompt:** _“List my active tunnels and stop the one I left running this morning”_
## Logs
Fetch and filter the logs for a session, ideal for debugging a failing run.
### getFailureLogs
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `sessionId` | string | Yes | – | The session whose logs you want. |
| `logTypes` | array | No | – | Any of `selenium`, `browser`, `chrome`, `vm`, `appium`. |
| `failuresOnly` | boolean | No | `false` | Return only failure-related entries. |
| `maxBytesPerLog` | number | No | `50000` | Truncate each log (1,000–1,000,000 bytes). |
| `timeoutMs` | number | No | `15000` | Fetch timeout (1,000–60,000 ms). |
**Example prompt:** _“Pull just the failure lines from the Appium and device logs for that session”_
## Project
Detect the test framework in a local project and scaffold a ready-to-use TestingBot configuration.
### listTestFiles
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `projectRoot` | string | Yes | – | Root directory of the project to scan. |
| `maxDepth` | number | No | `8` | How deep to recurse (1–15). |
| `maxResults` | number | No | `200` | Maximum files to return (1–2000). |
**Example prompt:** _“What test files are in this project?”_
### setupTestingBot
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `projectRoot` | string | Yes | – | Root directory of the project. |
| `frameworkOverride` | string | No | – | Force a framework instead of auto-detecting. |
Generates a ready-to-paste TestingBot config for the detected framework, Playwright, Cypress, WebdriverIO, Selenium (JS), Nightwatch or Puppeteer.
**Example prompt:** _“Set this project up to run on TestingBot”_
## Browser automation
These tools come from the bundled `@testingbot/automation-mcp` package and drive a live browser session over WebDriver. They work for desktop browsers and for mobile browsers (on emulators/simulators or real devices). Every session starts with `tb_openBrowser`, which returns a `sessionId` you pass to the other tools, plus a live-view URL (`https://testingbot.com/tests//live?auth=`) you can open to watch or take over.
### tb\_openBrowser
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `browserName` | string | Yes | – | `chrome`, `firefox`, `edge` or `safari`. |
| `browserVersion` | string | No | `latest` | Browser version (desktop only). |
| `platform` | string | Yes | – | Desktop codes like `WIN11`/`VENTURA`, or `Android`/`iOS` for mobile. |
| `deviceName` | string | No | – | Mobile only (e.g. `Google Pixel 9`, `iPhone 15 Pro`). Its presence switches the session to mobile. |
| `platformVersion` | string | No | – | Mobile OS version. |
| `automationName` | string | No | `UiAutomator2`/`XCUITest` | Mobile automation engine. |
| `realDevice` | boolean | No | `false` | `true` = physical device, `false` = emulator/simulator. |
| `name` | string | No | – | Session name. |
| `build` | string | No | – | Build to group the session under. |
| `screenResolution` | string | No | – | Screen resolution (desktop only). |
**Example prompt:** _“Open Chrome on Windows 11”_
Real-device pre-flight: if `realDevice:true` and the requested device or version is busy or not offered, `tb_openBrowser` returns guidance (wait, use an emulator, or pick an available alternative) **without** starting a session. See [Use cases](https://testingbot.com/support/ai/mcp/use-cases) and [Troubleshooting](https://testingbot.com/support/ai/mcp/troubleshooting).
### tb\_navigate
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `sessionId` | string | Yes | – | The active session. |
| `url` | string | Yes | – | The URL to navigate to. |
**Example prompt:** _“Go to my staging login page”_
### tb\_snapshot
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `sessionId` | string | Yes | – | The active session. |
| `bodyChars` | number | No | `5000` | Body text characters to include (500–50,000). |
Returns a structured summary of the page: title, headings, actionable elements and body text.
**Example prompt:** _“What is on the page right now?”_
### tb\_click
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `sessionId` | string | Yes | – | The active session. |
| `by` | string | No | `css` | Locator strategy: `css`, `xpath`, `id`, `name`, `tag`, `class`, `linkText`, `partialLinkText`. |
| `value` | string | Yes | – | The locator value. |
| `timeoutMs` | number | No | `5000` | Wait timeout (100–30,000 ms). |
**Example prompt:** _“Click the Sign in button”_
### tb\_type
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `sessionId` | string | Yes | – | The active session. |
| `by` | string | No | `css` | Locator strategy. |
| `value` | string | Yes | – | The locator value of the field. |
| `text` | string | Yes | – | Text to type. |
| `clearFirst` | boolean | No | `true` | Clear the field before typing. |
| `pressEnter` | boolean | No | `false` | Press Enter after typing. |
**Example prompt:** _“Type test@example.com into the email field and press Enter”_
### tb\_getText
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `sessionId` | string | Yes | – | The active session. |
| `by` | string | No | `css` | Locator strategy. |
| `value` | string | Yes | – | The locator value. |
| `timeoutMs` | number | No | `5000` | Wait timeout. |
**Example prompt:** _“Read back the dashboard heading”_
### tb\_getAttribute
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `sessionId` | string | Yes | – | The active session. |
| `by` | string | No | `css` | Locator strategy. |
| `value` | string | Yes | – | The locator value. |
| `attribute` | string | Yes | – | The attribute name to read. |
| `timeoutMs` | number | No | `5000` | Wait timeout. |
**Example prompt:** _“Is the submit button still disabled?”_
### tb\_executeScript
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `sessionId` | string | Yes | – | The active session. |
| `script` | string | Yes | – | JavaScript to run in the page. |
| `args` | array | No | `[]` | Arguments passed to the script. |
**Example prompt:** _“Scroll to the bottom of the page and tell me the document height”_
### tb\_screenshot
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `sessionId` | string | Yes | – | The active session. |
| `savePath` | string | No | OS temp dir | A `.png` path or directory to save to. |
Returns an inline image _and_ saves a file, returning a clickable `file://` link. Many clients do not render inline tool images, so open the saved file if you do not see the screenshot in chat.
**Example prompt:** _“Take a screenshot of what the browser is showing now”_
### tb\_pressKey
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `sessionId` | string | Yes | – | The active session. |
| `key` | string | Yes | – | A W3C key or chord, e.g. `Enter` or `Control+A`. |
**Example prompt:** _“Press Escape to close the modal”_
### tb\_closeBrowser
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `sessionId` | string | Yes | – | Ends the session and frees minutes. |
**Example prompt:** _“Close the session”_
### tb\_listSessions
Lists active automation sessions. Takes no parameters.
**Example prompt:** _“Which automation sessions do I have open?”_
## Mobile native apps
For testing native iOS/Android apps (not mobile browsers, use `tb_openBrowser` for those), the automation package proxies to the official appium-mcp. Upload your app first with `uploadFile`, then reference it as `appium:app="tb://"`.
### appium\_session\_management
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `action` | string | Yes | – | One of `create`, `delete`, `list`, `detach`, `select`. |
| `platform` | string | For create | – | `ios` or `android`. |
| `capabilities` | string | No | – | JSON string for `create`: `appium:deviceName`, `appium:platformVersion`, `appium:app="tb://"`. |
| `sessionId` | string | No | – | Required for `delete`, `detach` and `select`. |
Ready to put these together? See [Use cases & workflows](https://testingbot.com/support/ai/mcp/use-cases) for full tool sequences, or return to the [MCP overview](https://testingbot.com/support/ai/mcp).
**Example prompt:** _“Start a native Android session with the app I just uploaded”_
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/ai/mcp/use-cases
# Use cases & workflows
TestingBot MCP turns natural language into real testing work. You describe a goal to your AI client and the agent decides which TestingBot tools to call, in what order, to get it done. This page walks through the most common workflows: each scenario lists the goal, an example prompt you could give your agent, and the sequence of tools the agent would typically run behind the scenes.
You never call these tools by hand, your client does. For the full list of tools and their parameters, see the [Tool reference](https://testingbot.com/support/ai/mcp/tools). If you have not set the server up yet, start with [Setup & installation](https://testingbot.com/support/ai/mcp/setup) and [Authentication](https://testingbot.com/support/ai/mcp/authentication).
All of these workflows run on TestingBot's cloud grid through one locally-running server. The main `@testingbot/mcp-server` package handles your account, tests, builds, storage and screenshots; it bundles `@testingbot/automation-mcp`, which drives live browser and mobile sessions. You don't install browsers, emulators or Appium locally. See [the MCP overview](https://testingbot.com/support/ai/mcp) for the big picture.
## 1. Automate a real desktop browser
**Goal:** open a real desktop browser on TestingBot's cloud, navigate to a page and interact with it (click, type, read text and assert), all from natural language.
> "Open Chrome on Windows 11, go to my staging login page, sign in with test@example.com / hunter2, and tell me whether the dashboard heading appears."
A typical tool sequence the agent runs:
1. `tb_openBrowser`: start a Chrome session on `WIN11`; returns a `sessionId` and a live-view URL.
2. `tb_navigate`: load the staging login page.
3. `tb_snapshot`: get a structured summary of the page so the agent knows what to interact with.
4. `tb_type`: fill the email and password fields.
5. `tb_click`: submit the form.
6. `tb_getText`: read the dashboard heading to confirm the login worked.
7. `tb_closeBrowser`: end the session and free your minutes.
The same browser-automation tools work for desktop and mobile browsers, the difference is the capabilities passed to `tb_openBrowser`. You can watch the whole run live (see [Watch or take over a session](https://testingbot.com#live-view)).
## 2. Mobile browser on an emulator/simulator
**Goal:** test your mobile web experience on Android or iOS without owning the hardware. Emulators and simulators are the default, they are cheaper and plentiful, which makes them ideal for everyday responsive and functional checks.
> "Open Chrome on an Android emulator (Google Pixel 9), load testingbot.com and check that the mobile menu button is visible."
A typical tool sequence:
1. `getDevices`: (optional) list the mobile devices that are available.
2. `tb_openBrowser`: with `platform` `Android`, `deviceName` `"Google Pixel 9"` and `realDevice: false` (the default). Providing a `deviceName` switches the session to mobile.
3. `tb_navigate`: load the URL.
4. `tb_snapshot` / `tb_getAttribute`: inspect the page and confirm the menu button is present.
5. `tb_closeBrowser`: end the session.
Leaving `realDevice` at its default (`false`) runs on an emulator (Android) or simulator (iOS). This is the right choice for most mobile-web testing. Browse the physical device list at [/support/app-automate/devices](https://testingbot.com/support/app-automate/devices).
## 3. Mobile browser on a physical real device
**Goal:** run your mobile web test on real hardware when you need true GPU rendering, sensors, or the genuine mobile Chrome/Safari engine, not an emulator approximation.
> "Open Safari on a real iPhone 15 Pro running iOS 17, go to my checkout flow and read back the total price element."
A typical tool sequence:
1. `getDevices`: check which real devices and OS versions are currently free.
2. `tb_openBrowser`: with `platform` `iOS`, `deviceName` `"iPhone 15 Pro"`, `platformVersion` `"17"` and `realDevice: true`.
3. `tb_navigate`: load the checkout flow.
4. `tb_getText`: read the total price element.
5. `tb_closeBrowser`: end the session.
Real devices are a shared, finite resource. When `realDevice: true` and the requested device or OS version is busy or not offered, `tb_openBrowser` runs a pre-flight check and returns guidance _without_ starting (and billing) a session. It will suggest waiting for the device to free up, picking an available alternative device/version, or falling back to an emulator/simulator. Your agent can act on that guidance and retry.
## 4. Native mobile app testing
**Goal:** test a native Android (`.apk`) or iOS (`.ipa`) app, not a mobile website. Upload the build to TestingBot storage, then start an Appium session against it.
> "Upload the app at /Users/me/builds/app-release.apk to TestingBot, then start a native Android session with it."
A typical tool sequence:
1. `uploadFile`: upload the local `.apk`/`.ipa`/`.zip` by absolute path. (Use `uploadRemoteFile` if the build lives at a URL.) This returns an app URL like `tb://`.
2. `getStorageFiles`: (optional) confirm the upload and look up the app URL.
3. `appium_session_management`: `action: "create"`, `platform: "android"`, with `capabilities` including `appium:app="tb://"`, `appium:deviceName` and `appium:platformVersion`.
4. `appium_session_management`: `action: "list"` / `"select"` / `"delete"` to manage and end sessions.
Native app sessions are managed by `appium_session_management` (proxied to the official Appium MCP). For mobile _browser_ testing, use `tb_openBrowser` with a `deviceName` instead, see scenarios 2 and 3 above. To run an existing test suite rather than drive the app step by step, use App Automate, see scenarios 5 and 6 below.
## 5. Run a Maestro flow suite and fix what fails
**Goal:** run your [Maestro](https://testingbot.com/support/app-automate/maestro) flows against a build on real devices, emulators or simulators, then have the agent read the failures, correct the YAML and re-run only what broke.
> "Run the Maestro flows in ./flows against ./build/app.apk on a real Pixel 8 with Android 14. Fix any failures and re-run them."
A typical tool sequence:
1. `uploadMaestroApp`: upload the `.apk`/`.aab`/`.ipa`. Returns a `projectId`. An unchanged binary is deduplicated by checksum, so repeat runs skip the upload.
2. `maestroCheatSheet`: (when the agent is writing or repairing flows) fetch the YAML syntax reference first.
3. `uploadMaestroFlows`: upload the flow directory, a `.zip`, or inline YAML the agent wrote itself.
4. `runMaestroTest`: start the run with `platformName: "Android"`, `deviceName: "Pixel 8"`, `version: "14"` and `realDevice: true`. Returns a `runId` immediately.
5. `getMaestroRunStatus`: poll until every flow reports `DONE`, `FAILED` or `CANCELLED`.
6. `getMaestroRunResults`: per-flow outcomes plus the JUnit XML report.
7. `getMaestroFlowDetails`: for a failed flow, read the error, the step-level report, and the video, screenshot and device-log links.
8. `uploadMaestroFlows` then `retryMaestroRun`: push the corrected YAML and retry, passing `flowId` to re-run just that one flow.
Already have a project on the account? `listMaestroProjects` and `listMaestroRuns` let the agent find it without re-uploading anything. `listMaestroRuns` also takes a `buildName`, which is what makes a prompt like _"rerun whatever failed in last night's nightly build"_ resolve to a concrete run.
Useful extras on `runMaestroTest`: `includeTags`/`excludeTags` to run a subset (a smoke tag, say), `shardSplit` to spread the flows over parallel devices, `env` to inject credentials or feature flags, and `otherApps` (with `uploadMaestroCompanionApp`) to install up to 4 companion apps alongside the app under test. `cancelMaestroRun` stops a run you no longer need.
## 6. Run an Espresso or XCUITest suite
**Goal:** run a pre-built [Espresso](https://testingbot.com/support/app-automate/espresso) (Android) or [XCUITest](https://testingbot.com/support/app-automate/xcuitest) (iOS) suite on TestingBot devices and get the JUnit report back, without writing a CI job for it.
> "Run my Espresso suite on a real Galaxy S23, but only the com.example.LoginTest class, and show me the failures."
A typical tool sequence:
1. `uploadAppAutomateApp`: `framework: "espresso"` with the app `.apk`/`.aab` (or `"xcuitest"` with the `.ipa`). Returns a `projectId`.
2. `uploadAppAutomateTests`: the instrumented test `.apk` for Espresso, or the zipped XCUITest runner bundle.
3. `runAppAutomateTest`: start the run with `deviceName`, `version` and `realDevice`. For Espresso, narrow the scope with `testClasses`, `skipTestClasses`, `packages` or `annotations`.
4. `getAppAutomateRunStatus`: poll until the runs finish.
5. `getAppAutomateRunResults`: per-run outcomes plus the JUnit XML report.
6. `getTestDetails` / `getFailureLogs`: each run reports an Appium session id, so the usual session tools give you the video, screenshots and device logs for a failure.
`locale` and `timeZone` set the device environment for the run, and `build` groups related runs together in the dashboard.
## 7. Cross-browser screenshots
**Goal:** capture how a single URL renders across many browsers and operating systems at once for fast visual comparison, without opening interactive sessions.
> "Take full-page screenshots of testingbot.com in Chrome on Windows 11 and Safari on macOS, then give me the image links."
A typical tool sequence:
1. `getBrowsers`: (optional) confirm valid `browserName`/`version`/`os` combinations.
2. `takeScreenshot`: pass the `url` and a `browsers` array (each entry has `browserName`, optional `version` and `os`). Optionally set `resolution`, `waitTime` and `fullPage`. Returns a job id.
3. `retrieveScreenshots`: pass the `screenshotId` (the job id) to get the captured image URLs.
4. `getScreenshotList`: (optional) list earlier screenshot jobs.
This is a batch, fire-and-collect flow: `takeScreenshot` queues the job and `retrieveScreenshots` fetches the results once they are ready.
## 8. Drive a session with Playwright/Puppeteer via CDP
**Goal:** get a remote cloud browser plus a Chrome DevTools Protocol (CDP) endpoint so your own Playwright or Puppeteer scripts can connect to a TestingBot browser.
> "Start a CDP-enabled Chrome session on Windows 11 and give me the CDP URL so I can connect Playwright to it."
A typical tool sequence:
1. `createCdpSession`: pass `browserName` and `platform` (and optionally `browserVersion`, `screenResolution`, `timeZone`, `name`, `build`, `extraCapabilities`). Returns a remote browser and a CDP URL.
2. Connect your Playwright/Puppeteer code to the returned CDP URL and run your script.
3. `getTests` / `getTestDetails`: review the run afterwards on TestingBot.
Use this when you already have framework code and just want it pointed at the TestingBot grid, rather than driving the browser through the `tb_*` automation tools.
## 9. Watch or take over a session
**Goal:** watch an automation session in real time, or step in and drive it manually when something needs a human.
> "Open a Chrome session on Windows 11 and give me the live link so I can watch what the agent is doing."
How it works:
1. `tb_openBrowser` (or any session-starting tool) returns a live-view URL of the form `https://testingbot.com/tests//live?auth=`.
2. Open that URL in your browser to watch the session live; you can interact with it directly to take over.
3. `tb_listSessions`: list active automation sessions if you lose track of which is running.
4. `stopTest` or `tb_closeBrowser`, stop the session when you are done.
Every automation session returns its own live-view URL, so you can monitor or intervene at any point. This is useful for one-time captchas, MFA prompts, or simply seeing the agent's actions as they happen.
## 10. Inspect & debug a run
**Goal:** figure out why a test failed by pulling its details and logs after the fact.
> "Show me my last 5 tests, then pull the details and failure logs for the one that failed."
A typical tool sequence:
1. `getTests`: list recent tests (paginated via `offset`/`limit`) to find the session in question.
2. `getTestDetails`: full detail for a single session by `sessionId`.
3. `getFailureLogs`: fetch and filter session logs by `sessionId`; narrow with `logTypes` (selenium, browser, chrome, vm, appium) and `failuresOnly: true`.
4. `updateTest`: (optional) mark the test `passed`/`failed` or update its name/build once you know the outcome.
Working with builds instead of individual tests? Use `getBuilds` and `getTestsForBuild` to drill from a build down to its sessions, then debug each one the same way.
For the complete catalog of tools and every parameter referenced above, see the [Tool reference](https://testingbot.com/support/ai/mcp/tools). If a workflow misbehaves, the [Troubleshooting](https://testingbot.com/support/ai/mcp/troubleshooting) guide covers the most common issues, and the [FAQ](https://testingbot.com/support/ai/mcp/faq) answers the rest.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/ai/mcp/troubleshooting
# Troubleshooting
This page collects the issues we see most often when running the TestingBot MCP server, in an **Issue → Cause → Fix** format. If you have not configured the server yet, start with [Setup & installation](https://testingbot.com/support/ai/mcp/setup); for anything related to logging in, see [Authentication](https://testingbot.com/support/ai/mcp/authentication).
Most problems fall into one of two buckets: the server can't authenticate (so tools refuse to run), or a requested resource (a real device, a specific OS version) isn't available right now. Both are handled gracefully by the server, it returns guidance rather than failing silently.
When something goes wrong, the quickest diagnostic is to ask your agent to run `getUserInfo`. If it returns your account details, authentication is fine and the problem is elsewhere. If it replies "No TestingBot credentials configured", jump to [the credentials section](https://testingbot.com#credentials) below.
## Tools reply "Run tb\_login" or "No credentials"
### Issue
Every tool except `tb_login` replies with something like:
```bash
No TestingBot credentials configured. Run the tb_login tool to authenticate.
```
### Cause
The server is running in **degraded mode**. It could not find usable credentials, so it disables every tool that talks to TestingBot and only keeps `tb_login` available. The server looks for credentials in this order:
1. **Environment variables** : both `TESTINGBOT_KEY` _and_ `TESTINGBOT_SECRET` must be set. Setting only one is not enough.
2. **Credentials file** : `~/.testingbot/credentials`, written automatically after a successful `tb_login`.
3. **Degraded mode** : neither of the above was found.
### Fix
Pick whichever fits your environment:
- **Interactive desktop:** ask your agent to run the `tb_login` tool. A browser opens, you click Authorize, and credentials are captured automatically, with nothing pasted. No restart is needed; the next tool call will work.
- **SSH / remote / headless:** run `tb_login` with `mode` set to `device` (see [below](https://testingbot.com#headless-login)).
- **CI or a shared config:** set the key and secret in your client's `env` block. Generate them at [testingbot.com/members/user/edit](https://testingbot.com/members/user/edit).
```json
{
"mcpServers": {
"testingbot": {
"command": "npx",
"args": ["-y", "@testingbot/mcp-server@latest"],
"env": { "TESTINGBOT_KEY": "", "TESTINGBOT_SECRET": "" }
}
}
}
```
If you set only `TESTINGBOT_KEY` or only `TESTINGBOT_SECRET`, the env layer is ignored entirely and the server falls through to the credentials file (or degraded mode). Always set both, or use `tb_login` instead.
For the full authentication model (loopback vs device flow, profiles, and where credentials are stored), see [Authentication](https://testingbot.com/support/ai/mcp/authentication).
## Requested real device is unavailable, busy, or the wrong OS version
### Issue
You asked to automate a physical device (for example "open Safari on a real iPhone 15 Pro") and the agent comes back with guidance instead of a running session, it tells you the device is busy, the OS version isn't offered, or suggests an alternative.
### Cause
When `tb_openBrowser` is called with `realDevice: true`, the server runs a **pre-flight check** against the real-device cloud before consuming any minutes. If the exact device/version combination is currently in use or simply not offered, it returns advice _without_ starting a session, so you don't burn time on a session that can't start.
### Fix
- **Wait and retry** : physical devices are shared. If the device is just busy, try again shortly.
- **Pick an offered version** : ask the agent to list devices first (`getDevices`) and choose an available OS version. You can also browse the physical device list at [/support/app-automate/devices](https://testingbot.com/support/app-automate/devices).
- **Use an emulator or simulator instead** : drop `realDevice` (it defaults to `false`) to run the same mobile browser on an emulator/simulator. These are cheaper and far more plentiful, and are usually fine unless you specifically need real hardware (GPU, sensors, real Chrome/Safari).
Try a prompt like _"Open the mobile site on an emulated Pixel 9 running Android 14"_ to fall back to an emulator. See [Use cases & workflows](https://testingbot.com/support/ai/mcp/use-cases) for emulator vs real-device examples.
## Maestro / Espresso / XCUITest tools are missing or a run won't start
### Issue
Your agent says it has no tool for running Maestro flows or an Espresso suite, or `uploadMaestroApp` and `runAppAutomateTest` reject the files or options you gave them.
### Cause & fix
- **The tools don't exist at all** : App Automate support landed in `@testingbot/mcp-server` 1.1.0. If your client pinned an older version, update it. Running the server through `npx -y @testingbot/mcp-server@latest` always pulls the current release; a `.mcpb` bundle installed in Claude Desktop has to be re-downloaded from the [releases page](https://github.com/testingbot/mcp-server/releases). Restart the client afterwards so it re-reads the tool list.
- **The upload is rejected** : each tool accepts specific file types. Maestro takes `.apk`, `.aab`, `.ipa` or a zipped `.app`. For Espresso, the app is an `.apk`/`.aab` and the test suite is the instrumented test `.apk`. For XCUITest, the app is an `.ipa` and the test suite is a `.zip` of the runner bundle. Give the agent absolute paths.
- **An iOS run refuses to start** : `.ipa` builds need physical hardware. Set `realDevice: true` on `runMaestroTest` and `runAppAutomateTest` for iOS.
- **Flows upload but nothing runs** : `uploadMaestroFlows` replaces the flows for a project, so a later upload that omitted some files leaves only what you sent last. Upload the whole directory, and check `includeTags`/`excludeTags`, a tag filter that matches nothing produces a run with no flows.
- **Flows fail on the device but pass locally** : fetch `getMaestroFlowDetails` for the failing flow. It returns the error, the step-level report, and the video, screenshots and device logs, which is usually enough for the agent to spot a timing or selector problem.
### Runs that stay in WAITING
`WAITING` means the run is queued for a device. A specific `deviceName` on busy hardware can queue for a while, so widen it (wildcards and regex are accepted, for example `Pixel 8*` or `*`), drop `realDevice` to use an emulator or simulator, or check that your plan has a free parallel slot. `cancelMaestroRun` stops a run you no longer want.
## The screenshot doesn't show in my chat
### Issue
You asked for a screenshot with `tb_screenshot`, the tool reports success, but no image appears in your AI client's chat window.
### Cause
`tb_screenshot` returns the screenshot _two_ ways: an inline image and a saved PNG file with a clickable `file://` link. Many MCP clients do not render inline tool images in the conversation, so the picture is captured correctly, your client just isn't displaying it.
### Fix
- **Open the saved file** : the tool always returns a `file://` link to the PNG it wrote. Click or open that path to view the screenshot.
- **Control where it's written** : pass a `savePath` (a `.png` file path or a directory) so the image lands somewhere you can find easily. Without it, the server saves to your OS temp directory.
Example prompt:
> "Take a screenshot of the current page and save it to ~/Desktop/checkout.png"
## Browser login didn't open, or I'm on SSH / a remote host
### Issue
You ran `tb_login` but no browser window appeared, or you're working over SSH, in a container, or in a web-based client where opening a local browser isn't possible.
### Cause
The default `tb_login` mode is `auto`, which prefers the **loopback** flow (RFC 8252): it opens a browser on your machine and captures credentials on a local `127.0.0.1` callback. On a headless or remote host there is no local browser to open, so that flow can't complete.
### Fix
Use the **device-code** flow (RFC 8628) by running `tb_login` with `mode` set to `device`. The tool prints a short code; open [https://testingbot.com/device](https://testingbot.com/device) on any device, enter the code, and click Authorize. The flow is **resumable** , if it times out, just run `tb_login` again to finish.
| mode | Best for | How it works |
| --- | --- | --- |
| `auto` (default) | Most setups | Loopback on a desktop, falls back to device on headless hosts. |
| `loopback` | Local desktop | Opens your browser; credentials captured on a local 127.0.0.1 callback. Nothing pasted. |
| `device` | SSH / remote / headless / web clients | Prints a code; you enter it at https://testingbot.com/device. Resumable. |
You can also set a default with the `TESTINGBOT_AUTH_MODE` environment variable. More detail on [Authentication](https://testingbot.com/support/ai/mcp/authentication).
## Node version errors
### Issue
The server fails to start, or `npx` throws syntax/engine errors when launching `@testingbot/mcp-server`.
### Cause
The TestingBot MCP server requires a modern Node.js runtime. Older versions lack APIs the server (and its bundled `@testingbot/automation-mcp`) depend on.
### Fix
Install **Node.js 18 or newer** (20+ recommended). Check your version, then confirm the install command your client uses:
```bash
node --version
# should print v18.x or higher (v20+ recommended)
npx -y @testingbot/mcp-server@latest
```
If your MCP client uses a different Node than your shell (common with version managers like nvm), point the client's `command` at the correct Node binary, or make sure the right Node is first on your `PATH`.
## Sessions seem stuck or keep using minutes
### Issue
An automation session appears to be running long after you're done with it, and you're worried it's still consuming TestingBot minutes.
### Cause
Each `tb_openBrowser` call starts a real session that stays open until it's closed. If the agent moved on without calling `tb_closeBrowser`, the session can linger.
### Fix
- **List active sessions** : ask the agent to run `tb_listSessions` to see everything currently open.
- **Close them** : run `tb_closeBrowser` with the `sessionId` to end a session and free minutes immediately.
- **Stop an account-level test** : for a non-automation session you can use `stopTest` with its `sessionId`.
Idle sessions auto-close after a period of inactivity, so a forgotten session won't run indefinitely. but closing explicitly is faster and avoids wasted minutes. A good habit is to end a workflow with a prompt like _"close all my open browser sessions"_.
## Where are my logs
### Issue
A test failed and you want to see what happened, selenium, browser, Appium or VM logs for the session.
### Cause
Session logs live on TestingBot, not on your machine. The MCP server exposes them through the `getFailureLogs` tool rather than writing them locally.
### Fix
Ask the agent to fetch logs for the session with `getFailureLogs`. You can filter by log type and limit the volume:
| Param | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `sessionId` | string | Yes | – | The session to fetch logs for. |
| `logTypes` | array | No | all | Any of `selenium`, `browser`, `chrome`, `vm`, `appium`. |
| `failuresOnly` | boolean | No | false | Return only failure-related entries. |
| `maxBytesPerLog` | number | No | 50000 | Cap per log (1000–1000000 bytes). |
| `timeoutMs` | number | No | 15000 | Fetch timeout (1000–60000 ms). |
A typical debugging flow is `getTests` → `getTestDetails` → `getFailureLogs`. See the worked example under [Use cases & workflows](https://testingbot.com/support/ai/mcp/use-cases), or the full parameter list in the [Tool reference](https://testingbot.com/support/ai/mcp/tools).
## Noisy output or WDIO\_LOG\_LEVEL
### Issue
You're seeing unexpectedly verbose WebDriver output, or conversely you want more detail when diagnosing an automation problem.
### Cause
The bundled automation layer uses a WebDriver client whose verbosity is controlled by `WDIO_LOG_LEVEL`. The server sets this to `silent` by default so your client's chat stays clean, so this rarely needs changing.
### Fix
To get more detail while debugging, set `WDIO_LOG_LEVEL` in your client's `env` block (for example `error`, `warn`, `info` or `debug`). Leave it at `silent` for normal use.
```json
{
"mcpServers": {
"testingbot": {
"command": "npx",
"args": ["-y", "@testingbot/mcp-server@latest"],
"env": {
"TESTINGBOT_KEY": "",
"TESTINGBOT_SECRET": "",
"WDIO_LOG_LEVEL": "info"
}
}
}
}
```
Still stuck? Double-check your install and config on the [Setup](https://testingbot.com/support/ai/mcp/setup) page, confirm you're authenticated via [Authentication](https://testingbot.com/support/ai/mcp/authentication), or review the [FAQ](https://testingbot.com/support/ai/mcp/faq). For an overview of the whole integration, see [TestingBot MCP](https://testingbot.com/support/ai/mcp).
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/ai/mcp/faq
# Frequently asked questions
Answers to the most common questions about the [TestingBot MCP server](https://testingbot.com/support/ai/mcp): how it works with different AI assistants, how your data is handled, how authentication and billing work, and the difference between testing on emulators and real devices.
If you are just getting started, see the [Setup & installation](https://testingbot.com/support/ai/mcp/setup) guide, the [Authentication](https://testingbot.com/support/ai/mcp/authentication) guide and the full [Tool reference](https://testingbot.com/support/ai/mcp/tools).
Looking for in-browser, page-side AI tooling instead of the local MCP server? See [TestingBot WebMCP](https://testingbot.com/support/ai/webmcp), a different product. This page covers the local [TestingBot MCP server](https://testingbot.com/support/ai/mcp).
## General
Questions about which AI clients the server works with and what you need installed locally.
### Is this MCP server compatible with various AI models or specific to a particular provider?
MCP servers are LLM-agnostic, which means they work with any LLM model that supports the Model Context Protocol.
### Which MCP clients are supported?
The TestingBot MCP server works with Claude Desktop, Claude Code, Cursor, VS Code (Copilot/MCP), Windsurf, and any MCP-compatible client that can launch a stdio server. One-click installers are available for VS Code (https://testingbot.com/mcp/install?client=vscode) and Cursor (https://testingbot.com/mcp/install?client=cursor), and Claude can install the bundled `.mcpb` from the GitHub releases page. For any other client, use the standard JSON config shown on the [Setup page](https://testingbot.com/support/ai/mcp/setup).
### Do I need to install Appium, browsers or device emulators locally?
No. All browsers, emulators, simulators and real devices run on TestingBot's cloud. The only local requirement is Node.js 18+ (20+ recommended), which runs the MCP server itself via `npx -y @testingbot/mcp-server@latest`. Browser and mobile automation are handled by the bundled `@testingbot/automation-mcp` package, which drives remote sessions over WebDriver, you do not install Appium, Selenium, Chrome or any device tooling on your machine.
## Privacy & security
How your prompts, data and credentials flow between your AI client, the MCP server and TestingBot.
### Is the information that I feed through MCP used to train?
No, the MCP server does not send any information to an LLM. It's the LLM that is requesting information from the MCP server. The MCP server will perform API calls to the TestingBot API.
### Where are my credentials stored, and is it secure?
When you log in with the `tb_login` tool, your TestingBot key and secret are written to `~/.testingbot/credentials`, an INI file with profile sections such as `[default]`. The directory is created with `0700` permissions and the file with `0600`, so only your user account can read it. Credentials never leave your machine: they are used only by the MCP server to call the TestingBot API directly, and are never passed to the LLM provider. See [Authentication](https://testingbot.com/support/ai/mcp/authentication) for full details.
### What is the difference between TestingBot MCP and WebMCP?
TestingBot MCP is a local server you connect to your AI assistant (Claude, Cursor, VS Code, etc.) so it can manage your TestingBot account and drive real cloud browser and mobile sessions through natural language. [TestingBot WebMCP](https://testingbot.com/support/ai/webmcp) is a separate, in-browser approach. If you want your coding assistant or AI agent to interact with TestingBot's infrastructure, this MCP server is what you want.
## Authentication
How the server signs in to TestingBot and how to provide credentials in different environments.
### How does authentication to TestingBot happen?
There are three options. The easiest is to run the `tb_login` tool, which opens a browser login (loopback) on a desktop or prints a short device code for remote/headless setups, no key is ever pasted. Alternatively, set `TESTINGBOT_KEY` and `TESTINGBOT_SECRET` in your client's `env` block (get them at https://testingbot.com/members/user/edit). The server resolves credentials in this order: environment variables (both must be set), then the credentials file, then degraded mode. See [Authentication](https://testingbot.com/support/ai/mcp/authentication).
### Why do the tools reply "No TestingBot credentials configured. Run the tb\_login tool"?
That message means the server is running in degraded mode: it found no usable credentials in the environment or the credentials file, so every tool except `tb_login` is disabled. Run the `tb_login` tool to authenticate, or set `TESTINGBOT_KEY` and `TESTINGBOT_SECRET` in your client config. On SSH or headless hosts, use the device-code flow, see [Troubleshooting](https://testingbot.com/support/ai/mcp/troubleshooting).
### The browser login did not open, I am on SSH or a remote host. What do I do?
Use the device-code flow by running `tb_login` with `mode` set to `device`. It prints a short code; open https://testingbot.com/device in any browser, enter the code and click Authorize. The flow is resumable, if it times out, just run `tb_login` again to finish. The default `auto` mode already falls back to device login on headless machines.
## Devices & sessions
How emulators, simulators and real devices differ, and how to watch or take over a live run.
### What is the difference between an emulator/simulator and a real device?
Emulators (Android) and simulators (iOS) are virtual devices, they are cheaper and plentiful, so they are great for functional checks and CI. They are the default: `tb_openBrowser` with a `deviceName` and `platform` of Android or iOS uses `realDevice:false`. Set `realDevice:true` when you need physical hardware, real GPU, sensors and the real Chrome or Safari engine. Browse all available hardware at [/support/app-automate/devices](https://testingbot.com/support/app-automate/devices). You can also list devices through the agent with the `getDevices` tool.
### A requested real device is unavailable or busy, what happens?
When you request a real device with `realDevice:true` and that device or OS version is busy or not offered, `tb_openBrowser` runs a pre-flight check and returns guidance _without_ starting (and billing) a session. The agent can then wait and retry, pick an available OS version it suggests, or fall back to an emulator/simulator. See [Use cases](https://testingbot.com/support/ai/mcp/use-cases) for the full workflow.
### Can I watch or take over a session the agent is running?
Yes. Every automation session returns a live-view URL in the form `https://testingbot.com/tests//live?auth=`. Open it in your browser to watch the session in real time and interact with it directly when you need to step in.
## Usage, CI & teams
Questions about billing, continuous integration and sharing the configuration with your team.
### Does using the MCP server consume my TestingBot minutes?
Account and management actions, listing tests, fetching results, reading logs, managing builds or storage, do not consume testing minutes. Live and automation sessions do: opening a browser or device session (for example with `tb_openBrowser` or a live session tool) uses minutes while it runs. Close sessions with `tb_closeBrowser` when finished; idle sessions also auto-close. Use `tb_listSessions` to see what is currently open.
### Can I use the MCP server in CI or other automated environments?
Yes. For CI and headless environments, provide credentials through environment variables instead of the interactive login. Set `TESTINGBOT_KEY` and `TESTINGBOT_SECRET` in the `env` block of your MCP config:
```json
{
"mcpServers": {
"testingbot": {
"command": "npx",
"args": ["-y", "@testingbot/mcp-server@latest"],
"env": { "TESTINGBOT_KEY": "", "TESTINGBOT_SECRET": "" }
}
}
}
```
Get your key and secret at https://testingbot.com/members/user/edit. If you cannot use env vars, the device-code login flow also works on remote hosts.
### Can team members share a single configuration?
You can share the MCP config block itself (command and args), but each member should supply their own credentials. For shared or CI configs, an API key/secret in the `env` block is the most portable option. For local desktop use, each member runs `tb_login` once and credentials are written to their own `~/.testingbot/credentials` file. You can keep multiple sets of credentials side by side using profile sections and the `TESTINGBOT_PROFILE` environment variable. Team-level details (members, usage) are available through the `getTeam` and `getUsersInTeam` tools, see the [Tool reference](https://testingbot.com/support/ai/mcp/tools).
Still stuck? The [Troubleshooting](https://testingbot.com/support/ai/mcp/troubleshooting) guide covers common errors such as degraded mode, unavailable devices, screenshots that do not render in chat, and Node version issues.
## App Automate: Maestro, Espresso & XCUITest
Running native app test suites on devices through the agent.
### Can the agent run my Maestro, Espresso or XCUITest suite?
Yes, from version 1.1.0 of `@testingbot/mcp-server`. The agent uploads your app and tests, starts a run on a device of your choosing, polls it, and reports back with the JUnit report plus video, screenshots and device logs for anything that failed. See the [Tool reference](https://testingbot.com/support/ai/mcp/tools) for the tools and their parameters, and [Use cases](https://testingbot.com/support/ai/mcp/use-cases) for the full sequences.
### Does the agent wait while my suite runs?
No. App Automate runs are asynchronous by design: `runMaestroTest` and `runAppAutomateTest` return a run id straight away, and the agent polls `getMaestroRunStatus` or `getAppAutomateRunStatus` for progress. A suite that takes twenty minutes on devices does not hold your conversation open for twenty minutes.
### Can the agent write the Maestro flows for me?
It can. `uploadMaestroFlows` accepts inline YAML as well as a directory or a `.zip`, so an agent can author a flow and upload it without touching your filesystem. The `maestroCheatSheet` tool exists for exactly this: agents fetch the Maestro syntax reference before writing or repairing a flow. The usual loop is write, run, read `getMaestroFlowDetails` for the failure, fix, retry. Our [Maestro documentation](https://testingbot.com/support/app-automate/maestro) covers the flow format in full.
### What is the difference between these tools and the Appium tools?
The App Automate tools run a _pre-existing test suite_ (Maestro flows, an Espresso instrumentation APK, an XCUITest bundle) as a batch job on our device cloud, and hand you a report. The `appium_*` tools drive a single live app session step by step, with the agent deciding each tap and assertion as it goes. Use App Automate for regression suites, Appium for exploratory work and one-off interactions.
## More resources
- [TestingBot MCP overview](https://testingbot.com/support/ai/mcp), what it is and what it can do.
- [Setup & installation](https://testingbot.com/support/ai/mcp/setup), install and configure your AI client.
- [Authentication](https://testingbot.com/support/ai/mcp/authentication), browser login, device code and API keys.
- [Tool reference](https://testingbot.com/support/ai/mcp/tools), every available tool and its parameters.
- [Use cases & workflows](https://testingbot.com/support/ai/mcp/use-cases), example prompts and tool sequences.
- [Troubleshooting](https://testingbot.com/support/ai/mcp/troubleshooting), fixes for common issues.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/ai/webmcp
# Testing WebMCP Tools
WebMCP is a proposed browser API that lets a website expose its own actions as structured, callable tools to in-browser AI agents. Instead of an agent guessing where to click, your site registers tools with `navigator.modelContext.registerTool()`, each with a name, a description and a JSON Schema for its input.
If your website ships WebMCP tools, you need to verify they keep working: that every tool is registered, that its input schema is valid, and that calling it returns the result you expect. This page shows how to run those checks on real browsers in the TestingBot cloud with Selenium, Playwright and Puppeteer.
WebMCP is different from the [TestingBot MCP Server](https://testingbot.com/support/ai/mcp). The TestingBot MCP Server is a tool you install locally to let AI assistants (Claude, Cursor, VS Code) drive your TestingBot account through the REST API. WebMCP is a browser feature your own website implements so AI agents can call your site's actions. This page is about testing the latter.
## Why test your WebMCP tools
WebMCP tools are part of your public interface, just like a form or an API endpoint. They break in the same ways:
- **Registration** : a deploy removes or renames a tool, so the agent no longer finds it.
- **Schema** : an input schema becomes invalid or stops matching what the tool actually accepts.
- **Behavior** : the tool registers fine but its `execute()` callback returns the wrong result, throws, or silently changes shape.
- **Cross-browser** : a tool works in one Chrome version but not another, or only when the page reaches a certain state.
Running these checks on real browsers in CI catches regressions before your users' AI agents do.
## Prerequisites
- A TestingBot account with your [key and secret](https://testingbot.com/members/user/security).
- A website that registers WebMCP tools through `navigator.modelContext`.
- Google Chrome 149 or higher (WebMCP is currently a Chromium-only origin trial).
- The WebMCP testing surface enabled, so your test can list and invoke tools without a live AI agent. This exposes `navigator.modelContextTesting` with `getTools()` and `executeTool()`.
WebMCP is experimental and runs behind Chrome feature flags during the origin trial. Launch Chrome 149 or higher with `--enable-features=WebMCPTesting,DevToolsWebMCPSupport`, which exposes `navigator.modelContextTesting` so your tests can list and invoke tools without a live agent. See the [Chrome WebMCP documentation](https://developer.chrome.com/docs/ai/webmcp) for details. If you would like WebMCP enabled on the TestingBot grid for your account, [contact support](https://testingbot.com/support).
## Testing WebMCP tools
The pattern is the same in every framework: launch Chrome 149 or higher with the WebMCP testing flag, navigate to your page, then call `navigator.modelContextTesting.getTools()` to read the registered tools and `navigator.modelContextTesting.executeTool(name, input)` to invoke one and inspect its result.
[Selenium (Python)](https://testingbot.com#) [Selenium (Node.js)](https://testingbot.com#) [Playwright](https://testingbot.com#) [Puppeteer](https://testingbot.com#)
```python
from selenium import webdriver
options = webdriver.ChromeOptions()
options.browser_version = "latest" # Chrome 149 or higher
# Enable the WebMCP testing surface (navigator.modelContextTesting) and the DevTools WebMCP domain
options.add_argument("--enable-features=WebMCPTesting,DevToolsWebMCPSupport")
options.set_capability("platformName", "WINDOWS")
options.set_capability("tb:options", {
"key": "YOUR_KEY",
"secret": "YOUR_SECRET",
"name": "WebMCP tool test",
})
driver = webdriver.Remote(
command_executor="https://hub.testingbot.com/wd/hub",
options=options,
)
try:
driver.get("https://your-site.com")
if not driver.execute_script("return 'modelContextTesting' in navigator"):
raise RuntimeError("WebMCP testing surface is not available in this browser")
# 1. List the tools your site registered with navigator.modelContext
tools = driver.execute_async_script("""
const done = arguments[arguments.length - 1];
navigator.modelContextTesting.getTools().then(done);
""")
names = [t["name"] for t in tools]
assert "searchProducts" in names, names
# 2. Invoke a tool and check the structured result
result = driver.execute_async_script("""
const [name, input, done] = arguments;
navigator.modelContextTesting.executeTool(name, input).then(done);
""", "searchProducts", {"query": "shoes"})
assert result["total"] > 0
finally:
driver.quit()
```
```javascript
const { Builder } = require('selenium-webdriver')
const chrome = require('selenium-webdriver/chrome')
const options = new chrome.Options()
options.setBrowserVersion('latest') // Chrome 149 or higher
// Enable the WebMCP testing surface and the DevTools WebMCP domain (Chrome 149+)
options.addArguments('--enable-features=WebMCPTesting,DevToolsWebMCPSupport')
options.set('tb:options', {
key: 'YOUR_KEY',
secret: 'YOUR_SECRET',
name: 'WebMCP tool test'
})
;(async () => {
const driver = await new Builder()
.usingServer('https://hub.testingbot.com/wd/hub')
.forBrowser('chrome')
.setChromeOptions(options)
.build()
try {
await driver.get('https://your-site.com')
// 1. List the registered tools
const tools = await driver.executeAsyncScript(`
const done = arguments[arguments.length - 1];
navigator.modelContextTesting.getTools().then(done);
`)
console.log(tools.map(t => t.name))
// 2. Invoke a tool and check its result
const result = await driver.executeAsyncScript(`
const [name, input, done] = arguments;
navigator.modelContextTesting.executeTool(name, input).then(done);
`, 'searchProducts', { query: 'shoes' })
console.log(result)
} finally {
await driver.quit()
}
})()
```
```javascript
const playwright = require('playwright')
const capabilities = {
'tb:options': {
key: 'YOUR_KEY',
secret: 'YOUR_SECRET',
name: 'WebMCP tool test'
},
browserName: 'chrome',
browserVersion: 'latest', // Chrome 149 or higher
// Enable the WebMCP testing surface and the DevTools WebMCP domain (Chrome 149+)
'goog:chromeOptions': { args: ['--enable-features=WebMCPTesting,DevToolsWebMCPSupport'] }
}
;(async () => {
const browser = await playwright.chromium.connect(
`wss://cloud.testingbot.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}`
)
const page = await browser.newPage()
await page.goto('https://your-site.com')
// 1. List the registered tools
const tools = await page.evaluate(() => navigator.modelContextTesting.getTools())
console.log(tools.map(t => t.name))
// 2. Invoke a tool and check its result
const result = await page.evaluate(
([name, input]) => navigator.modelContextTesting.executeTool(name, input),
['searchProducts', { query: 'shoes' }]
)
console.log(result)
await browser.close()
})()
```
```javascript
// Puppeteer 25.1.0 or higher ships a native page.webmcp API
const puppeteer = require('puppeteer-core')
const capabilities = {
'tb:options': {
key: 'YOUR_KEY',
secret: 'YOUR_SECRET',
name: 'WebMCP tool test'
},
browserName: 'chrome',
browserVersion: 'latest', // Chrome 149 or higher
// Enable the WebMCP testing surface and the DevTools WebMCP domain
'goog:chromeOptions': { args: ['--enable-features=WebMCPTesting,DevToolsWebMCPSupport'] }
}
;(async () => {
const browser = await puppeteer.connect({
browserWSEndpoint: `wss://cloud.testingbot.com/puppeteer?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}`
})
const page = await browser.newPage()
await page.goto('https://your-site.com')
// 1. List the tools your site registered with navigator.modelContext
const tools = page.webmcp.tools()
console.log(tools.map(t => t.name))
// 2. Invoke a tool and check its structured result
const tool = tools.find(t => t.name === 'searchProducts')
if (tool) {
const result = await tool.execute({ query: 'shoes' })
if (result.status === 'Completed') {
console.log(result.output)
}
}
await browser.close()
})()
```
Older Puppeteer versions, and Selenium and Playwright, do not have a native WebMCP API. There you can reach the same tools by evaluating `navigator.modelContextTesting.getTools()` and `executeTool()` in the page, as shown in the other tabs.
After the run, mark the test as passed or failed in your TestingBot dashboard the same way you do for any other test, either from your test script or through the [REST API](https://testingbot.com/support/api).
## What to assert
A useful WebMCP test suite checks more than "the page loaded". Good assertions include:
- **Tool presence** : every tool you expect is listed by `getTools()`, with the right name and description.
- **Schema correctness** : each tool's `inputSchema` declares the parameters you intend, including required fields and enums.
- **Happy path** : `executeTool()` with valid input returns the expected structured result.
- **Error handling** : invalid input is rejected cleanly rather than throwing an unhandled error.
- **State changes** : write tools (for example "add to cart") actually change the page state, and read-only tools do not.
## Browser support and caveats
WebMCP is an early, evolving standard. It is a W3C Community Group draft, not yet on the standards track, and is currently implemented only in Chromium-based browsers behind an origin trial (Chrome 149 and higher). The API has changed several times, so treat any integration as experimental and isolate your WebMCP test helpers so you can update them when the API moves.
- WebMCP is Chromium-only today. Firefox and Safari have not committed to it.
- The exact flag and API names can change between Chrome releases during the origin trial.
- Your site must enable WebMCP (through its origin-trial token or flag) for `navigator.modelContext` to be available.
## FAQ
### What is the difference between WebMCP and the TestingBot MCP Server?
They are unrelated. WebMCP is a browser API your website implements so in-browser AI agents can call your site's actions. The [TestingBot MCP Server](https://testingbot.com/support/ai/mcp) is a separate tool you install locally so AI assistants can manage your TestingBot account through the REST API. This page covers testing the WebMCP tools your own site exposes.
### Which browsers support WebMCP?
WebMCP is currently available only in Chromium-based browsers (Chrome 149 and higher) as an origin trial. Firefox and Safari have not committed to it yet. You can run your tests on Chrome in the TestingBot cloud today.
### Do I need an AI agent to test my WebMCP tools?
No. The WebMCP testing surface (`navigator.modelContextTesting`) lets your test list and invoke tools directly, so you can assert on the results in regular Selenium, Playwright or Puppeteer tests without involving a live AI model.
### Is WebMCP production ready?
Not yet. WebMCP is an early standard in an origin trial and the API is still changing. It is a good time to experiment and to start testing your tools, but keep your WebMCP test helpers isolated so they are easy to update as the standard evolves.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/web-live
# Live Browser and Mobile Testing
Live testing lets you drive a remote browser, simulator or real mobile device in the TestingBot cloud with your own mouse and keyboard, exactly as if it were running locally. Below are answers to the questions we get most often about Live testing. For automated test execution see [Web Automate](https://testingbot.com/support/web-automate) instead.
## Is usage limited in terms of minutes or users?
With an upgraded plan you can use as many Live testing minutes as you like, with as many co-workers as your seat count allows. There are no per-test or per-day limits.
Trial accounts can use up to 60 Live testing minutes in total. See [our pricing plans](https://testingbot.com/pricing) for current seat and concurrency limits.
## What do I get with a free trial?
Trial accounts can use up to 60 free minutes of Live testing.
Once you open a browser in our cloud, you get 5 minutes of unlimited interactive access to that session. With an upgraded plan, Live testing minutes are unlimited within your plan.
## Is Live testing on TestingBot secure?
None of your data is stored on the test VMs. Every Live session runs on a freshly-provisioned virtual machine that is destroyed when the session ends.
VMs use up-to-date security patches and run inside a hardened EU datacenter behind firewalls. See [our security and compliance overview](https://trust.testingbot.com/) for GDPR and CSA STAR details.
## How can I test my staging site or a website on my own network?
You can use TestingBot to test a staging or internal site that is not publicly reachable. Start the [TestingBot Tunnel](https://testingbot.com/support/tunnel) on your local machine or in CI, pick a browser in the dashboard, and the remote browser will reach your private URLs through the encrypted tunnel as if it were on your network.
## Do I need to download any software?
No. You do not need to install any software, plugin or browser extension.
In the member area, pick the browser, version and platform you want to test on. Once we boot a virtual machine for you, you control the machine from inside your own browser. For example, you can drive a remote Safari on macOS from a local Chrome on Windows.
## Which browsers and platforms do you provide for Live testing?
The current and recent versions of Chrome, Firefox, Safari and Microsoft Edge on Windows, macOS and Linux, plus mobile Safari on real iPhones and iPads and mobile Chrome on real Android devices and emulators.
Legacy browsers including older Internet Explorer versions are still available for compliance and regression work.
See the full [list of supported browsers](https://testingbot.com/support/web-automate/browsers) for the current versions on the grid.
## What developer tools are pre-installed on the browsers?
Every Live browser has its built-in developer tools available: Chrome DevTools, Firefox DevTools, Safari Web Inspector, and Edge DevTools. Open them with F12 or the usual keyboard shortcut. The remote desktop also includes standard system utilities for network inspection. No extension installation is required.
## Is audio enabled during testing?
Audio is enabled on all desktop Live sessions and on iOS simulators and Android emulators.
On real Android devices, audio is available on devices running Android 12 or higher. On real iOS devices, audio is available on a select group of devices running iOS 14 or higher.
## Can I embed Live testing in my own website or product?
Yes, you can embed TestingBot's Live testing interface in your own app or website using an iframe. This is useful for QA dashboards, internal tools and bug-reporting flows that need an interactive remote browser.
1. Start a Live session through an API call.
2. Fetch the `sessionId` of the session and generate an `authentication hash`.
3. Pass the `sessionId` and `authentication hash` to a liveview URL and embed it with an iframe.
```javascript
const TestingBot = require('testingbot-api');
const tb = new TestingBot({
api_key: process.env.TESTINGBOT_KEY,
api_secret: process.env.TESTINGBOT_SECRET
});
const options = {
capabilities: {
browserName: 'chrome',
browserVersion: 'latest',
platform: 'WIN11',
'tb:manual': true,
}
};
const session = await tb.createSession(options);
const sessionId = session.session_id;
const authHash = tb.getAuthenticationHashForSharing(sessionId);
const liveViewUrl = `https://testingbot.com/tests/${sessionId}/live?auth=${authHash}`;
```
You can then embed the live view URL in an iframe as follows:
```html
```
The result is a fully functional Live testing interface embedded in your own application. For enterprise embed deals and dedicated infrastructure, [contact our team](https://testingbot.com/contact/new).
## Next steps
- [Live ChromeOS testing](https://testingbot.com/support/web-live/chromeos). Test on a real Chromebook with mouse and keyboard.
- [Browser extension testing](https://testingbot.com/support/web-live/extension). Install and test Chrome, Firefox and Edge extensions on the live browser.
- [Start from a link](https://testingbot.com/support/web-live/start-link). Open a browser and URL straight from a pull request or bug report.
- [Live app testing](https://testingbot.com/support/app-live). Interactive testing on real iPhones, iPads and Android phones.
- [Web Automate](https://testingbot.com/support/web-automate). Run Selenium, Playwright, Puppeteer and Cypress tests on the same grid.
- [TestingBot Tunnel](https://testingbot.com/support/tunnel). Test internal and staging sites.
- [Pricing](https://testingbot.com/pricing). Live and Automate plans, seat counts and concurrency limits.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/web-live/extension
# Browser Extension
- [What is the TestingBot Browser Extension?](https://testingbot.com#usage)
- [Installation](https://testingbot.com#installation)
- [Using the Browser Extension](https://testingbot.com#usage)
## What is the TestingBot Browser Extension?
The TestingBot Browser Extension can be installed on your [Chrome](https://chrome.google.com/webstore/detail/testingbot/nhjmiaiamlfblfagaodhkeabfbnmdmff) or [Firefox](https://addons.mozilla.org/addon/testingbot/) browser, for free.
Once installed, you can visit any website with your own Chrome or Firefox browser, and click the TestingBot extension icon. A window will pop up, allowing you to choose on which remote browser you want to test the current page on.
Upon selecting a browser, the current page will be available for testing on a remote TestingBot browser.
For example, you are using Chrome but want to make sure that the website you are currently seeing looks and functions correctly on a macOS Safari browser. With the extension, you can simply choose the remote browser configuration. You will be redirected to a live view of the remote browser, where you can use your mouse and keyboard to interact and test the webpage.
## Installation
The TestingBot Browser Extension is currently available for [Chrome](https://chrome.google.com/webstore/detail/testingbot/nhjmiaiamlfblfagaodhkeabfbnmdmff) and [Firefox](https://addons.mozilla.org/addon/testingbot/).
Simply click the **Add Extension** button to add the extension to your browser.
A TestingBot icon will now appear in your browser's navigation bar. You can pin the extension to make sure it remains visible at all times.
## Using the Browser Extension
After installation, you can simply navigate to any website you want to test. Click the TestingBot extension icon to open a popup with browser choices.
Make sure you are logged in to TestingBot while using the extension.
---
URL: https://testingbot.com/support/web-live/chromeos
# ChromeOS Manual Testing
TestingBot gives you the ability to test your website on a remote ChromeOS device emulator, such as a Chromebook.
Simply enter the URL of the website you want to test and choose between 3 different ChromeOS configurations:
- ChromeOS Small: 1366x768 - 160dpi
- ChromeOS Medium: 3840x2160 - 320dpi
- ChromeOS Large: 1920x1080 - 160dpi
## Starting a live ChromeOS session
Simply [start a new manual session](https://testingbot.com/members/manual/new) by clicking the Android category and choosing one of the three ChromeOS configurations.
A remote session will be started and you will be able to interact with the ChromeOS device emulator with your own mouse and keyboard.
An audio stream will be started, so you can hear the sounds from the ChromeOS device emulator.
## Enabling/disabling Dark Mode
You can easily enable or disable Dark Mode in the ChromeOS emulator by clicking the "Device Settings" button in taskbar, then toggling the "Enable Dark Mode" switch.
## Taking screenshots
To take a live screenshot of the current ChromeOS session, simply click the "Photo" button in the taskbar. This will capture the current view and open the screenshot in a photo editor, which you can use to annotate or modify the screenshot, before downloading or sharing it.
## Change GPS
You can use the "GPS" button in the taskbar to change the GPS location of the ChromeOS device emulator. This is useful for testing location-based features on your website.
---
URL: https://testingbot.com/support/web-live/start-link
# Open a Live Web Session from a Link
You can link straight to a live session on a specific browser, version and operating system, with your URL already loaded. Use it in a pull request to open the preview deploy in the browser that needs checking, or in a bug report to reproduce on the exact browser it was reported on.
```bash
https://testingbot.com/members/manual/start?url=&browserName=chrome&version=latest&platform=WIN11
```
## Parameters
`browserName`, `version` and `platform` take the same values as the capabilities of an automated test, so you can copy them from your Selenium or Playwright configuration or from [GET /v1/browsers](https://testingbot.com/support/api/devices#browsers).
| Parameter | Description |
| --- | --- |
| `url` | The page to open. URL-encode it. `https://` is added when the scheme is left off. |
| `browserName` | For example `chrome`, `firefox`, `microsoftedge` or `safari`. |
| `version` | A version such as `155` (any 155.x) or `>=150`, or `latest`, `latest-1`, `latest-2` and so on. Defaults to `latest`. |
| `platform` | The operating system, for example `WIN11`, `WIN10`, `TAHOE`, `SEQUOIA` or `LINUX`. |
| `browser` | Instead of the three above: the numeric `id` of a browser from [GET /v1/browsers](https://testingbot.com/support/api/devices#browsers). |
Examples:
```bash
https://testingbot.com/members/manual/start?url=https%3A%2F%2Fexample.com&browserName=safari&version=latest&platform=TAHOE
https://testingbot.com/members/manual/start?url=https%3A%2F%2Fexample.com&browserName=firefox&version=latest-2&platform=WIN10
```
## What happens on click
Opening the link never starts a session by itself. It shows a confirmation page with the URL and the browser. Click **Start session** to begin.
- If you are not logged in, you are asked to log in first and then return to the confirmation page.
- The session runs under the account of the person who clicks the link.
- If no browser matches the link, or the URL cannot be opened, the page says so and offers the browser picker instead.
- To test a local or staging URL, start [TestingBot Tunnel](https://testingbot.com/support/tunnel) before opening the link.
## Post a link on pull requests
If your CI deploys a preview of each pull request, comment a link that opens the preview in the browsers you care about. A GitHub Actions example:
```yaml
- name: Comment live test links
run: |
URL=$(jq -rn --arg u "$PREVIEW_URL" '$u|@uri')
BASE="https://testingbot.com/members/manual/start?url=$URL"
gh pr comment ${{ github.event.pull_request.number }} --body "Check this preview live:
- [Safari on macOS Tahoe]($BASE&browserName=safari&version=latest&platform=TAHOE)
- [Chrome on Windows 11]($BASE&browserName=chrome&version=latest&platform=WIN11)
- [Firefox on Windows 10]($BASE&browserName=firefox&version=latest&platform=WIN10)"
env:
PREVIEW_URL: ${{ steps.deploy.outputs.preview_url }}
GH_TOKEN: ${{ github.token }}
```
## Notes
- Use `latest` rather than a fixed version in links that stay around: browser versions are retired over time.
- Your plan's limits apply as usual, for example the number of parallel live sessions.
- To open a real device with one of your uploaded apps instead, see [Live App Testing: Start from a Link](https://testingbot.com/support/app-live/start-link).
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/app-live
# Live Mobile App Testing
Take control over our real devices with your mouse and keyboard, straight from your own browser, without having to install any plugin or program.
Simply upload your iOS or Android app to TestingBot, pick the device you want to test on, and TestingBot will give you a live session with the real device.
## [Upload App](https://testingbot.com/support/app-live/upload-app)
Upload, manage, and delete your iOS and Android apps via the UI or API.
## [Select Device](https://testingbot.com/support/app-live/select-device)
Choose from real devices and virtual devices. Understand time limits and device options.
## [Start from a Link](https://testingbot.com/support/app-live/start-link)
Choose from real devices and virtual devices. Understand time limits and device options.
## [Test Interface](https://testingbot.com/support/app-live/test-interface)
Explore toolbar actions: screenshots, rotation, clipboard, GPS, locale, and more.
## [Developer Tools](https://testingbot.com/support/app-live/developer-tools)
App Inspection, Appium Inspector, and real-time device logs for debugging.
## [Local Testing](https://testingbot.com/support/app-live/local-testing)
Test apps that connect to your local network using TestingBot Tunnel.
## [Platform-Specific](https://testingbot.com/support/app-live/chromeos)
ChromeOS app testing and Apple Pay testing on real devices.
## Developer Tools
TestingBot provides powerful developer tools built into the live testing interface. Inspect elements, find locators, view real-time logs, and use the full Appium Inspector — all without leaving your session.
[Explore Developer Tools →](https://testingbot.com/support/app-live/developer-tools)
## Audio
Audio is available during manual mobile app testing, for certain devices.
### Virtual Devices
Audio is available for all Android emulators and iOS Simulators. TestingBot will stream the audio generated by the virtual device to your web browser.
### Physical Devices
Android devices with Android version 12 and higher will stream the audio to your browser, you will hear all the audio generated on the device.
Older physical devices do not have the audio streaming capabilities. These will only stream a live video view of the device.
## Developer Resources
- [API Reference](https://testingbot.com/support/api)
- [Integrations](https://testingbot.com/integrations)
- [Devices](https://testingbot.com/support/app-automate/devices)
- [TestingBot Tunnel](https://testingbot.com/support/tunnel)
- [Features](https://testingbot.com/features/manual-mobile-testing)
- [Private Devices](https://testingbot.com/enterprise/private-device-cloud)
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/app-live/upload-app
# Uploading a Mobile App
There are several ways to get your app onto a TestingBot device for live testing. You can upload your own app file, install directly from the App Store or Play Store, use TestFlight for beta apps, or upload programmatically via our API.
## Upload via the UI
To upload a mobile app through the TestingBot UI, simply go to the member dashboard and:
- Click **Live App Testing**
- Under **Select Source** click **Uploaded Apps**
- Click **Upload App** to browse for the file ([see supported formats](https://testingbot.com#formats))
## Other App Sources
Besides uploading your own app file, you can install apps directly from these sources:
### [App Store](https://testingbot.com/support/app-live/app-store)
Install iOS apps directly from the Apple App Store on real iOS devices.
### [Play Store](https://testingbot.com/support/app-live/play-store)
Install Android apps directly from the Google Play Store on real Android devices.
### [TestFlight](https://testingbot.com/support/app-live/testflight)
Install beta iOS apps via Apple TestFlight on supported real devices.
## Upload via the API
You can programmatically upload apps using the [TestingBot REST API](https://testingbot.com/support/api/storage#upload). This is useful for CI/CD pipelines and automated workflows.
```bash
curl -X POST "https://api.testingbot.com/v1/storage" \
-u "YOUR_KEY:YOUR_SECRET" \
-F "file=@/path/to/your/app.apk"
```
The API returns a unique `app_url` that you can use to start a live testing session or pass to your automated tests.
See the full [API documentation](https://testingbot.com/support/api/storage#upload) for more details, including how to list and delete uploaded apps.
## Supported File Formats
Currently we accept these file formats:
- `.aab`, `.apk` for Android apps
- `.ipa` for physical iOS devices, `.zip` for iOS simulators
## Deleting a Mobile App
To delete a mobile app, you can use the TestingBot UI or use our [REST API](https://testingbot.com/support/api/storage#filedelete).
### Delete via the UI
To delete a mobile app through the TestingBot UI, simply go to the member dashboard and:
- Click **Live App Testing**
- Under **Select Source** click **Uploaded Apps**
- Click the trash icon in the bottom right corner of the list
- Select one or more apps you wish to delete
- Click **Delete Apps**
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/app-live/app-store
# Install Apps from the App Store
TestingBot allows you to install iOS apps directly from the Apple App Store onto real iOS devices during a live testing session. This lets you test production apps without needing the source code or an .ipa file.
## How to Install an App from the App Store
To install an app from the App Store on a TestingBot device:
- Go to the [Live App Testing](https://testingbot.com/members/apps/devices/new) page
- Under **Select Source** , click **App Store**
- Sign in with your Apple ID
- Search for the app you want to install
- Click **Install** and select a device to start your testing session
## Requirements
- A valid **Apple ID** is required to sign in to the App Store
- The app must be available in the App Store (not removed or region-restricted)
- Only available on **real iOS devices** (not simulators)
## Limitations
- Apps that require in-app purchases may have limited functionality during testing
- Some apps may have DRM restrictions that prevent installation on shared devices
- Your Apple ID credentials are only used during the session and are not stored by TestingBot
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/app-live/play-store
# Install Apps from the Google Play Store
TestingBot allows you to install Android apps directly from the Google Play Store onto real Android devices during a live testing session. This lets you test production apps without needing the source code or an .apk file.
## How to Install an App from the Play Store
To install an app from the Google Play Store on a TestingBot device:
- Go to the [Live App Testing](https://testingbot.com/members/apps/devices/new) page
- Under **Select Source** , click **Play Store**
- Sign in with your Google account
- Search for the app you want to install
- Click **Install** and select a device to start your testing session
## Requirements
- A valid **Google account** is required to sign in to the Play Store
- The app must be available on the Google Play Store
- Only available on **real Android devices** (and some Android emulators)
## Limitations
- Some apps may not be available on all device models due to compatibility restrictions set by the developer
- Apps that require in-app purchases may have limited functionality during testing
- Your Google account credentials are only used during the session and are not stored by TestingBot
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/app-live/testflight
# Install Beta Apps via TestFlight
TestingBot allows you to install beta iOS apps via Apple TestFlight on real devices. This is useful for testing pre-release versions of your app before they go live on the App Store.
## How to Install an App via TestFlight
To install a beta app via TestFlight on a TestingBot device:
- Go to the [Live App Testing](https://testingbot.com/members/apps/devices/new) page
- Under **Select Source** , click **TestFlight**
- Sign in with your Apple ID
- You will see a list of beta apps you have been invited to test
- Click **Install** on the app you want to test, then select a device
## Requirements
- A valid **Apple ID** is required to sign in to TestFlight
- You must have been **invited as a tester** by the app developer, or have a public TestFlight link
- Only available on **real iOS devices** (not simulators)
- TestFlight is pre-installed on TestingBot's real iOS devices
## Limitations
- TestFlight beta apps have an expiration date set by the developer
- Not all devices may be compatible with every beta build, depending on the developer's configuration
- Your Apple ID credentials are only used during the session and are not stored by TestingBot
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/app-live/select-device
# Selecting a Mobile Device
Once you've selected a mobile app, you will need to select the device you want to run tests on. You can choose to select a Virtual Device or a Real Device by using the navigation tabs on the page.
## Real Devices
To use a Real Device, click the [Real Device Testing](https://testingbot.com/members/apps/devices/new) tab.
- Click an available device from the devices list
- Make sure to select an iOS device if you've selected an iOS app (`.ipa`), or an Android device if you've selected an Android app (`.apk`)
- Click the **Start** button
## Virtual Devices
To use a Virtual Device, click the [Emulator & Simulator Testing](https://testingbot.com/members/apps/new) tab.
- Select an app you've uploaded, or the sample Android/iOS TestingBot app.
- Click the Android or iOS tab in the virtual device list
- Select an iOS simulator if you've selected an iOS app (`.ipa`), or an Android emulator if you've selected an Android app (`.apk`)
- Click the **Start** button
## Time Limits and Timeouts
- Live tests will last a maximum of 5 minutes at a time for trial users.
- Live tests for paid users will timeout after 15 minutes of inactivity
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/app-live/start-link
# Open a Live App Session from a Link
You can link straight to a live session on a real device with one of your uploaded apps installed. This is useful in CI: upload the build of a pull request, then post a link on the pull request so a reviewer can open the app on a device with one click.
```bash
https://testingbot.com/members/apps/start?app=tb://&device=
```
## Parameters
| Parameter | Description |
| --- | --- |
| `app` | The `app_url` returned when you [upload an app](https://testingbot.com/support/api/storage#upload) (`tb://`), or just the appkey. Must be an `.apk` or `.ipa` uploaded to the same TestingBot account. |
| `device` | The `id` of a real device, as returned by [GET /v1/devices](https://testingbot.com/support/api/devices#devices). |
| `deviceName` | Instead of `device`: the device name, for example `Pixel 8`. When several devices match, a free one is preferred, then the newest OS version. |
| `platformVersion` | Optional, with `deviceName`. `14` matches 14 and 14.x. |
Examples:
```bash
https://testingbot.com/members/apps/start?app=tb://35576f774feba47fa4e4b3b9&device=28
https://testingbot.com/members/apps/start?app=tb://35576f774feba47fa4e4b3b9&deviceName=Pixel%208&platformVersion=14
```
## What happens on click
Opening the link never starts a session by itself. It shows a confirmation page with the app (name, version and bundle identifier) and the device, and whether that device is free. Click **Start session** to begin.
- If you are not logged in, you are asked to log in first and then return to the confirmation page.
- The session runs under the account of the person who clicks the link, so the app must belong to that account. Team members share the team's apps.
- If the device is in use, the page offers to check again or to pick another device.
- If the app cannot run on the device (an iOS app on an Android device, or a simulator `.zip`), the page says so.
## Post a link on pull requests
Upload the build with the [storage API](https://testingbot.com/support/api/storage#upload), read `app_url` from the response and comment the link on the pull request. A GitHub Actions example:
```yaml
- name: Upload build to TestingBot
id: upload
run: |
APP_URL=$(curl -s -u "$TB_KEY:$TB_SECRET" \
-F "file=@app/build/outputs/apk/debug/app-debug.apk" \
https://api.testingbot.com/v1/storage | jq -r .app_url)
echo "app_url=$APP_URL" >> "$GITHUB_OUTPUT"
env:
TB_KEY: ${{ secrets.TB_KEY }}
TB_SECRET: ${{ secrets.TB_SECRET }}
- name: Comment live test link
run: |
gh pr comment ${{ github.event.pull_request.number }} --body \
"Test this build on a real device: https://testingbot.com/members/apps/start?app=${{ steps.upload.outputs.app_url }}&deviceName=Pixel%208"
env:
GH_TOKEN: ${{ github.token }}
```
## Notes
- Upload each pull request's build as a new app, so each link keeps pointing at its own build. Replacing the binary under an existing appkey (`POST /v1/storage/`) changes what older links open.
- Prefer `deviceName` over `device` in shared links: a specific device may be in use, another unit of the same model often is not.
- Your plan's limits apply as usual, for example the number of parallel live sessions.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/app-live/test-interface
# Live Test Interface
The Live Test Interface provides a comprehensive set of tools for interacting with your mobile app during live testing sessions. You can take screenshots, rotate the device, copy to clipboard, simulate GPS locations, change locale settings and more.
## Toolbar Actions
| Icon | Name | Description |
| --- | --- | --- |
| | Switch Device | Change to a different device. This will start a new session, you will lose the session with the current device. |
| | Zoom | Zoom in or out. This will make the device's screen appear larger or smaller. This only affects the view in your browser, it does not change anything on the device itself. |
| | Rotate | Rotates the device. Either from PORTRAIT to LANDSCAPE or vice-versa. |
| | Report Bug | Captures a bug: takes a screenshot from the device and opens an editor where you can annotate the image. Use **Report a bug** to create an issue in your bug tracker (Jira, GitHub or Bugsnag) or share the screenshot via e-mail or Slack, or use **Download** to save it locally. |
| | Copy/Paste | Copy or paste from or to the device. If you've selected the copy/paste permission in your browser, TestingBot will automatically keep your local and remote clipboard in sync. |
| | DevTools | This will open the developer tools for the device. Retrieve realtime logs from the device. Use the **Inspector** functionality to drill down in the view hierarchy to copy element locators (such as XPath). Learn more: [App Inspection](https://testingbot.com/support/app-live/app-inspection), [Appium Inspector](https://testingbot.com/support/app-live/appium-inspector), [Device Logs](https://testingbot.com/support/app-live/device-logs) |
| | GPS Location | Set a (new) mock location for the GPS of the device. If your app uses GPS functionality, it will receive this mock location |
| | Locale | Set a new locale for the entire device. Useful if your app is multilingual. Test your app in different languages. |
| | Device Settings | Change various device settings. Switch between light and dark mode instantly. |
| | Stop Session | Stops the current session. Logs and a video of the test will be available in the TestingBot member area. |
## Developer Tools
The DevTools button provides access to powerful debugging features. Learn more about each tool:
- [App Inspection](https://testingbot.com/support/app-live/app-inspection) — Find element locators by clicking on elements or searching the tree
- [Appium Inspector](https://testingbot.com/support/app-live/appium-inspector) — Full Appium Inspector built into TestingBot
- [Device Logs](https://testingbot.com/support/app-live/device-logs) — Real-time log streaming with filtering and download
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/app-live/developer-tools
# Developer Tools
TestingBot provides powerful developer tools built into the live testing interface. Access them by clicking the **DevTools** icon in the toolbar during any live session.
## [App Inspection](https://testingbot.com/support/app-live/app-inspection)
Click on any element on the device screen to see its locators, or search the element tree by text or attribute.
## [Appium Inspector](https://testingbot.com/support/app-live/appium-inspector)
The full Appium Inspector built directly into TestingBot. Inspect elements, view properties, and perform actions without any setup.
## [Device Logs](https://testingbot.com/support/app-live/device-logs)
Stream real-time device logs, filter by severity level, search through entries, and download logs for offline analysis.
## Accessing Developer Tools
During any live app testing session, click the **DevTools** icon in the toolbar on the left side of the screen. This will open a panel where you can switch between the available tools.
## Developer Resources
- [API Reference](https://testingbot.com/support/api)
- [Integrations](https://testingbot.com/support/integrations)
- [Devices](https://testingbot.com/support/app-automate/devices)
- [TestingBot Tunnel](https://testingbot.com/support/tunnel)
- [Device Features](https://testingbot.com/real-device-testing)
- [Private Devices](https://testingbot.com/enterprise/private-device-cloud)
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/app-live/app-inspection
# App Inspection
App Inspection allows you to find element locators directly during a live testing session. This is useful when building automated tests — you can identify the exact locators (XPath, Accessibility ID, etc.) for any element on the screen.
## Accessing App Inspection
To open App Inspection during a live session:
- Click the **DevTools** icon in the toolbar
- Select the **Inspector** tab
## Click to Inspect
Click on any element on the live device screen to immediately see its locators. The inspection panel will display all available locator strategies for the selected element.
## Search the Element Tree
Use the search bar at the top of the inspection panel to find elements by text content or attribute value. This is helpful when you need to locate elements that are off-screen or deeply nested in the view hierarchy.
## Available Locators
When you select an element, the following locator strategies are displayed:
| Locator | Description |
| --- | --- |
| XPath | Full XPath expression to uniquely identify the element in the view hierarchy. |
| Accessibility ID | The accessibility identifier set by the app developer. Preferred locator for stable tests. |
| Class Name | The UI element class (e.g., `android.widget.Button`, `XCUIElementTypeButton`). |
| ID | The resource ID of the element (Android) or the element name (iOS). |
## Copy Locators
Click the copy icon next to any locator to copy it to your clipboard. You can then paste it directly into your automated test code.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/app-live/appium-inspector
# Appium Inspector
TestingBot includes the full Appium Inspector built directly into the live testing interface. This provides the same functionality as the standalone Appium Inspector desktop application, without requiring any local setup.
## Accessing Appium Inspector
To launch the Appium Inspector during a live session:
- Click the **DevTools** icon in the toolbar
- Select the **Appium Inspector** tab
## Interface Overview
The Appium Inspector interface consists of:
- **Device Screenshot** — A snapshot of the current device screen with clickable elements
- **Element Tree** — A hierarchical view of all UI elements on the screen
- **Element Properties** — Detailed attributes and locators for the selected element
- **Actions Panel** — Buttons to perform actions on selected elements
## Inspecting Elements
Click on any element in the device screenshot or in the element tree to select it. The properties panel will show all available attributes and locators for the element.
You can copy any locator value with a single click for use in your automated test scripts.
## Element Actions
The Appium Inspector allows you to perform actions directly on selected elements:
- **Tap** — Tap on the selected element
- **Send Keys** — Type text into input fields
- **Swipe** — Perform swipe gestures on the element
These actions are executed on the real device in real-time, so you can see the result immediately on the live device screen.
## Advantages
- **No setup required** — No need to install the standalone Appium Inspector or configure connection settings
- **Works with any device** — Available for all real devices and virtual devices in the TestingBot device cloud
- **Session-aware** — Automatically connected to your current live testing session
- **Real-time** — The element tree updates as you interact with the device
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/app-live/device-logs
# Device Logs
TestingBot provides a real-time stream of the device logs during the manual live session.
## Log Levels
You can switch between these device log levels:
- `VERBOSE` (default)
- `DEBUG`
- `INFO`
- `WARNING`
- `ERROR`
## Search & Filter
You can use the search input bar to search between the generated logs. The search filters the log entries in real-time as you type, making it easy to find specific messages or errors.
## Log Actions
| Icon | Functionality |
| --- | --- |
| | The download icon allows you to download the entire device logs generated thus far, without any filtering. |
| | The trash icon will clear the history of the logs. Only new log entries will be added to the realtime view. |
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/app-live/local-testing
# Testing Local Websites/Apps
With our [TestingBot Tunnel](https://testingbot.com/support/tunnel) you can easily and securely connect your local computer/network with TestingBot's Real Devices.
## Known Limitations
- **Localhost testing does not work on iOS**.
Due to an iOS Restriction, it is currently not possible to run tests against `http(s)://localhost` with TestingBot Tunnel.
A possible workaround would be to change the hosts file on the machine running TestingBot Tunnel, and add a custom hostname (for example `mylocalwebsite` and the IP address `127.0.0.1`).
Then configure your tests to use `http(s)://mylocalwebsite/...` instead of localhost.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/app-live/apple-pay
# Testing Apple Pay
Apple Pay, developed by Apple Inc., serves as a mobile payment and digital wallet service enabling transactions through various Apple devices, such as iPhones and iPads. Testing with Apple Pay comes with challenges. This document aims to clarify some methods that are available for testing with iOS and Apple Pay.
There are important differences between Apple Pay on Real Devices and iOS Simulators. The iOS Simulator has the following limitations:
- The simulator only targets the front-end integration of Apple Pay and does not test the back-end integration.
- You can not add payment cards to the wallet on iOS simulators. So no Apple Pay sandbox testing cards, or any other real credit card.
- It is not possible to test the Apple Pay in-web flow.
- It is possible to test the Apple Pay in-app flow, but the Apple Pay in-app flow will not work the same way as with a real iOS device. It will not return a payment token and will not properly process a payment.
## Apple Pay on Real Private Devices
TestingBot provides [private iOS devices](https://testingbot.com/enterprise/private-device-cloud), which are able to use Apple Pay. To make Apple Pay work on TestingBot's real private devices, please follow the steps below:
1. Follow Apple's documentation to [enable Apple Pay](https://developer.apple.com/documentation/passkit/apple_pay/setting_up_apple_pay)
2. Set up Apple Pay integration in your iOS app, using the Apple documentation.
3. Register your Merchant ID in your Apple developer account.
4. Set up an Apple sandbox tester account. See the [Apple Sandbox documentation](https://developer.apple.com/help/app-store-connect/test-in-app-purchases/create-sandbox-apple-ids/) for more information.
5. Build your ipa file. Apple Pay will not work with enterprise certificates, which means it will not work by default on TestingBot's private devices. You will need to add the `udid` (iPhone/iPad unique id) of each private device assigned to you, to your Apple provisioning profile.
- Log in to [Apple's Developer Account](https://developer.apple.com/account/resources/identifiers/list)
- Click [Add Device](https://developer.apple.com/account/resources/devices/add)
- Type in a device name and the UDID that you received from TestingBot.
- Save the device.
- Now add the new device to your [Provisioning Profile](https://developer.apple.com/account/resources/profiles/list)
### Passcode
One of the requirements to use Apple Pay, is that a passcode needs to be set on the device. Make sure that you set a passcode on your private iOS device.
TestingBot cleans every private device after each test session. You will need to configure your Apple Pay Sandbox Testing account, including a passcode and sandbox cards, every time you want to test Apple Pay on a real iOS device.
If you want to disable this automated cleaning process, please [reach out](https://testingbot.com/contact/new).
## Apple Pay on iOS Simulators
iOS Simulators offer a feature where you can authorize Apple Pay by clicking a button in the Simulator's menu bar.
- Navigate to a point in your app where a payment sheet is visible.
- Click **Features** , then enable **Authorize Apple Pay** , or use the key combination (Command+SHIFT+A).
## Apple Sandbox Test Cards
You can find a list of Test Cards and their details on [Apple Pay's Sandbox Testing documentation page](https://developer.apple.com/apple-pay/sandbox-testing/).
These test cards include Discover, Master Card, American Express, VISA and others. There are cards available for the USA, Europe, China and even Test Cards for Point of Sale Systems.
To use these, simply open the **Wallet** application on your iOS device and add the card number and expiration date.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/app-live/chromeos
# ChromeOS Manual App Testing
With TestingBot, you can upload your native Android app and test it on a remote ChromeOS device emulator, such as a Chromebook.
After you upload your app, simply choose one of the three ChromeOS configurations to start testing:
- ChromeOS Small: 1366x768 - 160dpi
- ChromeOS Medium: 3840x2160 - 320dpi
- ChromeOS Large: 1920x1080 - 160dpi
## Starting a live ChromeOS session
To get started, make sure you uploaded a `.apk` file on the [manual app session](https://testingbot.com/members/apps/new) page. Then, click one of the three ChromeOS configurations to start a new manual session.
A remote session will be started and you will be able to interact with the ChromeOS device emulator with your own mouse and keyboard.
An audio stream will be started, so you can hear the sounds from the ChromeOS device emulator.
## Enabling/disabling Dark Mode
You can easily enable or disable Dark Mode in the ChromeOS emulator by clicking the "Device Settings" button in taskbar, then toggling the "Enable Dark Mode" switch.
## Taking screenshots
To take a live screenshot of the current ChromeOS session, simply click the "Photo" button in the taskbar. This will capture the current view and open the screenshot in a photo editor, which you can use to annotate or modify the screenshot, before downloading or sharing it.
## Change GPS
You can use the "GPS" button in the taskbar to change the GPS location of the ChromeOS device emulator. This is useful for testing location-based features on your website.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/api
# TestingBot API Documentation
Access and modify all your TestingBot data through our comprehensive REST API. Build powerful integrations and automate your testing workflows.
- **Endpoint:** api.testingbot.com
- **Version:** v1
- **Format:** JSON
- **Auth:** HTTP Basic
[Download OpenAPI 2.0 spec (JSON)](https://testingbot.com/api/v1/openapi.json)
## API Reference
The reference is split by resource. Every endpoint below is authenticated with the API key and secret described in [Authentication](https://testingbot.com#authentication).
- [Browsers & Devices 6 endpoints Discover which browsers, operating systems and real devices you can run on, and look up the IP ranges to allowlist.](https://testingbot.com/support/api/devices)
- [User & Team Management 16 endpoints Your own account and credentials, plus everything needed to provision team members and service accounts from a script.](https://testingbot.com/support/api/user)
- [Tests & Builds 19 endpoints Fetch test results, report pass/fail state back from your CI, and work with the builds that group them.](https://testingbot.com/support/api/tests)
- [Insights 5 endpoints The data behind the Test Analytics dashboard, exposed so you can build your own reporting.](https://testingbot.com/support/api/insights)
- [Screenshots 3 endpoints Kick off a screenshot run across browsers and fetch the images when it finishes.](https://testingbot.com/support/api/screenshots)
- [Tunnel 7 endpoints Manage the secure tunnels that let the grid reach your staging and internal environments.](https://testingbot.com/support/api/tunnel)
- [Codeless Automation & TestLab 29 endpoints Drive the codeless test builder and the TestLab suites that group those tests, end to end.](https://testingbot.com/support/api/codeless)
- [TestingBot Storage 5 endpoints Upload app binaries and test bundles once, then reference them from your capabilities with a tb:// URL.](https://testingbot.com/support/api/storage)
- [Webhooks 7 endpoints Manage the callbacks TestingBot posts when a test finishes, and verify them without waiting for a run.](https://testingbot.com/support/api/webhooks)
- [App Automate 32 endpoints Everything for native mobile test runs: Maestro flows, Espresso and XCUITest suites, and their reports.](https://testingbot.com/support/api/app-automate)
## API Clients
There are several open source client libraries available to easily interact with the TestingBot API:
- [NodeJS](https://github.com/testingbot/testingbot-api)
- [Python](https://github.com/testingbot/testingbotclient)
- [Java](https://github.com/testingbot/testingbot-java)
- [Ruby](https://github.com/testingbot/testingbot_ruby)
- [PHP](https://github.com/testingbot/testingbot-php)
- [.NET](https://github.com/testingbot/testingbot-dotnet)
## Authentication
Every TestingBot API request is authenticated with your **API key** and **API secret** using [HTTP Basic Auth](http://en.wikipedia.org/wiki/Basic_access_authentication) over HTTPS. The only unauthenticated endpoints are [GET /v1/browsers](https://testingbot.com#browsers), [GET /v1/devices](https://testingbot.com#devices) and [GET /v1/configuration/ip-ranges](https://testingbot.com/support/api/devices#ipranges).
### 1 · Find your credentials
Sign in to TestingBot and open [Account → Account Info](https://testingbot.com/members/user/edit). You'll find:
- **`key` string:** Your public API client key. Safe to share between your CI and the TestingBot grid; not a secret on its own.
- **`secret` string sensitive:** Treat this like a password. Never commit it to git, never paste it into screenshots, never embed it in client-side code. Use environment variables or a secret manager in CI.
### 2 · Send credentials on every request
Pass the credentials as the standard `Authorization` header. Most HTTP clients accept a `user:password` tuple and handle the Base64 encoding for you — examples on the right.
### 3 · Verify it works
Hit [GET /v1/user](https://testingbot.com#user). A JSON response with your name and plan means auth is set up correctly. A `401 Unauthorized` means the key or secret is wrong.
**Best practice.** Store credentials in environment variables — `TESTINGBOT_KEY` and `TESTINGBOT_SECRET` — and inject them into your CI/CD pipeline as secrets. Rotate the secret immediately if it leaks (Account → Reset API credentials).
`Authentication Examples`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Authenticated request
```bash
$ curl https://api.testingbot.com/v1/user \
-u "$TESTINGBOT_KEY:$TESTINGBOT_SECRET"
```
```csharp
using TestingBot.Api;
// reads TESTINGBOT_KEY / TESTINGBOT_SECRET from the environment
using var client = TestingBotClient.FromEnvironment();
var user = await client.User.GetAsync();
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(
ENV['TESTINGBOT_KEY'],
ENV['TESTINGBOT_SECRET']
)
api.get_user_info
```
```python
import os, testingbotclient
tb = testingbotclient.TestingBotClient(
os.environ['TESTINGBOT_KEY'],
os.environ['TESTINGBOT_SECRET']
)
tb.user.get_user_information()
```
```php
$api = new TestingBot\TestingBotAPI(
getenv('TESTINGBOT_KEY'),
getenv('TESTINGBOT_SECRET')
);
$api->getUserInfo();
```
```java
TestingbotREST restApi = new TestingbotREST(
System.getenv("TESTINGBOT_KEY"),
System.getenv("TESTINGBOT_SECRET")
);
TestingbotUser user = restApi.getUserInfo();
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: process.env.TESTINGBOT_KEY,
api_secret: process.env.TESTINGBOT_SECRET
});
const userInfo = await api.getUserInfo();
```
## Reducing response size
Any endpoint accepts an optional `omit` query parameter to drop fields you don't need from the JSON response. Pass a comma-separated list of field names. This is handy for trimming large payloads, for example skipping the embedded `report` and `report_xml` on App Automate run details.
- **`omit` string:** Comma-separated field names to exclude. Matching is by field name and applies anywhere it appears in the response, including inside `data` list items and nested objects. Unknown names are ignored.
**Note.** `omit` never affects whether a request succeeds, only the shape of the body. Omitting nothing returns the full, default response.
`omit example`
Skip heavy fields on a run
```bash
$ curl "https://api.testingbot.com/v1/app-automate/maestro/12/45?omit=report,report_xml" \
-u "$TESTINGBOT_KEY:$TESTINGBOT_SECRET"
```
Response
```json
{
"id": 45,
"status": "DONE",
"success": true,
"created_at": "2026-06-01T10:00:00.000Z"
}
```
## Rate limits
Requests are counted in a rolling five-minute window. Authenticated requests count against your **API key** , so one busy CI job cannot spend a colleague's budget, and a key that leaks is capped rather than free.
- **`2000 requests / 5 minutes` per API key:** Roughly 6 requests per second sustained. Applies to every authenticated endpoint.
- **`300 requests / 5 minutes` per IP:** For requests sent without credentials, which can only reach the public endpoints such as [GET /v1/browsers](https://testingbot.com/support/api/devices#browsers).
- **`10000 requests / 5 minutes` per IP, all calls:** An overall ceiling on everything sent from one IP address, whichever API keys it uses. Set well above a single key's allowance, so several keys behind one CI egress address fit comfortably.
Every [tunnel endpoint](https://testingbot.com/support/api/tunnel) is exempt. Tunnels are infrastructure rather than a data surface: the client polls while a tunnel boots, and the tunnel VM itself reports readiness, so a limit there would stall provisioning rather than stop abuse.
[Sending a test webhook delivery](https://testingbot.com/support/api/webhooks#testwebhook) has its own, much tighter allowance of **20 per 5 minutes per API key** , on top of the limit above. Each call makes TestingBot perform an outbound request to a URL you choose, and verifying a webhook is a setup-time action rather than something to do in bulk.
Every API response carries your current usage, so you can slow down before you are cut off rather than after:
- **`X-RateLimit-Limit` integer:** Requests allowed in the current window.
- **`X-RateLimit-Remaining` integer:** Requests left in the current window.
- **`X-RateLimit-Reset` integer:** UNIX timestamp at which the window resets and the allowance returns to full.
**Going over.** Requests past the limit return `429 Too Many Requests` with a `Retry-After` header giving the seconds to wait. The body is JSON in the usual shape, so an SDK surfaces it as an API error rather than a parse failure. Retrying after `Retry-After` is always safe.
`rate limit headers`
On every response
```
X-RateLimit-Limit: 2000
X-RateLimit-Remaining: 1994
X-RateLimit-Reset: 1785736800
```
Over the limit
```json
{
"error": "429 Too Many Requests. Retry in 300 seconds."
}
```
---
URL: https://testingbot.com/support/api/devices
# Browsers & Devices
Discover which browsers, operating systems and real devices you can run on, and look up the IP ranges to allowlist.
- **Endpoint:** api.testingbot.com
- **Version:** v1
- **Format:** JSON
- **Auth:** [HTTP Basic](https://testingbot.com/support/api#authentication)
GET `/v1/browsers`
## List supported browsers
Returns every browser environment available on the TestingBot grid: name, version, platform, and the `browser_id` used when attaching browsers to Codeless tests. Mobile simulators, emulators and Apple TV also carry `deviceName`, `platformName` and `device_type`. Every filter is optional and they combine with AND; with none, the full catalog is returned. An invalid filter value is a 400 rather than an unfiltered list. No authentication required.
### Arguments
- **`type` string:** Filter by automation protocol: "webdriver" (default) or "rc" (legacy Selenium RC).
- **`platformName` string:** Only devices of this OS family: "iOS", "Android" or "tvOS". Case-insensitive.
- **`platform` string:** Only environments on this platform, as returned in `platform` (e.g. "WIN11", "SEQUOIA", "ANDROID"). Case-insensitive.
- **`name` string:** Only this browser, as returned in `name` (e.g. "firefox", "safari"). "chrome" matches both desktop ("googlechrome") and Android ("chrome") rows. Case-insensitive.
- **`device_type` string:** Only devices of this form factor: "phone", "tablet", "desktop" (ChromeOS) or "tv" (tvOS).
- **`min_version` string:** Only versions at or above this one, compared numerically (e.g. "18.0" or "18"). Environments with a non-numeric version ("beta", "dev") are excluded.
- **`max_version` string:** Only versions at or below this one, compared numerically. Environments with a non-numeric version are excluded.
### Response fields
- **`selenium_name` string:** Capability value to send as `browserName` in WebDriver.
- **`name` string:** Human-readable browser identifier (e.g. "firefox", "iexplore").
- **`version` string:** Version as a string, not a number: it may carry minor/patch components (e.g. "11.4"). For mobile environments this is the OS version. Only present when the environment has a version set.
- **`long_version` string:** Full version string (e.g. "121.0.6167.184"), or null when unknown. Only present alongside `version`.
- **`platform` string:** Operating system (e.g. "WIN11", "SEQUOIA", "LINUX", "ANDROID").
- **`browser_id` integer:** Unique TestingBot browser ID used to attach browsers to Codeless tests.
- **`deviceName` string:** Device name to send as `deviceName` (e.g. "iPhone 16", "Pixel 9", "Apple TV 4K"). Only present on mobile and tvOS environments.
- **`platformName` string:** OS family to send as `platformName`: "iOS", "Android" or "tvOS". Only present on mobile and tvOS environments.
- **`device_type` string:** Form factor: "phone", "tablet", "desktop" (ChromeOS) or "tv" (tvOS). Only present on mobile and tvOS environments with a device name.
GET `/v1/browsers`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
# Everything
curl "https://api.testingbot.com/v1/browsers"
# iOS tablet simulators, iOS 18 and up
curl "https://api.testingbot.com/v1/browsers?platformName=iOS&device_type=tablet&min_version=18"
```
```csharp
var client = new TestingBotClient(key, secret);
var browsers = await client.Browsers.ListAsync();
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.get_browsers
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.information.get_browsers()
```
```php
$api = new TestingBot\TestingBotAPI($key, $secret);
$api->getBrowsers();
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
ArrayList browsers = restApi.getBrowsers();
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
const browsers = await api.getBrowsers();
```
Response
```json
[
{"selenium_name": "IE8", "name": "iexplore", "version": "8", "platform": "WINDOWS", "browser_id": 1, "long_version": "8"},
{"selenium_name": "FF7", "name": "firefox", "version": "7", "platform": "WINDOWS", "browser_id": 22, "long_version": "7"},
{"selenium_name": "safari", "name": "safari", "version": "18.6", "platform": "SEQUOIA", "browser_id": 7712, "long_version": null, "deviceName": "iPad Air 11-inch (M3)", "platformName": "iOS", "device_type": "tablet"}
]
```
GET `/v1/browsers/{framework}`
## List browsers and devices supported by a test framework
The subset of GET /v1/browsers that a framework can run on: desktop browsers and versions for Playwright, Puppeteer and Cypress, and the devices for Appium (iOS, Android, tvOS), Maestro, Espresso and XCUITest. `selenium` is the whole catalog. Accepts the same filters as GET /v1/browsers, and returns the same fields. For the full list of physical devices with live availability, see GET /v1/devices. No authentication required.
### Arguments
- **`framework` string required:** Test framework: selenium, appium, maestro, espresso, xcuitest, playwright, puppeteer, cypress. Case-insensitive.
- **`platformName` string:** Only devices of this OS family: "iOS", "Android" or "tvOS". Case-insensitive.
- **`platform` string:** Only environments on this platform, as returned in `platform` (e.g. "WIN11", "SEQUOIA", "ANDROID"). Case-insensitive.
- **`name` string:** Only this browser, as returned in `name` (e.g. "firefox", "safari"). "chrome" matches both desktop ("googlechrome") and Android ("chrome") rows. Case-insensitive.
- **`device_type` string:** Only devices of this form factor: "phone", "tablet", "desktop" (ChromeOS) or "tv" (tvOS).
- **`min_version` string:** Only versions at or above this one, compared numerically (e.g. "18.0" or "18"). Environments with a non-numeric version ("beta", "dev") are excluded.
- **`max_version` string:** Only versions at or below this one, compared numerically. Environments with a non-numeric version are excluded.
### Response fields
- **`selenium_name` string:** Capability value to send as `browserName` in WebDriver.
- **`name` string:** Human-readable browser identifier (e.g. "firefox", "iexplore").
- **`version` string:** Version as a string, not a number: it may carry minor/patch components (e.g. "11.4"). For mobile environments this is the OS version. Only present when the environment has a version set.
- **`long_version` string:** Full version string (e.g. "121.0.6167.184"), or null when unknown. Only present alongside `version`.
- **`platform` string:** Operating system (e.g. "WIN11", "SEQUOIA", "LINUX", "ANDROID").
- **`browser_id` integer:** Unique TestingBot browser ID used to attach browsers to Codeless tests.
- **`deviceName` string:** Device name to send as `deviceName` (e.g. "iPhone 16", "Pixel 9", "Apple TV 4K"). Only present on mobile and tvOS environments.
- **`platformName` string:** OS family to send as `platformName`: "iOS", "Android" or "tvOS". Only present on mobile and tvOS environments.
- **`device_type` string:** Form factor: "phone", "tablet", "desktop" (ChromeOS) or "tv" (tvOS). Only present on mobile and tvOS environments with a device name.
### Per framework
- Selenium `/v1/browsers/selenium` [Selenium docs](https://testingbot.com/support/web-automate/browsers)
- **Lists:** Every desktop browser and device on the grid: the same rows as GET /v1/browsers.
- Appium `/v1/browsers/appium` [Appium docs](https://testingbot.com/support/app-automate/appium)
- **Devices:** iOS, Android, and tvOS (Apple TV)
- **Rows:** Per device and browser, for mobile web testing.
- Maestro `/v1/browsers/maestro` [Maestro docs](https://testingbot.com/support/app-automate/maestro#run)
- **Devices:** iOS and Android
- **Rows:** One per device: a native-app run picks a device, not a browser.
- Espresso `/v1/browsers/espresso` [Espresso docs](https://testingbot.com/support/app-automate/espresso)
- **Devices:** Android
- **Rows:** One per device: a native-app run picks a device, not a browser.
- XCUITest `/v1/browsers/xcuitest` [XCUITest docs](https://testingbot.com/support/app-automate/xcuitest)
- **Devices:** iOS, real devices only (no simulators)
- **Rows:** One per device: a native-app run picks a device, not a browser.
- Playwright `/v1/browsers/playwright` [Playwright docs](https://testingbot.com/support/web-automate/playwright/browsers)
- **Browsers:** Chrome 72+, Edge 80+, Firefox 110+, Safari (WebKit)
- **Platforms:** WIN10, WIN11, CATALINA, BIGSUR, VENTURA, SONOMA, SEQUOIA, TAHOE, LINUX
- **Mobile:** Chrome on Android emulators and real devices
- Puppeteer `/v1/browsers/puppeteer` [Puppeteer docs](https://testingbot.com/support/web-automate/puppeteer/browsers)
- **Browsers:** Chrome 72+, Edge 80+, Firefox 110+
- **Platforms:** WIN10, WIN11, BIGSUR, MONTEREY, VENTURA, SONOMA, SEQUOIA, TAHOE
- Cypress `/v1/browsers/cypress` [Cypress docs](https://testingbot.com/support/web-automate/cypress/capabilities)
- **Browsers:** Chrome 70+, Firefox 70+, Edge 80+ (WIN10 only)
- **Platforms:** WIN10, WIN11, MONTEREY, VENTURA, SEQUOIA, TAHOE
GET `/v1/browsers/{framework}`
Selenium Appium Maestro Espresso XCUITest Playwright Puppeteer Cypress
Request
```bash
curl "https://api.testingbot.com/v1/browsers/selenium"
```
Response 5,177 rows today · sample
```json
[
{
"selenium_name": "chrome150",
"name": "googlechrome",
"platform": "WIN11",
"browser_id": 32208,
"version": "156",
"long_version": null
},
{
"selenium_name": "safari",
"name": "safari",
"platform": "GOLDENGATE",
"browser_id": 31625,
"version": "27.1",
"long_version": null,
"deviceName": "iPhone Duo",
"platformName": "iOS",
"device_type": "phone"
}
]
```
Request
```bash
curl "https://api.testingbot.com/v1/browsers/appium"
```
Response 254 rows today · sample
```json
[
{
"selenium_name": "safari",
"name": "safari",
"platform": "GOLDENGATE",
"browser_id": 31625,
"version": "27.1",
"long_version": null,
"deviceName": "iPhone Duo",
"platformName": "iOS",
"device_type": "phone"
},
{
"selenium_name": "chrome",
"name": "chrome",
"platform": "ANDROID",
"browser_id": 32169,
"version": "17.0",
"long_version": null,
"deviceName": "Pixel 10 16KB",
"platformName": "Android",
"device_type": "phone"
},
{
"selenium_name": "tvos",
"name": "tvos",
"platform": "TVOS",
"browser_id": 6655,
"version": "17.4",
"long_version": null,
"deviceName": "Apple TV 4K",
"platformName": "tvOS",
"device_type": "tv"
}
]
```
Request
```bash
curl "https://api.testingbot.com/v1/browsers/maestro"
```
Response 244 rows today · sample
```json
[
{
"selenium_name": "safari",
"name": "safari",
"platform": "GOLDENGATE",
"browser_id": 31625,
"version": "27.1",
"long_version": null,
"deviceName": "iPhone Duo",
"platformName": "iOS",
"device_type": "phone"
},
{
"selenium_name": "chrome",
"name": "chrome",
"platform": "ANDROID",
"browser_id": 32169,
"version": "17.0",
"long_version": null,
"deviceName": "Pixel 10 16KB",
"platformName": "Android",
"device_type": "phone"
},
{
"selenium_name": "safari",
"name": "safari",
"platform": "GOLDENGATE",
"browser_id": 31607,
"version": "27.0",
"long_version": null,
"deviceName": "iPad Pro 13-inch (M5)",
"platformName": "iOS",
"device_type": "tablet"
}
]
```
Request
```bash
curl "https://api.testingbot.com/v1/browsers/espresso"
```
Response 56 rows today · sample
```json
[
{
"selenium_name": "chrome",
"name": "chrome",
"platform": "ANDROID",
"browser_id": 32169,
"version": "17.0",
"long_version": null,
"deviceName": "Pixel 10 16KB",
"platformName": "Android",
"device_type": "phone"
},
{
"selenium_name": "chrome",
"name": "chrome",
"platform": "ANDROID",
"browser_id": 32175,
"version": "17.0",
"long_version": null,
"deviceName": "Pixel Tablet 16KB",
"platformName": "Android",
"device_type": "tablet"
}
]
```
Request
```bash
curl "https://api.testingbot.com/v1/browsers/xcuitest"
```
Response 26 rows today · sample
```json
[
{
"selenium_name": "safari",
"name": "safari",
"platform": "MONTEREY",
"browser_id": 32280,
"version": "27.0",
"long_version": null,
"deviceName": "iPhone 13 Pro",
"platformName": "iOS",
"device_type": "phone"
},
{
"selenium_name": "safari",
"name": "safari",
"platform": "TAHOE",
"browser_id": 9301,
"version": "26.6",
"long_version": null,
"deviceName": "iPad (9th generation)",
"platformName": "iOS",
"device_type": "tablet"
}
]
```
Request
```bash
curl "https://api.testingbot.com/v1/browsers/playwright"
```
Response 1,829 rows today · sample
```json
[
{
"selenium_name": "chrome150",
"name": "googlechrome",
"platform": "WIN11",
"browser_id": 32208,
"version": "156",
"long_version": null
},
{
"selenium_name": "microsoftedge150",
"name": "microsoftedge",
"platform": "WIN11",
"browser_id": 32232,
"version": "156",
"long_version": null
},
{
"selenium_name": "ff152",
"name": "firefox",
"platform": "WIN11",
"browser_id": 32265,
"version": "158",
"long_version": null
},
{
"selenium_name": "Safari26",
"name": "safari",
"platform": "TAHOE",
"browser_id": 7613,
"version": "26",
"long_version": null
},
{
"selenium_name": "chrome",
"name": "chrome",
"platform": "ANDROID",
"browser_id": 32175,
"version": "17.0",
"long_version": null,
"deviceName": "Pixel Tablet 16KB",
"platformName": "Android",
"device_type": "tablet"
}
]
```
Request
```bash
curl "https://api.testingbot.com/v1/browsers/puppeteer"
```
Response 1,669 rows today · sample
```json
[
{
"selenium_name": "chrome150",
"name": "googlechrome",
"platform": "WIN11",
"browser_id": 32208,
"version": "156",
"long_version": null
},
{
"selenium_name": "microsoftedge150",
"name": "microsoftedge",
"platform": "WIN11",
"browser_id": 32232,
"version": "156",
"long_version": null
},
{
"selenium_name": "ff152",
"name": "firefox",
"platform": "WIN11",
"browser_id": 32265,
"version": "158",
"long_version": null
}
]
```
Request
```bash
curl "https://api.testingbot.com/v1/browsers/cypress"
```
Response 1,146 rows today · sample
```json
[
{
"selenium_name": "chrome150",
"name": "googlechrome",
"platform": "WIN11",
"browser_id": 32208,
"version": "156",
"long_version": null
},
{
"selenium_name": "ff152",
"name": "firefox",
"platform": "WIN11",
"browser_id": 32265,
"version": "158",
"long_version": null
},
{
"selenium_name": "microsoftedge150",
"name": "microsoftedge",
"platform": "WIN10",
"browser_id": 32229,
"version": "156",
"long_version": null
}
]
```
GET `/v1/devices`
## List physical mobile devices
Returns every real Android and iOS device in the TestingBot grid, with an `available` flag indicating whether it can be acquired right now. Filter by `platform` for a single OS family.
### Arguments
- **`platform` string:** Filter to a single mobile OS family. Case-insensitive.
- **`web` boolean:** Internal flag used by the dashboard — returns extra UI-only fields.
### Response fields
- **`id` integer:** Unique numeric device ID.
- **`name` string:** Marketing name (e.g. "iPhone 15 Pro").
- **`model_number` string:** Manufacturer model number (e.g. "SM-G991B").
- **`platform_name` string:** Mobile OS family (e.g. "iOS", "Android").
- **`version` string:** OS version (e.g. "17.4", "14").
- **`resolution` string:** Display resolution as `x` (e.g. "1170x2532").
- **`width` integer:** Display width in pixels, parsed from `resolution`.
- **`height` integer:** Display height in pixels, parsed from `resolution`.
- **`cpu` string:** CPU architecture.
- **`available` boolean:** Whether the device is free for this account to acquire right now.
- **`multiple_browsers` array of object:** Selectable browsers on this device. Only present on devices that offer more than one.
GET `/v1/devices`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl -u key:secret "https://api.testingbot.com/v1/devices"
```
```csharp
var client = new TestingBotClient(key, secret);
var devices = await client.Devices.ListAsync();
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.get_devices
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.information.get_devices()
```
```php
$api = new TestingBot\TestingBotAPI($key, $secret);
$api->getDevices();
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
List devices = restApi.getDevices(0, 100);
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
const devices = await api.getDevices();
```
Response
```json
[
{
"id": 1,
"available": true,
"name": "Galaxy S8",
"model_number": "SM-G950U1",
"platform_name": "Android",
"version": "9.0",
"resolution": "1080x2220",
"multiple_browsers": [
{ "id": 1727, "name": "chrome", "version": "103" },
{ "id": 2133, "name": "firefox", "version": "98.2" },
{ "id": 2953, "name": "samsung", "version": "13.2.1.70" },
{ "id": 2954, "name": "opera", "version": "67.1" }
]
},
{
"id": 5,
"available": false,
"name": "iPhone XR",
"platform_name": "iOS",
"version": "16.3",
"resolution": "828x1792"
}
]
```
GET `/v1/devices/available`
## List currently available devices
Subset of /v1/devices filtered to devices the authenticated account can acquire right now (not in use by another customer, not under maintenance).
### Response fields
- **`id` integer:** Unique numeric device ID.
- **`name` string:** Marketing name (e.g. "iPhone 15 Pro").
- **`model_number` string:** Manufacturer model number (e.g. "SM-G991B").
- **`platform_name` string:** Mobile OS family (e.g. "iOS", "Android").
- **`version` string:** OS version (e.g. "17.4", "14").
- **`resolution` string:** Display resolution as `x` (e.g. "1170x2532").
- **`width` integer:** Display width in pixels, parsed from `resolution`.
- **`height` integer:** Display height in pixels, parsed from `resolution`.
- **`cpu` string:** CPU architecture.
- **`available` boolean:** Whether the device is free for this account to acquire right now.
- **`multiple_browsers` array of object:** Selectable browsers on this device. Only present on devices that offer more than one.
GET `/v1/devices/available`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl -u key:secret "https://api.testingbot.com/v1/devices/available"
```
```csharp
var client = new TestingBotClient(key, secret);
var devices = await client.Devices.ListAvailableAsync();
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.get_available_devices
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.information.get_available_devices()
```
```php
$api = new TestingBot\TestingBotAPI($key, $secret);
$api->getAvailableDevices();
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
List devices = restApi.getAvailableDevices();
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
const devices = await api.getAvailableDevices();
```
Response
```json
[
{
"id": 1,
"available": true,
"name": "Galaxy S8",
"platform_name": "Android",
"version": "9.0",
"resolution": "1080x2220"
}
]
```
GET `/v1/devices/:id`
## Get a specific device
Returns the spec and current availability for a single device by numeric ID.
### Arguments
- **`id` integer required:** Numeric device ID.
### Response fields
- **`id` integer:** Unique numeric device ID.
- **`name` string:** Marketing name (e.g. "iPhone 15 Pro").
- **`model_number` string:** Manufacturer model number (e.g. "SM-G991B").
- **`platform_name` string:** Mobile OS family (e.g. "iOS", "Android").
- **`version` string:** OS version (e.g. "17.4", "14").
- **`resolution` string:** Display resolution as `x` (e.g. "1170x2532").
- **`width` integer:** Display width in pixels, parsed from `resolution`.
- **`height` integer:** Display height in pixels, parsed from `resolution`.
- **`cpu` string:** CPU architecture.
- **`available` boolean:** Whether the device is free for this account to acquire right now.
- **`multiple_browsers` array of object:** Selectable browsers on this device. Only present on devices that offer more than one.
GET `/v1/devices/:id`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/devices/:id" \
-u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
var device = await client.Devices.GetAsync(deviceId);
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.get_device(device_id)
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.information.get_device(device_id)
```
```php
$api = new TestingBot\TestingBotAPI($key, $secret);
$api->getDevice($deviceId);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
TestingbotDevice device = restApi.getDevice(deviceId);
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
const device = await api.getDevice(deviceId);
```
Response
```json
{
"id": 12,
"available": true,
"name": "Galaxy S20",
"platform_name": "Android",
"version": "10.0",
"resolution": "1440x3200",
"multiple_browsers": [
{ "id": 2598, "name": "chrome", "version": "101" },
{ "id": 2919, "name": "firefox", "version": "98.2" },
{ "id": 2957, "name": "samsung", "version": "13.2.1.70" }
]
}
```
GET `/v1/configuration/ip-ranges`
## TestingBot IP ranges for firewall whitelisting
Returns the up-to-date list of public IPv4 addresses used by TestingBot test machines. Allow these in your firewall when running tests against staging or production behind corporate VPN/network rules. The response is a flat JSON array of IPv4 strings (no CIDR ranges, no metadata) — re-fetch periodically since the pool changes as we scale capacity. Unauthenticated; no `key`/`secret` required.
GET `/v1/configuration/ip-ranges`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/configuration/ip-ranges"
```
```csharp
var client = new TestingBotClient(key, secret);
var ranges = await client.Configuration.GetIpRangesAsync();
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.get_ip_ranges
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.configuration.get_ip_ranges()
```
```php
$client = new TestingBot\Client($key, $secret);
$ranges = $client->configuration()->ipRanges(); // no authentication required
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
JsonElement ipRanges = restApi.getIpRanges();
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
const ipRanges = await api.getIpRanges();
```
Response
```json
[
"109.68.162.161",
"109.68.162.165",
"78.20.187.211",
"162.55.135.22",
"95.217.83.184",
"95.216.39.110",
"95.216.229.109",
"142.132.137.7",
"195.201.167.43",
"162.55.24.100",
"185.223.133.13"
]
```
[Next User & Team Management](https://testingbot.com/support/api/user)
---
URL: https://testingbot.com/support/api/user
# User & Team Management
Your own account and credentials, plus everything needed to provision team members and service accounts from a script.
- **Endpoint:** api.testingbot.com
- **Version:** v1
- **Format:** JSON
- **Auth:** [HTTP Basic](https://testingbot.com/support/api#authentication)
GET `/v1/user`
## Get your user info
Returns the data associated with the authenticated account: name, plan, billing details, remaining credits, and concurrency caps.
### Response fields
- **`id` integer:** Unique numeric user ID. Also the path param for team endpoints (e.g. /users/:id) and the parent\_id reference on team members.
- **`first_name` string:** Given name on the account.
- **`last_name` string:** Family name on the account.
- **`seconds` integer:** Remaining test seconds (credit balance) on the account; -1 means unlimited (e.g. Live plans).
- **`last_login` timestamp:** ISO-8601 timestamp of the most recent dashboard login.
- **`plan` string:** Subscription plan name (e.g. "Free Trial", "Live - 5 Sessions - Unlimited", "Automated Pro ..."). For a sub-account this is the team owner's plan.
- **`max_concurrent` integer:** Maximum number of parallel VM-based sessions allowed by the plan. For a sub-account this is the team owner's cap.
- **`max_concurrent_mobile` integer:** Maximum number of parallel physical device sessions allowed by the plan. For a sub-account this is the team owner's cap.
- **`company` string:** Optional company name for billing.
- **`street` string:** Billing address line.
- **`city` string:** Billing city.
- **`country` string:** Billing country.
- **`vat` string:** VAT number (EU only).
- **`read_only` boolean:** Whether this account has read-only access within the team.
- **`roles` array of string:** Roles held by the account: "admin" and/or "owner".
- **`current_vm_concurrency` integer:** VM-based sessions this account is running right now.
- **`current_physical_concurrency` integer:** Physical-device sessions this account is running right now.
GET `/v1/user`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/user" \
-u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
var user = await client.User.GetAsync();
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.get_user_info
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.user.get_user_information()
```
```php
$api = new TestingBot\TestingBotAPI($key, $secret);
$api->getUserInfo();
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
TestingbotUser user = restApi.getUserInfo();
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
const userInfo = await api.getUserInfo();
```
Response
```json
{
"first_name": "Steven",
"last_name": "King",
"seconds": 17745,
"plan": "small",
"max_concurrent": 10,
"max_concurrent_mobile": 2,
"company": "companyName",
"street": "companyStreet",
"city": "companyCity",
"country": "companyCountry",
"vat": "ifInEurope"
}
```
PUT `/v1/user`
## Update your user info
Updates the authenticated account. Only `first_name` and `last_name` are mutable through this endpoint; other profile fields require the dashboard.
### Arguments
- **`user[first_name]` string:** New given name.
- **`user[last_name]` string:** New family name.
### Response fields
- **`success` boolean:** Whether the update was applied.
- **`user` object:** Updated user object. Only present when `success` is true.
- **`errors` object:** Validation errors keyed by field name. Only present when `success` is false.
PUT `/v1/user`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/user" \
-X PUT \
-d "user[first_name]=new" \
-u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
await client.User.UpdateAsync(new UserUpdate { FirstName = "Bruno", LastName = "Mars" });
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.update_user_info({ "first_name" => 'new' })
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.user.update_user_information({'user[first_name]': 'new'})
```
```php
$api = new TestingBot\TestingBotAPI($key, $secret);
$api->updateUserInfo(array('first_name' => 'new'));
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
TestingbotUser user = restApi.updateUserInfo(TestingBotUser);
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
const newUserData = { first_name: 'new' };
await api.updateUserInfo(newUserData);
```
Response
```json
{
"success": true,
"user": {
"first_name": "new",
"last_name": "King",
"seconds": 17745,
"last_login": "2011-08-06T16:47:04Z"
}
}
```
GET `/v1/user/keys`
## Get your API key and secret
Returns the API client key and secret for the authenticated account. Useful for confirming the credentials currently in use, but never embed this response in client-side code.
### Response fields
- **`key` string:** API client key.
- **`secret` string:** API client secret. Treat as a password: never embed it in client-side code.
GET `/v1/user/keys`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/user/keys" \
-u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
var keys = await client.User.GetKeysAsync();
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.get_user_keys
```
```php
$client = new TestingBot\Client($key, $secret);
$keys = $client->user()->keys();
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
JsonElement keys = restApi.getUserKeys();
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
const { key, secret } = await api.getUserKeys();
```
GET `/v1/team-management`
## Get team concurrency info
Returns allowed vs current concurrent session counts for the team across both VMs and physical mobile devices, plus how many sessions are waiting for a free slot. Poll this before fanning out a parallel build. Counts are team-wide and come from the Hub, so they match what the dashboard shows.
### Response fields
- **`concurrency` object:** Allowed vs current concurrency caps for the team, split in VM and physical device sessions.
GET `/v1/team-management`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/team-management" \
-u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
var concurrency = await client.Team.GetConcurrencyAsync();
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.get_team
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.team.get_concurrency()
```
```php
$client = new TestingBot\Client($key, $secret);
$team = $client->teamManagement()->get();
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
TestingbotTeam team = restApi.getTeam();
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
const teamInfo = await api.getTeam();
```
Response
```json
{
"concurrency": {
"allowed": { "vms": 10, "physical": 2 },
"current": { "vms": 10, "physical": 1 },
"queued": { "vms": 3, "physical": 0 }
}
}
```
GET `/v1/team-management/users`
## List users in your team
Paginated list of all sub-accounts in the team. Requires admin role on the calling account.
### Arguments
- **`offset` integer:** Skip this many users from the start of the result set.
- **`count` integer max=`500`:** Number of users to return .
### Response fields
- **`data` array of team member objects:** Team member accounts.
- **`meta` meta object:** —
GET `/v1/team-management/users`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/team-management/users" \
-u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
var members = await client.Team.ListUsersAsync();
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.get_users_in_team(0, 10)
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.team.get_users(offset=0, limit=10)
```
```php
$client = new TestingBot\Client($key, $secret);
$members = $client->teamManagement()->listUsers(0, 10);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
TestingbotTeamMemberCollection members = restApi.getTeamMembers();
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
const users = await api.getUsersInTeam();
```
Response
```json
{
"data": [
{
"id": 337,
"first_name": "test",
"last_name": "user",
"seconds": 12000,
"plan": "Free Trial",
"max_concurrent": 2,
"max_concurrent_mobile": 2
}
],
"meta": { "offset": 0, "count": 1, "total": 1 }
}
```
GET `/v1/team-management/users/:id`
## Get a specific team user
Returns the user record for a specific team member. Admins can fetch any team member; non-admins can only fetch themselves.
### Arguments
- **`id` integer required:** Numeric ID of the team user to fetch.
### Response fields
- **`id` integer:** Unique numeric user ID. Also the path param for team endpoints (e.g. /users/:id) and the parent\_id reference on team members.
- **`first_name` string:** Given name on the account.
- **`last_name` string:** Family name on the account.
- **`seconds` integer:** Remaining test seconds (credit balance) on the account; -1 means unlimited (e.g. Live plans).
- **`last_login` timestamp:** ISO-8601 timestamp of the most recent dashboard login.
- **`plan` string:** Subscription plan name (e.g. "Free Trial", "Live - 5 Sessions - Unlimited", "Automated Pro ..."). For a sub-account this is the team owner's plan.
- **`max_concurrent` integer:** Maximum number of parallel VM-based sessions allowed by the plan. For a sub-account this is the team owner's cap.
- **`max_concurrent_mobile` integer:** Maximum number of parallel physical device sessions allowed by the plan. For a sub-account this is the team owner's cap.
- **`company` string:** Optional company name for billing.
- **`street` string:** Billing address line.
- **`city` string:** Billing city.
- **`country` string:** Billing country.
- **`vat` string:** VAT number (EU only).
- **`read_only` boolean:** Whether this account has read-only access within the team.
- **`roles` array of string:** Roles held by the account: "admin" and/or "owner".
- **`current_vm_concurrency` integer:** VM-based sessions this account is running right now.
- **`current_physical_concurrency` integer:** Physical-device sessions this account is running right now.
GET `/v1/team-management/users/:id`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/team-management/users/:id" \
-u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
var member = await client.Team.GetUserAsync(userId);
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.get_user_in_team(user_id)
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.team.get_user(user_id)
```
```php
$client = new TestingBot\Client($key, $secret);
$member = $client->teamManagement()->getUser($userId);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
TestingbotTeamMember member = restApi.getTeamMember(userId);
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
const user = await api.getUserFromTeam(userId);
```
Response
```json
{
"id": 337,
"first_name": "test",
"last_name": "user",
"seconds": 12000,
"plan": "Free Trial",
"max_concurrent": 2,
"max_concurrent_mobile": 2
}
```
POST `/v1/team-management/users`
## Create a user in your team
Provisions a new sub-account inside the team, copying the caller's subscription level and assigning a slice of the credit pool. Requires an upgraded plan and admin role. If the address already has a TestingBot account outside your team, no account is created: the address is invited to your team instead (202, `{"status": "invited"}`) and joins once its owner accepts.
### Arguments
- **`email` string required:** Email address for the new account; must be unique across TestingBot.
- **`password` string required:** Initial password for the new user.
- **`first_name` string required:** New user's given name.
- **`last_name` string required:** New user's family name.
- **`concurrency` integer:** Max parallel VM sessions (≤ team owner's `maxParallel`).
- **`concurrencyPhysical` integer:** Max parallel physical-device sessions (≤ team owner's `maxParallelDevice`).
POST `/v1/team-management/users`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl -X POST -u key:secret \
https://api.testingbot.com/v1/team-management/users \
--header 'Content-Type: application/json' \
--data-raw '{
"first_name": "Bruno",
"last_name": "Mars",
"email": "bmars@example.com",
"password": "$bmaRs*RULES"
}'
```
```csharp
var client = new TestingBotClient(key, secret);
var member = await client.Team.CreateUserAsync(new TeamMemberCreate
{
Email = "bmars@example.com",
Password = "$bmaRs*RULES",
FirstName = "Bruno",
LastName = "Mars"
});
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.create_user_in_team({ first_name: 'Bruno', last_name: 'Mars', email: 'bmars@example.com', password: '$bmaRs*RULES' })
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.team.create_user(email='bmars@example.com', password='$bmaRs*RULES', first_name='Bruno', last_name='Mars')
```
```php
$client = new TestingBot\Client($key, $secret);
$member = $client->teamManagement()->createUser([
'email' => 'bmars@example.com',
'password' => '$bmaRs*RULES',
'first_name' => 'Bruno',
'last_name' => 'Mars',
]);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
Map params = new HashMap<>();
params.put("first_name", "Bruno");
params.put("last_name", "Mars");
params.put("email", "bmars@example.com");
params.put("password", "$bmaRs*RULES");
TestingbotTeamMember member = restApi.createTeamMember(params);
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
const userData = {
first_name: 'John',
last_name: 'Doe',
email: 'john@example.com',
password: 'StrongPassword!'
};
const result = await api.createUserInTeam(userData);
```
Response
```json
{
"id": 337,
"first_name": "Bruno",
"last_name": "Mars",
"seconds": 12000,
"plan": "Free Trial",
"max_concurrent": 2,
"max_concurrent_mobile": 2
}
```
PUT `/v1/team-management/users/:id`
## Update a user in your team
Updates a team user's profile, credit allocation and concurrency. Credit fields cannot exceed the team owner's remaining balance; `concurrency` and `concurrencyPhysical` must be between 0 and the team owner's `maxParallel` / `maxParallelDevice`. Members can change their own email or password only, and must send `current_password` to do so.
### Arguments
- **`id` integer required:** Numeric ID of the team user to update.
- **`first_name` string:** New given name.
- **`last_name` string:** New family name.
- **`email` string:** New email address.
- **`password` string:** New password.
- **`current_password` string:** Your current password. Required when changing your own email or password.
- **`credits` integer:** Allocated VM credit seconds (≤ team owner's remaining VM credits).
- **`device_credits` integer:** Allocated physical-device credit seconds (≤ team owner's remaining device credits).
- **`concurrency` integer:** Max parallel VM sessions for this user (0..team owner's `maxParallel`).
- **`concurrencyPhysical` integer:** Max parallel physical-device sessions for this user (0..team owner's `maxParallelDevice`).
### Response fields
- **`id` integer:** Unique numeric user ID. Also the path param for team endpoints (e.g. /users/:id) and the parent\_id reference on team members.
- **`first_name` string:** Given name on the account.
- **`last_name` string:** Family name on the account.
- **`seconds` integer:** Remaining test seconds (credit balance) on the account; -1 means unlimited (e.g. Live plans).
- **`last_login` timestamp:** ISO-8601 timestamp of the most recent dashboard login.
- **`plan` string:** Subscription plan name (e.g. "Free Trial", "Live - 5 Sessions - Unlimited", "Automated Pro ..."). For a sub-account this is the team owner's plan.
- **`max_concurrent` integer:** Maximum number of parallel VM-based sessions allowed by the plan. For a sub-account this is the team owner's cap.
- **`max_concurrent_mobile` integer:** Maximum number of parallel physical device sessions allowed by the plan. For a sub-account this is the team owner's cap.
- **`company` string:** Optional company name for billing.
- **`street` string:** Billing address line.
- **`city` string:** Billing city.
- **`country` string:** Billing country.
- **`vat` string:** VAT number (EU only).
- **`read_only` boolean:** Whether this account has read-only access within the team.
- **`roles` array of string:** Roles held by the account: "admin" and/or "owner".
- **`current_vm_concurrency` integer:** VM-based sessions this account is running right now.
- **`current_physical_concurrency` integer:** Physical-device sessions this account is running right now.
PUT `/v1/team-management/users/:id`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl -X PUT -u key:secret \
https://api.testingbot.com/v1/team-management/users/:id \
--header 'Content-Type: application/json' \
--data-raw '{
"first_name": "Bruno",
"last_name": "Mars"
}'
```
```csharp
var client = new TestingBotClient(key, secret);
var member = await client.Team.UpdateUserAsync(userId, new TeamMemberUpdate { FirstName = "Bruno", LastName = "Mars" });
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.update_user_in_team(user_id, { first_name: 'Bruno', last_name: 'Mars' })
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.team.update_user(user_id, first_name='Bruno', last_name='Mars')
```
```php
$client = new TestingBot\Client($key, $secret);
$client->teamManagement()->updateUser($userId, ['first_name' => 'Bruno']);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
Map params = new HashMap<>();
params.put("first_name", "Bruno");
params.put("last_name", "Mars");
TestingbotTeamMember member = restApi.updateTeamMember(userId, params);
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
const result = await api.updateUserInTeam(userId, { first_name: 'Jane', last_name: 'Smith' });
```
Response
```json
{
"id": 337,
"first_name": "Bruno",
"last_name": "Mars",
"seconds": 12000,
"plan": "Free Trial"
}
```
POST `/v1/team-management/users/:id/reset-keys`
## Reset credentials for a team user
Rotates both the API key and secret for a team user. The old credentials stop working immediately, so any deployed automation using them must be updated.
### Arguments
- **`id` integer required:** Numeric ID of the team user whose credentials to rotate.
POST `/v1/team-management/users/:id/reset-keys`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl -X POST -u key:secret \
https://api.testingbot.com/v1/team-management/users/:id/reset-keys
```
```csharp
var client = new TestingBotClient(key, secret);
var reset = await client.Team.ResetUserKeysAsync(userId);
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.reset_credentials(user_id)
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.team.reset_keys(user_id)
```
```php
$client = new TestingBot\Client($key, $secret);
$credentials = $client->teamManagement()->resetUserKeys($userId);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
TestingbotTeamCredentialReset reset = restApi.resetTeamMemberKeys(userId);
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
const result = await api.resetCredentials(userId);
```
Response
```json
{
"success": true,
"client_key": "d69e0800xc32ed2f059b7a630ee255b"
}
```
GET `/v1/team-management/service-accounts`
## List service accounts
Returns the team's API-only service accounts, each with its own key and secret. Service accounts run tests on the grid and REST API but cannot sign in to the dashboard. Admin only.
### Arguments
- **`offset` integer:** Skip this many service accounts from the start of the result set.
- **`count` integer:** Number of service accounts to return.
### Response fields
- **`data` array of service account objects:** Service accounts for this team.
- **`meta` meta object:** —
GET `/v1/team-management/service-accounts`
[cURL](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/team-management/service-accounts" \
-u key:secret
```
Response
```json
{
"data": [
{
"id": 412,
"identifier": "ci-runner",
"key": "a1b2c3d4e5f60718293a4b5c6d7e8f90",
"secret": "f0e9d8c7b6a5948372615f4e3d2c1b0a",
"created_at": "2026-06-05T10:08:35Z"
}
],
"meta": { "offset": 0, "count": 10, "total": 1 }
}
```
GET `/v1/team-management/service-accounts/:id`
## Get a service account
Returns a single service account, including its key and secret. Admin only.
### Arguments
- **`id` integer required:** Numeric ID of the service account.
### Response fields
- **`id` integer:** Unique numeric service-account ID.
- **`identifier` string:** Human-friendly label shown on the dashboard and test detail pages.
- **`key` string:** TestingBot API key (username) for this service account.
- **`secret` string:** TestingBot API secret for this service account. Store it as a CI secret.
- **`created_at` timestamp:** When the service account was created.
GET `/v1/team-management/service-accounts/:id`
[cURL](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/team-management/service-accounts/:id" \
-u key:secret
```
Response
```json
{
"id": 412,
"identifier": "ci-runner",
"key": "a1b2c3d4e5f60718293a4b5c6d7e8f90",
"secret": "f0e9d8c7b6a5948372615f4e3d2c1b0a",
"created_at": "2026-06-05T10:08:35Z"
}
```
POST `/v1/team-management/service-accounts`
## Create a service account
Creates a new API-only service account with its own generated key and secret. The identifier is a label (min 3 characters, unique within the team) shown next to the account's tests on the dashboard. Admin only.
### Arguments
- **`identifier` string required min=`3 characters`:** Human-friendly label for the service account (min 3 characters, unique within the team).
POST `/v1/team-management/service-accounts`
[cURL](https://testingbot.com#)
Request
```bash
$ curl -X POST -u key:secret \
https://api.testingbot.com/v1/team-management/service-accounts \
--header 'Content-Type: application/json' \
--data-raw '{
"identifier": "ci-runner"
}'
```
Response
```json
{
"id": 412,
"identifier": "ci-runner",
"key": "a1b2c3d4e5f60718293a4b5c6d7e8f90",
"secret": "f0e9d8c7b6a5948372615f4e3d2c1b0a",
"created_at": "2026-06-05T10:08:35Z"
}
```
PUT `/v1/team-management/service-accounts/:id`
## Update a service account
Renames a service account by changing its identifier. The key and secret are unchanged. Admin only.
### Arguments
- **`id` integer required:** Numeric ID of the service account to update.
- **`identifier` string required min=`3 characters`:** New identifier (min 3 characters, unique within the team).
### Response fields
- **`id` integer:** Unique numeric service-account ID.
- **`identifier` string:** Human-friendly label shown on the dashboard and test detail pages.
- **`key` string:** TestingBot API key (username) for this service account.
- **`secret` string:** TestingBot API secret for this service account. Store it as a CI secret.
- **`created_at` timestamp:** When the service account was created.
PUT `/v1/team-management/service-accounts/:id`
[cURL](https://testingbot.com#)
Request
```bash
$ curl -X PUT -u key:secret \
https://api.testingbot.com/v1/team-management/service-accounts/:id \
--header 'Content-Type: application/json' \
--data-raw '{
"identifier": "nightly-regression"
}'
```
Response
```json
{
"id": 412,
"identifier": "nightly-regression",
"key": "a1b2c3d4e5f60718293a4b5c6d7e8f90",
"secret": "f0e9d8c7b6a5948372615f4e3d2c1b0a",
"created_at": "2026-06-05T10:08:35Z"
}
```
POST `/v1/team-management/service-accounts/:id/rotate`
## Rotate a service account access key
Generates a fresh key and secret for the service account. The previous key and secret stop working immediately, so update any automation using them. Admin only.
### Arguments
- **`id` integer required:** Numeric ID of the service account whose key and secret to rotate.
POST `/v1/team-management/service-accounts/:id/rotate`
[cURL](https://testingbot.com#)
Request
```bash
$ curl -X POST -u key:secret \
https://api.testingbot.com/v1/team-management/service-accounts/:id/rotate
```
Response
```json
{
"id": 412,
"identifier": "ci-runner",
"key": "9f8e7d6c5b4a3i2h1g0f9e8d7c6b5a40",
"secret": "40a5b6c7d8e9f0g1h2i3a4b5c6d7e8f9",
"created_at": "2026-06-05T10:08:35Z"
}
```
DELETE `/v1/team-management/service-accounts/:id`
## Delete a service account
Permanently deletes the service account. Its key and secret stop working immediately. Tests it already ran stay in your history, still labelled with its identifier. Admin only.
### Arguments
- **`id` integer required:** Numeric ID of the service account to delete.
DELETE `/v1/team-management/service-accounts/:id`
[cURL](https://testingbot.com#)
Request
```bash
$ curl -X DELETE -u key:secret \
https://api.testingbot.com/v1/team-management/service-accounts/:id
```
Response
```json
{
"success": true,
"message": "Service account \"ci-runner\" deleted."
}
```
GET `/v1/team-management/users/{id}/client-key`
## Get a team user's client key
Admin-only endpoint that returns the API client key for a specific team user. Used by tooling that impersonates team members in CI; the secret is not exposed here — use reset-keys to rotate.
### Arguments
- **`id` integer required:** Numeric ID of the team user whose key to fetch.
### Response fields
- **`client_key` string:** The API client key. The matching secret is never returned here; rotate both with reset-keys.
GET `/v1/team-management/users/{id}/client-key`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/team-management/users/{id}/client-key" \
-u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
var clientKey = await client.Team.GetUserClientKeyAsync(userId);
```
```php
$client = new TestingBot\Client($key, $secret);
$clientKey = $client->teamManagement()->getUserClientKey($userId);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
JsonElement clientKey = restApi.getTeamMemberClientKey(userId);
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
const { client_key } = await api.getUserClientKey(userId);
```
[Previous Browsers & Devices](https://testingbot.com/support/api/devices) [Next Tests & Builds](https://testingbot.com/support/api/tests)
---
URL: https://testingbot.com/support/api/tests
# Tests & Builds
Fetch test results, report pass/fail state back from your CI, and work with the builds that group them.
- **Endpoint:** api.testingbot.com
- **Version:** v1
- **Format:** JSON
- **Auth:** [HTTP Basic](https://testingbot.com/support/api#authentication)
**Changed August 2026:** the `browser` field is now a stable identifier without the version (for example `googlechrome` rather than `Chrome 150`), so it is safe to match on. The human-readable label moved to the new `browser_display_name` field, and `browser_long_version` carries the full version string when known. Integrations that matched on the old combined label should match on `browser` plus `browser_version` instead.
GET `/v1/tests`
## List your tests
Paginated list of every test session belonging to the authenticated account, newest first. Filter by `browser_id`, `group`, or `build` to narrow the result. Use `since` to fetch only tests updated after a UNIX timestamp (poll-friendly).
### Arguments
- **`offset` integer:** Skip this many tests from the start of the result set.
- **`count` integer max=`500`:** Number of tests to return .
- **`since` integer:** UNIX timestamp; return only tests updated at or after this time.
- **`browser_id` integer:** Filter to tests that ran on this browser\_id (from /v1/browsers).
- **`group` string:** Filter to tests tagged with this group name.
- **`build` string:** Filter to tests in this build (matches `capabilities.build`).
- **`skip_fields` string:** Comma-separated fields to omit (logs, thumbs).
### Response fields
- **`data` array of test case objects:** Test sessions for this page.
- **`meta` meta object:** —
- **`realtime` object:** ActionCable channel details for live updates. Only present on the `since`-polling form of this endpoint.
GET `/v1/tests`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/tests?offset=0&count=10" \
-u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
var tests = await client.Tests.ListAsync();
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.get_tests(0, 10)
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.tests.get_tests(offset=0, limit=10)
```
```php
$api = new TestingBot\TestingBotAPI($key, $secret);
$api->getJobs(0, 10);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
TestingbotTestCollection tests = restApi.getTests(0, 10);
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
const tests = await api.getTests({ offset, limit });
```
Response
```json
{
"data": [
{
"created_at": "2011-07-30T23:21:23Z",
"completed_at": "2011-07-30T23:22:44Z",
"id": 3,
"name": "MyTest::testTitle",
"session_id": "f7903f9e93e74fe1b0e924bf9d2ce9fc",
"status_message": "Failed asserting that 1 equals 0.",
"status_id": 0,
"success": false,
"browser": "iexplore",
"browser_version": "8",
"browser_long_version": "8.0.7601",
"browser_display_name": "IE8",
"os": "WINDOWS",
"duration": 13,
"build": "buildid",
"video": "https://s3.amazonaws.com/rectestingbot/sample.mp4",
"groups": ["testingbot.com"]
}
],
"meta": { "offset": 0, "count": 10, "total": 789 }
}
```
GET `/v1/tests/:id`
## Get a specific test
Returns the full detail record for a single test session: status, environment (browser/OS or device/platform), assets (video, logs, screenshots), groups, build, and duration. Accepts either the numeric test ID or the WebDriver session\_id.
### Arguments
- **`id` string required:** Numeric test ID or WebDriver session\_id (UUID).
- **`skip_fields` string:** Comma-separated fields to omit from the response (logs, thumbs, visual\_run, groups).
### Response fields
- **`id` integer:** Unique numeric test ID.
- **`session_id` string:** Selenium / WebDriver session ID (UUID).
- **`name` string:** Human-readable test name set via capabilities or `update_test`.
- **`state` string:** Lifecycle state (RUNNING, COMPLETE, TIMEOUT, …).
- **`success` boolean:** Whether the test passed.
- **`status_id` integer:** Numeric status code (0=fail, 1=pass, 2=unknown).
- **`unknown` boolean:** Convenience flag, true when `status_id` is 2 (the test finished without reporting a pass/fail).
- **`status_message` string:** Failure reason or arbitrary status set via `test[status_message]`.
- **`created_at` timestamp:** When the test session started.
- **`completed_at` timestamp:** When the session ended; null while running.
- **`duration` integer:** Total run time in seconds.
- **`browser` string:** Browser identifier, without the version (e.g. "googlechrome", "iexplore", "firefox", "safari"). Stable across versions, so it is the field to match on.
- **`browser_version` string:** Browser version as a string, not a number: it may carry minor/patch components (e.g. "150", "11.4"), and is "0" for environments with no meaningful version.
- **`browser_long_version` string:** Full version string when known (e.g. "121.0.6167.184"); null otherwise.
- **`browser_display_name` string:** Human-readable label for UIs (e.g. "Chrome 150", "IE11", "Safari 11"), with the device and OS appended on mobile. Presentation only: the format is not stable, so build UI strings from it but never match on it.
- **`os` string:** OS (e.g. "WINDOWS", "MAC").
- **`device_name` string:** Mobile device name when the session ran on a physical device; null otherwise.
- **`platform_name` string:** Mobile OS name; null on desktop.
- **`build` string:** Build identifier (free-form string set via capabilities).
- **`groups` array of string:** Tag/group names attached to the test.
- **`video` string:** Signed S3 URL to the recorded video, or false if video was disabled.
- **`thumbs` array of string:** Signed S3 URLs to screenshot thumbnails.
- **`logs` object:** Map of log names → signed S3 URLs (selenium, browser, …).
- **`assets_available` boolean:** Whether assets (video, logs, screenshots) have finished processing.
- **`extra` string:** Arbitrary metadata string set via `test[extra]`.
- **`type` string:** Driver type (WEBDRIVER, APPIUM).
- **`steps` string:** Rendered HTML of the recorded step list. Omit with `skip_fields=steps`.
- **`running` integer:** Completion percentage. Only present while a Codeless test is still running.
GET `/v1/tests/:id`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/tests/:id" -u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
var test = await client.Tests.GetAsync(testId);
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.get_test(test_id)
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.tests.get_test(test_id)
```
```php
$api = new TestingBot\TestingBotAPI($key, $secret);
$api->getJob($test_id);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
TestingbotTest test = restApi.getTest(test_id);
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
const test = await api.getTestDetails(testId);
```
Response
```json
{
"id": 3,
"name": "MyTest::testTitle",
"session_id": "f7903f9e93e74fe1b0e924bf9d2ce9fc",
"state": "COMPLETE",
"success": false,
"status_id": 0,
"unknown": false,
"status_message": "Failed asserting that 1 equals 0.",
"created_at": "2011-07-30T23:21:23Z",
"completed_at": "2011-07-30T23:22:23Z",
"duration": 60,
"browser": "iexplore",
"browser_version": "8",
"browser_long_version": "8.0.7601",
"browser_display_name": "IE8",
"os": "WINDOWS",
"device_name": null,
"platform_name": null,
"build": null,
"groups": ["testingbot.com"],
"video": "https://s3.amazonaws.com/rectestingbot/sample.mp4",
"thumbs": ["https://s3.amazonaws.com/euthumbtestingbot/3_f93782fji.jpg"],
"logs": { "selenium": "https://s3-eu-west-1.amazonaws.com/eulogtestingbot/session.txt" },
"assets_available": true,
"type": "WEBDRIVER"
}
```
PUT `/v1/tests/:id`
## Update a test
Updates a test's metadata after it's been recorded. Use this to mark a test as passed/failed from the test runner, attach groups/tags, set a build identifier, or add a status message. Accepts either a numeric test ID or a WebDriver session\_id.
### Arguments
- **`id` string required:** Numeric test ID or WebDriver session\_id (UUID).
- **`test[name]` string:** Human-readable test name.
- **`test[success]` boolean:** true=pass, false=fail.
- **`test[status_message]` string:** Failure reason or arbitrary status text.
- **`test[extra]` string:** Arbitrary metadata.
- **`test[build]` string:** Build identifier to associate this test with.
- **`test[public]` boolean:** When true, makes the test publicly viewable via share URL.
- **`groups` string:** Comma-separated string or array of tag/group names to attach to the test.
- **`build` string:** Build identifier (alternative location to `test[build]`).
### Response fields
- **`success` boolean:** Whether the operation succeeded.
- **`errors` object:** Validation errors keyed by field name. Only present when `success` is false.
- **`error` string:** Single human-readable reason, used by the older endpoints in place of `errors`. Only present when `success` is false.
PUT `/v1/tests/:id`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/tests/:id" \
-X PUT \
-d "test[success]=1" \
-d "groups[]=regression" \
-u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
await client.Tests.UpdateAsync(testId, new TestUpdate { Success = true, Groups = new[] { "regression" } });
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.update_test(test_id, { name: 'new_name', success: true })
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.tests.update_test(test_id, status_message='..', passed=1, build='..', name='..')
```
```php
$api = new TestingBot\TestingBotAPI($key, $secret);
$api->updateJob($test_id, ['name' => 'mytest', 'success' => true]);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
boolean success = restApi.updateTest(testId, details);
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
await api.updateTest({ 'test[success]': '1', 'test[status_message]': 'failure reason' }, testId);
```
Response
```json
{
"success": true
}
```
DELETE `/v1/tests/:id`
## Delete a test
Permanently deletes a test session and every associated asset (video, logs, screenshots). This action cannot be undone.
### Arguments
- **`id` string required:** Numeric test ID or WebDriver session\_id of the test to delete.
DELETE `/v1/tests/:id`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/tests/:id" \
-X DELETE \
-u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
await client.Tests.DeleteAsync(testId);
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.delete_test(test_id)
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.tests.delete_test(test_id)
```
```php
$api = new TestingBot\TestingBotAPI($key, $secret);
$api->deleteJob($test_id);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
boolean success = restApi.deleteTest(testId);
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
await api.deleteTest(testId);
```
Response
```json
{
"success": true
}
```
GET `/v1/tests/:id/assets`
## List a test's assets
Returns presigned download URLs for everything the session produced: video, per-type log files, HAR network capture, performance metrics and screenshots. URLs expire (video/logs/HAR after 1 hour, screenshots after 7 days) -- fetch this again for fresh ones. Ideal for archiving artifacts from CI after a run.
### Arguments
- **`id` string required:** Numeric test ID or WebDriver session\_id.
### Response fields
- **`session_id` string:** WebDriver session id of the test the assets belong to.
- **`video` string:** Presigned video URL (valid 1 hour). Null when no video was recorded or assets were deleted.
- **`logs` object:** Presigned log-file URLs keyed by log type (selenium, appium, logcat, browser, ...), each valid 1 hour. Empty object when none remain.
- **`har` string:** Presigned HAR network-capture URL (valid 1 hour). Null unless network capture was enabled for the session.
- **`performance` string:** Presigned performance-metrics log URL (valid 1 hour). Null unless performance data was collected.
- **`screenshots` array of string:** Presigned screenshot URLs in capture order, each valid 7 days.
GET `/v1/tests/:id/assets`
[cURL](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/tests/:id/assets" -u key:secret
```
Response
```json
{
"session_id": "2b5a38f3c71e97a2265b52b9382db1c5",
"video": "https://eurectestingbot.../screencast.mp4?X-Amz-Expires=3600...",
"logs": {
"selenium": "https://eulogtestingbot.../selenium.log?X-Amz-Expires=3600..."
},
"har": null,
"performance": null,
"screenshots": [
"https://euthumbtestingbot.../2b5a38f3_1.jpg?X-Amz-Expires=604800..."
]
}
```
DELETE `/v1/tests/:id/assets`
## Delete a test's assets
Permanently deletes the session's stored files -- video, log files, HAR capture and screenshots -- ahead of the normal retention window, e.g. for compliance. The test result itself (status, metadata, command history) is kept; use DELETE /v1/tests/:id to remove the whole test. Idempotent: deleting a test with no remaining assets succeeds.
### Arguments
- **`id` string required:** Numeric test ID or WebDriver session\_id.
DELETE `/v1/tests/:id/assets`
[cURL](https://testingbot.com#)
Request
```bash
$ curl -X DELETE "https://api.testingbot.com/v1/tests/:id/assets" -u key:secret
```
Response
```json
{ "success": true }
```
PUT `/v1/tests/:id/stop`
## Stop a running test
Terminates an in-flight test session. The test is marked complete; any assets gathered (video, logs, screenshots) become available for download. Returns 404 if the test has already finished.
### Arguments
- **`id` string required:** Numeric test ID or WebDriver session\_id of the test to stop.
### Response fields
- **`success` boolean:** Whether the operation succeeded.
- **`errors` object:** Validation errors keyed by field name. Only present when `success` is false.
- **`error` string:** Single human-readable reason, used by the older endpoints in place of `errors`. Only present when `success` is false.
PUT `/v1/tests/:id/stop`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/tests/:id/stop" \
-X PUT \
-u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
await client.Tests.StopAsync(testId);
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.stop_test(test_id)
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.tests.stop_test(test_id)
```
```php
$api = new TestingBot\TestingBotAPI($key, $secret);
$api->stopJob($test_id);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
boolean success = restApi.stopTest(testId);
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
await api.stopTest(testId);
```
Response
```json
{
"success": true
}
```
DELETE `/v1/tests`
## Bulk delete tests
Deletes up to 25 tests -- and every associated asset (video, logs, screenshots) -- in one request. Each entry may be a numeric test ID or a WebDriver session\_id. Tests that cannot be found (or belong to another account) are reported in `missing`; the rest are gone for good, exactly like DELETE /v1/tests/:id.
### Arguments
- **`ids` array required:** Up to 25 numeric test IDs or WebDriver session\_ids.
DELETE `/v1/tests`
[cURL](https://testingbot.com#)
Request
```bash
$ curl -X DELETE "https://api.testingbot.com/v1/tests?ids[]=123&ids[]=2b5a38f3c71e97a2265b52b9382db1c5" -u key:secret
```
Response
```json
{ "success": true, "deleted": ["123", "2b5a38f3c71e97a2265b52b9382db1c5"], "missing": [] }
```
POST `/v1/tests`
## Create a test record (manual sessions)
Creates a TestCase record from outside the WebDriver/Appium flow. Mainly used by clients that want to log a manual or external test result against their TestingBot account. Most users will not need this — the normal flow is to start a Selenium/Appium session, and the record is created automatically.
### Arguments
- **`test[name]` string required:** Test name.
- **`test[test_environment_id]` integer required:** Browser environment the result is attributed to (`browser_id` from GET /v1/browsers).
- **`test[success]` boolean:** true=pass, false=fail.
- **`test[status_message]` string:** Failure reason or status string.
- **`test[extra]` string:** Arbitrary metadata.
- **`test[build]` string:** Build identifier.
POST `/v1/tests`
[cURL](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#)
Request
```bash
$ curl -X POST "https://api.testingbot.com/v1/tests" \
-u key:secret \
-d "test[name]=MyTest::testTitle" \
-d "test[test_environment_id]=1" \
-d "test[success]=true"
```
```php
$client = new TestingBot\Client($key, $secret);
$client->tests()->create([
'name' => 'MyTest::testTitle',
'test_environment_id' => 1,
'success' => true,
]);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
Map test = Map.of(
"name", "MyTest::testTitle",
"test_environment_id", 1,
"success", true);
boolean success = restApi.createTest(test);
```
POST `/v1/tests/{id}`
## Update a test (POST alias)
Alias of `PUT /v1/tests/:id` for clients that cannot send PUT requests (some older HTTP libraries). Same body schema and behavior — see PUT for the canonical form.
### Arguments
- **`id` string required:** Numeric test ID or WebDriver session\_id (UUID).
- **`test[name]` string:** Human-readable test name.
- **`test[success]` boolean:** true=pass, false=fail.
- **`test[status_message]` string:** Failure reason or arbitrary status text.
- **`test[extra]` string:** Arbitrary metadata.
- **`test[build]` string:** Build identifier to associate this test with.
- **`test[public]` boolean:** When true, makes the test publicly viewable via share URL.
- **`groups` string:** Comma-separated string or array of tag/group names to attach to the test.
POST `/v1/tests/{id}`
[cURL](https://testingbot.com#)
Request
```bash
$ curl -X POST "https://api.testingbot.com/v1/tests/{id}" \
-u key:secret
```
GET `/v1/builds`
## List your builds
Returns a paginated list of test builds. A build is an aggregation of test cases that share the same `capabilities.build` identifier, useful for grouping CI runs.
### Arguments
- **`offset` integer:** Skip this many builds from the start of the result set.
- **`count` integer max=`500`:** Number of builds to return .
- **`tag` string:** Only return builds carrying this tag (set tags with PUT /v1/builds/:id).
### Response fields
- **`data` array of build objects:** Builds for this page.
- **`meta` meta object:** —
GET `/v1/builds`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/builds?offset=0&count=10" \
-u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
var builds = await client.Builds.ListAsync();
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.get_builds(0, 10)
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.build.get_builds(offset=0, limit=10)
```
```php
$api = new TestingBot\TestingBotAPI($key, $secret);
$api->getBuilds(0, 10);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
TestingbotBuildCollection builds = restApi.getBuilds(0, 10);
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
const builds = await api.getBuilds(offset, limit);
```
Response
```json
{
"data": [
{
"id": 4331,
"build_identifier": "first-build",
"created_at": "2016-08-01T11:09:02.000Z",
"updated_at": "2016-08-01T11:09:02.000Z"
}
],
"meta": { "offset": 0, "count": 10, "total": 3 }
}
```
GET `/v1/builds/:id`
## Get tests for a build
Returns all test cases that belong to a single build, with pagination. The build can be referenced by either its numeric internal ID or the string identifier you set via `capabilities.build`.
### Arguments
- **`id` string required:** Numeric build ID or string build identifier.
- **`offset` integer:** Skip this many tests in the build.
- **`count` integer max=`500`:** Number of tests to return .
- **`skip_fields` string:** Comma-separated fields to omit from each test (logs, thumbs).
### Response fields
- **`data` array of test case objects:** Test sessions for this page.
- **`meta` meta object:** —
- **`realtime` object:** ActionCable channel details for live updates. Only present on the `since`-polling form of this endpoint.
GET `/v1/builds/:id`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/builds/:id" \
-u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
var tests = await client.Builds.GetTestsAsync(buildId);
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.get_build(build_identifier)
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.build.get_tests_for_build(build_id)
```
```php
$api = new TestingBot\TestingBotAPI($key, $secret);
$api->getBuild($build_id);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
TestingbotTestBuildCollection tests = restApi.getTestsForBuild(buildId);
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
const tests = await api.getTestsForBuild(buildId);
```
Response
```json
{
"data": [
{
"id": 3,
"name": "MyTest::testTitle",
"session_id": "f7903f9e93e74fe1b0e924bf9d2ce9fc",
"state": "COMPLETE",
"success": false,
"status_id": 0,
"created_at": "2011-07-30T23:21:23Z",
"completed_at": "2011-07-30T23:22:44Z",
"duration": 13,
"browser": "iexplore",
"browser_version": "8",
"browser_long_version": "8.0.7601",
"browser_display_name": "IE8",
"os": "WINDOWS",
"build": "buildid",
"video": "https://s3.amazonaws.com/rectestingbot/sample.mp4"
}
],
"meta": { "offset": 0, "count": 10, "total": 1 }
}
```
DELETE `/v1/builds/:id`
## Delete a build
Permanently deletes a build and every test, asset (video, logs, screenshots) attached to it. This action cannot be undone.
### Arguments
- **`id` string required:** Numeric build ID or string build identifier to delete.
DELETE `/v1/builds/:id`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/builds/:id" \
-X DELETE \
-u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
await client.Builds.DeleteAsync(buildId);
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.delete_build(build_identifier)
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.build.delete_build(build_id)
```
```php
$api = new TestingBot\TestingBotAPI($key, $secret);
$api->deleteBuild($build_id);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
boolean success = restApi.deleteBuild(buildId);
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
await api.deleteBuild(buildId);
```
Response
```json
{
"success": true
}
```
PUT `/v1/builds/:id`
## Update a build (rename, tags)
Renames a build and/or replaces its tag set. Tags are short labels (letters, digits, dot, dash, underscore, colon, space; max 50 chars each, 10 per build) you can filter the builds list by (`GET /v1/builds?tag=...`) -- handy for marking CI runs as smoke, nightly, release-x and so on. Passing an empty `tags` array (JSON body) clears them.
### Arguments
- **`id` string required:** Numeric build ID or string build identifier.
- **`name` string:** New build identifier (capabilities.build value).
- **`tags` array:** Replacement tag set; empty array clears all tags.
### Response fields
- **`success` boolean:** Whether the build was updated.
- **`build` object:** The updated build: id, build\_identifier and tags.
PUT `/v1/builds/:id`
[cURL](https://testingbot.com#)
Request
```bash
$ curl -X PUT "https://api.testingbot.com/v1/builds/:id" \
-d "name=nightly-2026-08-11" -d "tags[]=smoke" -d "tags[]=release-1.4" \
-u key:secret
```
Response
```json
{
"success": true,
"build": { "id": 4711, "build_identifier": "nightly-2026-08-11", "tags": ["smoke", "release-1.4"] }
}
```
DELETE `/v1/builds`
## Bulk delete builds
Deletes up to 25 builds in one request. Each entry may be a numeric build ID or a string build identifier. Builds that cannot be found (or belong to another account) are reported in `missing`; the rest are deleted like DELETE /v1/builds/:id (tests keep their results, only the build grouping is removed).
### Arguments
- **`ids` array required:** Up to 25 numeric build IDs or string build identifiers.
DELETE `/v1/builds`
[cURL](https://testingbot.com#)
Request
```bash
$ curl -X DELETE "https://api.testingbot.com/v1/builds?ids[]=4711&ids[]=nightly-2026-08-10" -u key:secret
```
Response
```json
{ "success": true, "deleted": ["4711", "nightly-2026-08-10"], "missing": [] }
```
POST `/v1/builds/{id}/ci`
## Link a build to a commit for CI status checks
Associates a CI build with the commit/PR it ran for on a code host and posts an in-progress status check. Currently supports `github` (requires the TestingBot GitHub App installed on the repository owner). Call this at the start of a workflow run.
### Arguments
- **`id` string required:** Build identifier (capabilities.build) for this run.
- **`repo` string required:** Repository in owner/name form.
- **`commit_sha` string required:** Head commit SHA the build ran for. For pull\_request events use github.event.pull\_request.head.sha (not the merge commit) so the check lands on the SHA branch protection evaluates.
- **`provider` string:** Code host provider. Currently only "github".
- **`branch` string:** Git branch name.
- **`pull_request_number` integer:** Pull request number, if any.
POST `/v1/builds/{id}/ci`
[cURL](https://testingbot.com#)
Request
```bash
$ curl -X POST "https://api.testingbot.com/v1/builds/{id}/ci" \
-u key:secret \
-d "repo=my-org/my-repo" \
-d "commit_sha=9fceb02d0ae598e95dc970b74767f19372d61af8" \
-d "branch=main"
```
PUT `/v1/builds/{id}/ci`
## Report a CI status-check build as finished
Signals that the CI run for this build has finished, so the status check is updated with the aggregate pass/fail result. Call this after your tests complete.
### Arguments
- **`id` string required:** Build identifier (capabilities.build) for this run.
- **`provider` string:** Code host provider. Currently only "github".
### Response fields
- **`success` boolean:** Whether the operation succeeded.
- **`errors` object:** Validation errors keyed by field name. Only present when `success` is false.
- **`error` string:** Single human-readable reason, used by the older endpoints in place of `errors`. Only present when `success` is false.
PUT `/v1/builds/{id}/ci`
[cURL](https://testingbot.com#)
Request
```bash
$ curl -X PUT "https://api.testingbot.com/v1/builds/{id}/ci" \
-u key:secret
```
POST `/v1/visual/tests/{id}/run`
## Run a visual test from CI
Runs a hosted visual test in all of its browsers and returns a check id to poll. Pass `url` to screenshot a preview deployment instead of the test's URL for this run only: it is compared with the test's normal baselines and never becomes one. The preview URL must be on the same domain as the test's URL or match one of the preview hosts set on the test, and must resolve to a public address (use a tunnel for private hosts). Pass `repo` and `commit` to also report the result as a GitHub check on that commit (requires the TestingBot GitHub App on the repository owner). Limited to 60 calls per 5 minutes per API key.
### Arguments
- **`id` integer required:** ID of the visual test.
- **`url` string:** Preview URL to screenshot instead of the test's URL, for this run only.
- **`repo` string:** GitHub repository in owner/name form, to report the result as a check.
- **`commit` string:** Head commit SHA the check is posted on. Required with repo.
- **`pr` integer:** Pull request number, for the summary comment on the pull request.
- **`branch` string:** Git branch name.
POST `/v1/visual/tests/{id}/run`
[cURL](https://testingbot.com#)
Request
```bash
$ curl -X POST "https://api.testingbot.com/v1/visual/tests/{id}/run" \
-u key:secret \
-d "url=https://pr-42.preview.example.com/" \
-d "repo=my-org/my-repo" \
-d "commit=9fceb02d0ae598e95dc970b74767f19372d61af8" \
-d "pr=42"
```
Response
```json
{
"success": true,
"check": "4f1c2b9a0d3e5f67",
"test_id": 1234,
"url": "https://pr-42.preview.example.com/",
"preview": true,
"browsers": 3,
"github": true,
"status_url": "https://api.testingbot.com/v1/visual/checks/4f1c2b9a0d3e5f67"
}
```
GET `/v1/visual/checks/{check}`
## Get a visual check
Returns the state of a check started with POST /v1/visual/tests/{id}/run: per browser whether it has been compared, whether it passed, the mismatch percentage and a link to the run page. `state` turns `done` when every browser has finished; `passed` is true only when every browser compared and passed. Checks can be read for 7 days.
### Arguments
- **`check` string required:** Check id returned when the check was started.
### Response fields
- **`check` string:** Check id.
- **`test_id` integer:** The visual test.
- **`url` string:** The preview URL; null when the check ran the test's own URL, or once the check is older than 7 days.
- **`preview` boolean:** Whether a preview URL replaced the test's URL.
- **`state` string:** `running` until every browser finished, then `done`.
- **`passed` boolean:** true when every browser compared and passed, false otherwise. null while `running`.
- **`test_url` string:** The visual test in the TestingBot dashboard.
- **`browsers` array of visual check browser objects:** One entry per browser of the test.
GET `/v1/visual/checks/{check}`
[cURL](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/visual/checks/{check}" \
-u key:secret
```
Response
```json
{
"check": "4f1c2b9a0d3e5f67",
"test_id": 1234,
"url": "https://pr-42.preview.example.com/",
"preview": true,
"state": "done",
"passed": false,
"test_url": "https://testingbot.com/members/uitest/1234",
"browsers": [
{
"configuration_id": 801,
"browser": "Chrome 149, Windows 11",
"state": "done",
"passed": false,
"mismatch": 2.41,
"threshold": 1.0,
"run_url": "https://testingbot.com/members/visual/5521/runs/99812"
}
]
}
```
[Previous User & Team Management](https://testingbot.com/support/api/user) [Next Insights](https://testingbot.com/support/api/insights)
---
URL: https://testingbot.com/support/api/insights
# Insights
The data behind the Test Analytics dashboard, exposed so you can build your own reporting.
- **Endpoint:** api.testingbot.com
- **Version:** v1
- **Format:** JSON
- **Auth:** [HTTP Basic](https://testingbot.com/support/api#authentication)
## Insights API
The Insights API exposes read-only analytics over your test results so you can build your own dashboards. It returns the same data that powers the [Test Analytics](https://testingbot.com/members/tests) page in your dashboard: daily pass/fail trends, a headline summary, top failure messages, per-test flakiness, and the filter values to drive it all. For a raw list of individual test executions, use [GET /v1/tests](https://testingbot.com/support/api/tests#tests) instead.
Every endpoint accepts a period via `start` and `end` (ISO-8601 or UNIX timestamps), or the `time_range` shorthand (`7d`, `30d`), defaulting to the last 30 days. Filter with `status`, `browser`, `build`, `group_ids` and `team` (comma-separated IDs discoverable via [GET /v1/insights/filters](https://testingbot.com#insights-filters)). Dates in results follow the server timezone (UTC).
GET `/v1/insights/trends`
## Test trends
Time-bucketed success / failure / unknown counts for your tests, the same series that powers the Test Analytics chart in the dashboard. Use `group_by` to aggregate by day (default), by browser (coverage), or as a single status roll-up. Filter by date range, status, browser, build, group and team.
### Arguments
- **`start` string:** Period start. ISO-8601 (2026-06-01 or 2026-06-01T00:00:00Z) or a UNIX timestamp. Defaults to 30 days before `end`.
- **`end` string:** Period end. ISO-8601 or a UNIX timestamp. Defaults to now.
- **`time_range` string:** Relative window as a shorthand for `start`: 24h, 7d, 4w (units h/d/w). Ignored when `start` is given.
- **`status` string:** Filter by test status. One of all (default), success, failure, unknown.
- **`browser` array:** Filter by test environment (browser/OS/device) IDs, comma-separated. Discover IDs via /v1/insights/filters.
- **`build` array:** Filter by build IDs, comma-separated.
- **`group_ids` array:** Filter by group ("My Groups") IDs, comma-separated.
- **`team` array:** Filter by team member user IDs (who ran the test), comma-separated.
- **`name` string:** Filter to an exact test name.
- **`group_by` string:** Aggregation dimension: date (daily buckets, default), browser, or status.
### Response fields
- **`data` array of insight trend bucket objects:** Series buckets, newest first.
- **`meta` insight trend meta object:** —
GET `/v1/insights/trends`
[cURL](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Python](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/insights/trends?group_by=date&time_range=30d" \
-u key:secret
```
```javascript
const auth = Buffer.from(`${key}:${secret}`).toString('base64');
const res = await fetch(
'https://api.testingbot.com/v1/insights/trends?group_by=date&time_range=30d',
{ headers: { Authorization: `Basic ${auth}` } }
);
const trends = await res.json();
```
```python
import requests
res = requests.get(
'https://api.testingbot.com/v1/insights/trends',
params={'group_by': 'date', 'time_range': '30d'},
auth=(key, secret))
trends = res.json()
```
Response
```json
{
"data": [
{ "date": "2026-06-30", "success": 8, "failure": 0, "unknown": 0, "total": 8 },
{ "date": "2026-06-26", "success": 2, "failure": 6, "unknown": 0, "total": 8 }
],
"meta": {
"group_by": "date",
"start": "2026-06-04T00:00:00Z",
"end": "2026-07-04T10:37:00Z",
"status": "all",
"success": 300, "failure": 190, "unknown": 22, "total": 512,
"pass_rate": 0.5859
}
}
```
GET `/v1/insights/summary`
## Test summary
Headline totals (success, failure, unknown) plus pass and failure rates for the selected period and filters. Ideal for a single-number dashboard tile.
### Arguments
- **`start` string:** Period start. ISO-8601 (2026-06-01 or 2026-06-01T00:00:00Z) or a UNIX timestamp. Defaults to 30 days before `end`.
- **`end` string:** Period end. ISO-8601 or a UNIX timestamp. Defaults to now.
- **`time_range` string:** Relative window as a shorthand for `start`: 24h, 7d, 4w (units h/d/w). Ignored when `start` is given.
- **`status` string:** Filter by test status. One of all (default), success, failure, unknown.
- **`browser` array:** Filter by test environment (browser/OS/device) IDs, comma-separated. Discover IDs via /v1/insights/filters.
- **`build` array:** Filter by build IDs, comma-separated.
- **`group_ids` array:** Filter by group ("My Groups") IDs, comma-separated.
- **`team` array:** Filter by team member user IDs (who ran the test), comma-separated.
- **`name` string:** Filter to an exact test name.
### Response fields
- **`start` timestamp:** Period start (ISO-8601, UTC).
- **`end` timestamp:** Period end (ISO-8601, UTC).
- **`status` string:** Status filter applied.
- **`total` integer:** Total tests in the period.
- **`success` integer:** Passed tests.
- **`failure` integer:** Failed tests.
- **`unknown` integer:** Unknown-status tests.
- **`pass_rate` number:** success / total, 0..1.
- **`failure_rate` number:** failure / total, 0..1.
GET `/v1/insights/summary`
[cURL](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Python](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/insights/summary?time_range=30d" \
-u key:secret
```
```javascript
const auth = Buffer.from(`${key}:${secret}`).toString('base64');
const res = await fetch(
'https://api.testingbot.com/v1/insights/summary?time_range=30d',
{ headers: { Authorization: `Basic ${auth}` } }
);
const summary = await res.json();
```
```python
import requests
res = requests.get(
'https://api.testingbot.com/v1/insights/summary',
params={'time_range': '30d'},
auth=(key, secret))
summary = res.json()
```
Response
```json
{
"start": "2026-06-04T00:00:00Z",
"end": "2026-07-04T10:37:00Z",
"status": "all",
"total": 512,
"success": 300,
"failure": 190,
"unknown": 22,
"pass_rate": 0.5859,
"failure_rate": 0.3711
}
```
GET `/v1/insights/errors`
## Top failure messages
Groups failed tests by their status message and returns the most common failures with occurrence counts. Always operates on failed tests, so the `status` filter is ignored. Paginated with offset/count.
### Arguments
- **`start` string:** Period start. ISO-8601 (2026-06-01 or 2026-06-01T00:00:00Z) or a UNIX timestamp. Defaults to 30 days before `end`.
- **`end` string:** Period end. ISO-8601 or a UNIX timestamp. Defaults to now.
- **`time_range` string:** Relative window as a shorthand for `start`: 24h, 7d, 4w (units h/d/w). Ignored when `start` is given.
- **`status` string:** Filter by test status. One of all (default), success, failure, unknown.
- **`browser` array:** Filter by test environment (browser/OS/device) IDs, comma-separated. Discover IDs via /v1/insights/filters.
- **`build` array:** Filter by build IDs, comma-separated.
- **`group_ids` array:** Filter by group ("My Groups") IDs, comma-separated.
- **`team` array:** Filter by team member user IDs (who ran the test), comma-separated.
- **`name` string:** Filter to an exact test name.
- **`offset` integer:** Skip this many error groups.
- **`count` integer max=`500`:** Number of error groups to return .
### Response fields
- **`data` array of insight error objects:** Error groups, most frequent first.
- **`meta` insight error meta object:** —
GET `/v1/insights/errors`
[cURL](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Python](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/insights/errors?time_range=7d&count=10" \
-u key:secret
```
```javascript
const auth = Buffer.from(`${key}:${secret}`).toString('base64');
const res = await fetch(
'https://api.testingbot.com/v1/insights/errors?time_range=7d&count=10',
{ headers: { Authorization: `Basic ${auth}` } }
);
const errors = await res.json();
```
```python
import requests
res = requests.get(
'https://api.testingbot.com/v1/insights/errors',
params={'time_range': '7d', 'count': 10},
auth=(key, secret))
errors = res.json()
```
Response
```json
{
"data": [
{ "error": "Element not found: #login", "count": 42 },
{ "error": "Timeout after 30s waiting for element", "count": 17 }
],
"meta": {
"start": "2026-06-27T10:37:00Z",
"end": "2026-07-04T10:37:00Z",
"total_failures": 190,
"distinct_errors": 12,
"offset": 0,
"count": 10
}
}
```
GET `/v1/insights/test-cases`
## Test cases
Groups your test runs by test name and returns per-test statistics: total runs, success/failure/unknown counts, pass rate, a flaky flag (both passes and failures seen), and average duration in seconds. Sort by total (default), pass\_rate, failure, flaky or avg\_duration. Paginated with offset/count.
### Arguments
- **`start` string:** Period start. ISO-8601 (2026-06-01 or 2026-06-01T00:00:00Z) or a UNIX timestamp. Defaults to 30 days before `end`.
- **`end` string:** Period end. ISO-8601 or a UNIX timestamp. Defaults to now.
- **`time_range` string:** Relative window as a shorthand for `start`: 24h, 7d, 4w (units h/d/w). Ignored when `start` is given.
- **`status` string:** Filter by test status. One of all (default), success, failure, unknown.
- **`browser` array:** Filter by test environment (browser/OS/device) IDs, comma-separated. Discover IDs via /v1/insights/filters.
- **`build` array:** Filter by build IDs, comma-separated.
- **`group_ids` array:** Filter by group ("My Groups") IDs, comma-separated.
- **`team` array:** Filter by team member user IDs (who ran the test), comma-separated.
- **`name` string:** Filter to an exact test name.
- **`sort` string:** Sort field.
- **`sort_order` string:** Sort direction.
- **`offset` integer:** Skip this many test cases.
- **`count` integer max=`500`:** Number of test cases to return .
### Response fields
- **`data` array of insight test case objects:** Per-test-name statistics.
- **`meta` insight test case meta object:** —
GET `/v1/insights/test-cases`
[cURL](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Python](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/insights/test-cases?sort=flaky&count=20" \
-u key:secret
```
```javascript
const auth = Buffer.from(`${key}:${secret}`).toString('base64');
const res = await fetch(
'https://api.testingbot.com/v1/insights/test-cases?sort=flaky&count=20',
{ headers: { Authorization: `Basic ${auth}` } }
);
const testCases = await res.json();
```
```python
import requests
res = requests.get(
'https://api.testingbot.com/v1/insights/test-cases',
params={'sort': 'flaky', 'count': 20},
auth=(key, secret))
test_cases = res.json()
```
Response
```json
{
"data": [
{
"name": "Login::testValidLogin",
"total": 120,
"success": 110,
"failure": 8,
"unknown": 2,
"pass_rate": 0.9167,
"flaky": true,
"avg_duration": 34.2
}
],
"meta": {
"start": "2026-06-04T00:00:00Z",
"end": "2026-07-04T10:37:00Z",
"total": 87,
"offset": 0,
"count": 50,
"sort": "flaky",
"sort_order": "desc"
}
}
```
GET `/v1/insights/filters`
## Available filter values
Returns the browsers, builds, groups and team members available on your account, so a dashboard can populate its filter dropdowns. The returned IDs are the values to pass to the other Insights endpoints (browser, build, group\_ids, team).
### Response fields
- **`browsers` array of insight filter option objects:** Test environments used recently (pass ids as `browser`).
- **`builds` array of object:** Builds ({id, identifier}); pass ids as `build`.
- **`groups` array of insight filter option objects:** User-defined groups (pass ids as `group_ids`).
- **`team_members` array of insight filter option objects:** Account owner + team members (pass ids as `team`).
- **`statuses` array of string:** Valid status filter values.
GET `/v1/insights/filters`
[cURL](https://testingbot.com#) [NodeJS](https://testingbot.com#) [Python](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/insights/filters" \
-u key:secret
```
```javascript
const auth = Buffer.from(`${key}:${secret}`).toString('base64');
const res = await fetch(
'https://api.testingbot.com/v1/insights/filters',
{ headers: { Authorization: `Basic ${auth}` } }
);
const filters = await res.json();
```
```python
import requests
res = requests.get(
'https://api.testingbot.com/v1/insights/filters',
auth=(key, secret))
filters = res.json()
```
Response
```json
{
"browsers": [
{ "id": 1234, "name": "Chrome 126, Windows 11" },
{ "id": 1288, "name": "Safari 17, macOS Sonoma" }
],
"builds": [{ "id": 4331, "identifier": "ci-run-42" }],
"groups": [{ "id": 7, "name": "smoke" }],
"team_members": [{ "id": 9, "name": "Jane Doe" }],
"statuses": ["success", "failure", "unknown"]
}
```
[Previous Tests & Builds](https://testingbot.com/support/api/tests) [Next Screenshots](https://testingbot.com/support/api/screenshots)
---
URL: https://testingbot.com/support/api/screenshots
# Screenshots
Kick off a screenshot run across browsers and fetch the images when it finishes.
- **Endpoint:** api.testingbot.com
- **Version:** v1
- **Format:** JSON
- **Auth:** [HTTP Basic](https://testingbot.com/support/api#authentication)
POST `/v1/screenshots`
## Capture a new screenshot batch
Queues a cross-browser screenshot of a given URL at a specified resolution. Rendering is asynchronous: this returns the batch ID only, so poll GET /v1/screenshots/:id until its `state` is `done` to collect the images.
### Arguments
- **`url` string required:** Page URL to screenshot.
- **`resolution` string required:** Browser viewport (e.g. "1920x1080").
- **`browsers` array required:** Array of `{ browser, version, os }` triples (or `browser_id`s) to render with.
- **`wait_time` integer:** Seconds to wait after page load before snapping (0-60).
- **`fullpage` boolean:** Capture the entire scrollable page instead of just the viewport.
- **`callback_url` string:** POST callback URL invoked when the batch finishes processing.
POST `/v1/screenshots`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl -H 'Content-Type: application/json' -X POST "https://api.testingbot.com/v1/screenshots" \
-u key:secret \
-d '{
"url": "https://www.google.com",
"resolution": "1280x1024",
"browsers": [
{ "browserName": "chrome", "version": 134, "os": "WIN10" },
{ "browserName": "safari", "version": "17.2", "platformName": "iOS", "deviceName": "iPhone 15" }
]
}'
```
```csharp
var client = new TestingBotClient(key, secret);
var batch = await client.Screenshots.CaptureAsync(new ScreenshotRequest
{
Url = "https://www.google.com",
Resolution = "1280x1024",
BrowserIds = new[] { 1, 22 }
});
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.take_screenshots({ url: 'https://www.google.com', resolution: '1280x1024', browsers: [{ browserName: 'chrome', version: 134, os: 'WIN10' }] })
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.screenshots.take_screenshots('https://www.google.com', '1280x1024', [1, 22])
```
```php
$client = new TestingBot\Client($key, $secret);
$batch = $client->screenshots()->create('https://example.com', [1, 2], [
'resolution' => '1920x1080',
]);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
Map params = new HashMap<>();
params.put("url", "https://www.google.com");
params.put("resolution", "1280x1024");
TestingbotScreenshot screenshots = restApi.createScreenshots(params);
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
const screenshots = await api.takeScreenshot(
'https://example.com',
[{ browserName: 'chrome', version: 'latest', os: 'WIN11' }],
'1920x1080'
);
```
Response
```json
{
"id": 3454,
"wait_time": 0,
"resolution": "1280x1024",
"url": "https://www.google.com"
}
```
GET `/v1/screenshots/{id}`
## Get screenshot batch detail
Returns per-browser screenshot results for a single batch, including thumbnail/image URLs and processing state.
### Arguments
- **`id` integer required:** Numeric screenshot batch ID.
- **`excludeIds` string:** Comma-separated screenshot IDs to exclude (useful for delta-fetch).
### Response fields
- **`id` integer:** Unique screenshot batch ID.
- **`url` string:** URL the screenshots were taken from.
- **`resolution` string:** Browser viewport resolution.
- **`state` string:** Batch state: `processing` until every browser has finished, then `done`.
- **`created_at` timestamp:** —
- **`screenshots` array of screenshot result objects:** Per-browser results with download URLs.
GET `/v1/screenshots/{id}`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/screenshots/{id}" -u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
var batch = await client.Screenshots.GetAsync(batchId);
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.get_screenshots(screenshot_id)
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.screenshots.get_screenshot(screenshot_id)
```
```php
$client = new TestingBot\Client($key, $secret);
$batch = $client->screenshots()->get($screenshotId);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
TestingbotScreenshot screenshot = restApi.getScreenshot(screenshotId);
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
const screenshotResult = await api.retrieveScreenshots(screenshotJobId);
```
Response
```json
{
"id": 3454,
"url": "https://www.google.com",
"resolution": "1280x1024",
"created_at": "2017-10-13T14:47:41.000Z",
"state": "done",
"screenshots": [
{
"image_url": "https://....",
"thumb_url": "https://....",
"state": "done",
"id": "9c7deea4-6f23-4041-842b-39cccee447fa",
"session_id": 0,
"created_at": "2017-10-13T14:47:44.000Z",
"os": "SEQUOIA",
"browser_name": "googlechrome",
"browser_version": "138",
"browser_id": 1146,
"device_name": null,
"platform_name": null
}
]
}
```
GET `/v1/screenshots`
## List screenshot batches
Returns batches of cross-browser screenshots the account has queued historically.
### Arguments
- **`offset` integer:** Skip this many batches.
- **`count` integer:** Number of batches to return.
### Response fields
- **`data` array of screenshot summary objects:** Screenshot batches for this page.
- **`meta` meta object:** —
GET `/v1/screenshots`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/screenshots" -u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
var history = await client.Screenshots.ListAsync();
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.get_screenshots_history(0, 10)
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.screenshots.get_screenshots(offset=0, limit=10)
```
```php
$client = new TestingBot\Client($key, $secret);
$batches = $client->screenshots()->list(0, 10);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
TestingbotScreenshotCollection screenshots = restApi.getScreenshots();
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
const screenshots = await api.getScreenshotList();
```
Response
```json
{
"data": [
{ "id": 3454, "url": "https://google.com", "resolution": "1280x1024", "created_at": "2017-10-13T14:47:41.000Z" },
{ "id": 3452, "url": "https://testingbot.com", "resolution": "1280x1024", "created_at": "2017-10-12T09:22:03.000Z" }
],
"meta": { "offset": 0, "count": 3, "total": 3 }
}
```
[Previous Insights](https://testingbot.com/support/api/insights) [Next Tunnel](https://testingbot.com/support/api/tunnel)
---
URL: https://testingbot.com/support/api/tunnel
# Tunnel
Manage the secure tunnels that let the grid reach your staging and internal environments.
- **Endpoint:** api.testingbot.com
- **Version:** v1
- **Format:** JSON
- **Auth:** [HTTP Basic](https://testingbot.com/support/api#authentication)
GET `/v1/tunnel/list`
## List active tunnels
Returns every TestingBot Tunnel currently running under the account. Use this to monitor running tunnels in CI dashboards or to find a tunnel ID before tearing it down.
### Response fields
- **`id` integer:** Unique numeric tunnel ID.
- **`identifier` string:** The `--tunnel-identifier` the tunnel was started with, or null when it was started without one. Use this to tell concurrent tunnels on the same account apart.
- **`state` string:** Tunnel lifecycle state (READY, STOPPED, …).
- **`private_ip` string:** Internal IP of the tunnel VM.
- **`ip` string:** Public IP of the tunnel VM.
- **`requested_at` timestamp:** When the tunnel was requested.
- **`user_id` integer:** ID of the user the tunnel belongs to.
GET `/v1/tunnel/list`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/tunnel/list" \
-u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
var tunnels = await client.Tunnels.ListAsync();
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.get_tunnels
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.tunnel.get_tunnels()
```
```php
$api = new TestingBot\TestingBotAPI($key, $secret);
$api->getTunnels();
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
ArrayList tunnels = restApi.getTunnels();
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
const tunnels = await api.getTunnelList();
```
Response
```json
[
{
"id": 1,
"identifier": "my-ci-tunnel",
"state": "READY",
"ip": "xx",
"private_ip": "xx",
"requested_at": "2026-05-13 01:34:23"
}
]
```
DELETE `/v1/tunnel/{id}`
## Stop a tunnel
Tears down a running tunnel. Use this in CI teardown steps to release the slot for the next run; idle tunnels also self-terminate after the configured timeout.
### Arguments
- **`id` integer required:** Numeric tunnel ID to stop.
DELETE `/v1/tunnel/{id}`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/tunnel/{id}" \
-X DELETE \
-u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
await client.Tunnels.StopAsync(tunnelId);
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.delete_tunnel(tunnel_id)
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.tunnel.delete_tunnel(tunnel_id)
```
```php
$api = new TestingBot\TestingBotAPI($key, $secret);
$api->deleteTunnel($tunnelID);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
boolean success = restApi.deleteTunnel(tunnelId);
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
await api.deleteTunnel(tunnelId);
```
Response
```json
{
"success": true
}
```
GET `/v1/tunnel`
## Get the user's active tunnel
Returns the first active tunnel for the authenticated account (regardless of ID). Convenience endpoint when you only run one tunnel at a time.
### Response fields
- **`id` integer:** Unique numeric tunnel ID.
- **`state` string:** Tunnel lifecycle state (READY, STOPPED, …).
- **`private_ip` string:** Internal IP of the tunnel VM, used to chain the tunnel with BrowserMob.
- **`ip` string:** Public IP of the tunnel VM. Only present once `state` is READY.
- **`version` string:** Latest tunnel client version TestingBot supports.
- **`ssh_fingerprint` string:** OpenSSH SHA-256 fingerprint (`SHA256:...`) of the host key the tunnel VM presents on `ip`. The tunnel client authenticates to that VM with the account key and secret as SSH username and password, so it requires this key before connecting rather than trusting whatever answers. Only present once `state` is READY; absent for tunnels whose key was not recorded, which clients treat as unverified.
GET `/v1/tunnel`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/tunnel" \
-u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
var tunnel = await client.Tunnels.GetActiveAsync();
```
```php
$client = new TestingBot\Client($key, $secret);
$tunnel = $client->tunnels()->getActive();
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
TestingbotTunnel tunnel = restApi.getTunnel();
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
const tunnel = await api.getTunnel();
```
DELETE `/v1/tunnel`
## Stop the user's active tunnel
Tears down whichever tunnel is currently active on the account, without needing to specify an ID. Useful when you only ever run one tunnel and want a simple cleanup call.
DELETE `/v1/tunnel`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl -X DELETE "https://api.testingbot.com/v1/tunnel" \
-u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
await client.Tunnels.StopActiveAsync();
```
```php
$client = new TestingBot\Client($key, $secret);
$client->tunnels()->deleteActive();
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
boolean success = restApi.deleteTunnel();
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
await api.deleteActiveTunnel();
```
POST `/v1/tunnel/create`
## Create a tunnel
Launches a new TestingBot Tunnel VM and returns its connection metadata. Most users start tunnels via the tunnel client/CLI rather than this endpoint directly.
POST `/v1/tunnel/create`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#)
Request
```bash
$ curl -X POST "https://api.testingbot.com/v1/tunnel/create" \
-u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
var tunnel = await client.Tunnels.CreateAsync();
```
```php
$client = new TestingBot\Client($key, $secret);
$tunnel = $client->tunnels()->create();
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
TestingbotTunnel tunnel = restApi.createTunnel();
```
GET `/v1/tunnel/isalive-check`
## Liveness check
Unauthenticated ping endpoint used by the tunnel binary itself to confirm it can reach the TestingBot API. Always returns `{ success: true }`.
### Response fields
- **`success` boolean:** Whether the operation succeeded.
- **`errors` object:** Validation errors keyed by field name. Only present when `success` is false.
- **`error` string:** Single human-readable reason, used by the older endpoints in place of `errors`. Only present when `success` is false.
GET `/v1/tunnel/isalive-check`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Java](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/tunnel/isalive-check" \
-u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
var alive = await client.Tunnels.IsAliveAsync();
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
boolean alive = restApi.isTunnelAlive();
```
GET `/v1/tunnel/{id}`
## Get a specific tunnel
Returns the current state of a tunnel by ID. Returns 500 with the underlying error if the tunnel is in a transition state or no longer exists.
### Arguments
- **`id` integer required:** Numeric tunnel ID returned by /v1/tunnel/list or /v1/tunnel/create.
### Response fields
- **`id` integer:** Unique numeric tunnel ID.
- **`state` string:** Tunnel lifecycle state (READY, STOPPED, …).
- **`private_ip` string:** Internal IP of the tunnel VM, used to chain the tunnel with BrowserMob.
- **`ip` string:** Public IP of the tunnel VM. Only present once `state` is READY.
- **`version` string:** Latest tunnel client version TestingBot supports.
- **`ssh_fingerprint` string:** OpenSSH SHA-256 fingerprint (`SHA256:...`) of the host key the tunnel VM presents on `ip`. The tunnel client authenticates to that VM with the account key and secret as SSH username and password, so it requires this key before connecting rather than trusting whatever answers. Only present once `state` is READY; absent for tunnels whose key was not recorded, which clients treat as unverified.
GET `/v1/tunnel/{id}`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/tunnel/{id}" \
-u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
var tunnel = await client.Tunnels.GetAsync(tunnelId);
```
```php
$client = new TestingBot\Client($key, $secret);
$tunnel = $client->tunnels()->get($tunnelId);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
TestingbotTunnel tunnel = restApi.getTunnel(tunnelId);
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
const tunnel = await api.getTunnelById(tunnelId);
```
[Previous Screenshots](https://testingbot.com/support/api/screenshots) [Next Codeless Automation & TestLab](https://testingbot.com/support/api/codeless)
---
URL: https://testingbot.com/support/api/codeless
# Codeless Automation & TestLab
Drive the codeless test builder and the TestLab suites that group those tests, end to end.
- **Endpoint:** api.testingbot.com
- **Version:** v1
- **Format:** JSON
- **Auth:** [HTTP Basic](https://testingbot.com/support/api#authentication)
GET `/v1/lab`
## List your Codeless tests
Paginated list of every Codeless test (a.k.a. Lab test) on the account. Codeless tests are recorded in TestingBot's in-browser recorder and can be scheduled on a cron, fired via API, or chained into suites.
### Arguments
- **`offset` integer:** Skip this many tests from the start of the result set.
- **`count` integer max=`500`:** Number of tests to return .
### Response fields
- **`data` array of lab test objects:** —
- **`meta` meta object:** —
GET `/v1/lab`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/lab" \
-u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
var tests = await client.CodelessTests.ListAsync();
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.get_lab_tests(0, 10)
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.lab.get_tests(offset=0, limit=10)
```
```php
$client = new TestingBot\Client($key, $secret);
$tests = $client->lab()->listTests(0, 10);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
TestingbotLabTestCollection labTests = restApi.getLabTests();
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
const codelessTests = await api.getCodelessTests();
```
Response
```json
{
"data": [
{
"id": 215,
"enabled": false,
"alerts": [],
"url": "https://testingbot.com/",
"name": "testingbot",
"created_at": "2012-03-12T20:03:45Z",
"updated_at": "2012-03-13T20:26:43Z",
"last_run": "2012-03-13T06:50:48Z",
"browsers": [
{ "name": "firefox", "version": "10", "os": "LINUX" }
]
}
],
"meta": { "offset": 0, "count": 10, "total": 25 }
}
```
GET `/v1/lab/{id}`
## Get a specific Codeless test
Returns a single Codeless test's configuration: schedule, alerts, attached browsers, and timestamps. Use the steps endpoint to retrieve the actual test recording.
### Arguments
- **`id` integer required:** Numeric Codeless test ID.
### Response fields
- **`id` integer:** Unique numeric Codeless test ID.
- **`name` string:** Test name.
- **`url` string:** Target URL the test runs against.
- **`enabled` boolean:** Whether scheduled runs are active.
- **`cron` string:** Cron expression for scheduled runs; null if unscheduled.
- **`created_at` timestamp:** —
- **`updated_at` timestamp:** —
- **`last_run` timestamp:** Most recent run timestamp.
- **`alerts` array of lab alert objects:** Configured alert destinations.
- **`browsers` array of browser objects:** Browsers this test is configured to run on.
GET `/v1/lab/{id}`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/lab/{id}" \
-u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
var test = await client.CodelessTests.GetAsync(testId);
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.get_lab_test(lab_test_id)
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.lab.get_test(lab_test_id)
```
```php
$client = new TestingBot\Client($key, $secret);
$test = $client->lab()->getTest($labTestId);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
TestingbotLabTest labTest = restApi.getLabTest(labTestId);
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
const codelessTest = await api.getCodelessTest(labTestId);
```
Response
```json
{
"id": 215,
"enabled": false,
"alerts": [],
"url": "https://testingbot.com/",
"name": "testingbot",
"created_at": "2012-03-12T20:03:45Z",
"updated_at": "2012-03-13T20:26:43Z",
"last_run": "2012-03-13T06:50:48Z",
"browsers": [
{ "name": "firefox", "version": "10", "os": "LINUX" }
]
}
```
POST `/v1/lab`
## Create a Codeless test
Creates a new Codeless test. Pass `test[name]` and either `test[url]` (target URL the test visits) or `file` (Selenium IDE export to import). Optional `test[ai_prompt]` lets you describe what the AI test agent should verify in plain English.
### Arguments
- **`test[name]` string:** Test name.
- **`test[url]` string:** Target URL to test (required if no `file` is uploaded).
- **`test[cron]` string:** Cron expression for scheduled runs.
- **`test[screenshot]` boolean:** Take screenshots at every step.
- **`test[video]` boolean:** Record a video of the test.
- **`test[idletimeout]` integer:** Seconds of idle time before the runner aborts the test.
- **`test[screenresolution]` string:** Browser viewport (e.g. "1920x1080").
- **`test[ai_prompt]` string:** Plain-English instruction for the AI test agent.
- **`file` file:** Selenium IDE `.side` export to import as the test's steps.
POST `/v1/lab`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl -X POST "https://api.testingbot.com/v1/lab" \
-u key:secret \
-d "test[name]=test" \
-d "test[cron]=31 * * * *"
```
```csharp
var client = new TestingBotClient(key, secret);
long testId = await client.CodelessTests.CreateAsync(new CodelessTestCreate { Name = "My test", Url = "https://example.com" });
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.create_lab_test({ "test[name]" => 'test', "test[cron]" => '31 * * * *' })
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.lab.create_test(name='test', cron='31 * * * *')
```
```php
$client = new TestingBot\Client($key, $secret);
$test = $client->lab()->createTest(['name' => 'smoke', 'url' => 'https://example.com']);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
Map fields = new HashMap<>();
fields.put("name", "test");
fields.put("cron", "31 * * * *");
TestingbotLabCreateAck ack = restApi.createLabTest(fields);
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
const result = await api.createCodelessTest({
name: 'My Codeless Test',
url: 'https://example.com',
ai_prompt: 'Test the login flow',
cron: '0 0 * * *',
screenshot: true,
video: false,
idletimeout: 60,
screenresolution: '1920x1080'
});
```
Response
```json
{
"success": true,
"lab_test_id": 59392
}
```
DELETE `/v1/lab/{id}`
## Delete a Codeless test
Permanently deletes a Codeless test and its steps. Test runs in history remain.
### Arguments
- **`id` integer required:** Numeric Codeless test ID to delete.
DELETE `/v1/lab/{id}`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/lab/{id}" \
-X DELETE \
-u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
await client.CodelessTests.DeleteAsync(testId);
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.delete_lab_test(lab_test_id)
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.lab.delete_test(lab_test_id)
```
```php
$client = new TestingBot\Client($key, $secret);
$client->lab()->deleteTest($labTestId);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
boolean success = restApi.deleteLabTest(labTestId);
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
const result = await api.deleteCodelessTest(testId);
```
Response
```json
{
"success": true
}
```
PUT `/v1/lab/{id}`
## Update a Codeless test
Updates a Codeless test's metadata: name, target URL, cron schedule, enabled state. Accepts either a numeric Codeless test ID or a WebDriver session\_id of a test run that originated from this Codeless test.
### Arguments
- **`id` string required:** Numeric Codeless test ID or WebDriver session\_id.
- **`test[name]` string:** New test name.
- **`test[url]` string:** New target URL.
- **`test[cron]` string:** New cron expression.
- **`test[enabled]` boolean:** Enable or pause scheduled runs.
### Response fields
- **`success` boolean:** Whether the operation succeeded.
- **`errors` object:** Validation errors keyed by field name. Only present when `success` is false.
- **`error` string:** Single human-readable reason, used by the older endpoints in place of `errors`. Only present when `success` is false.
PUT `/v1/lab/{id}`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/lab/{id}" \
-X PUT \
-d "test[cron]=* * * * *" \
-u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
await client.CodelessTests.UpdateAsync(testId, new CodelessTestUpdate { Name = "Renamed test" });
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.update_lab_test(lab_test_id, { "test[cron]" => '* * * * *' })
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.lab.update_test(lab_test_id, cron='* * * * *')
```
```php
$client = new TestingBot\Client($key, $secret);
$client->lab()->updateTest($labTestId, ['name' => 'smoke test']);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
Map fields = new HashMap<>();
fields.put("cron", "* * * * *");
boolean success = restApi.updateLabTest(labTestId, fields);
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
const result = await api.updateCodelessTest({
test: { name: 'Updated Test Name', cron: '0 12 * * *' }
}, testId);
```
Response
```json
{
"success": true
}
```
POST `/v1/lab/{id}/schedule`
## Set or update a Codeless test schedule
Schedules a Codeless test to run on a recurring interval. Choose between once / daily / weekly presets or a raw cron expression.
### Arguments
- **`id` integer required:** Numeric Codeless test ID.
- **`type` string:** Schedule preset; use "custom" with `cronFormat` for fine control.
- **`day` string:** Date (YYYY-MM-DD) for "once" or weekday for "weekly".
- **`hour` string:** Time (HH:MM) for "once", "daily", or "weekly".
- **`cronFormat` string:** Raw cron expression (5-field) used when `type=custom`.
POST `/v1/lab/{id}/schedule`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl -X POST "https://api.testingbot.com/v1/lab/{id}/schedule" \
-u key:secret \
-d "type=daily" -d "hour=00:01"
```
```csharp
var client = new TestingBotClient(key, secret);
await client.CodelessTests.ScheduleAsync(testId, new CodelessSchedule { Type = CodelessScheduleType.Daily, Hour = "09:00" });
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.schedule_lab_test(lab_test_id, { type: 'daily', hour: '00:01' })
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.lab.schedule(lab_test_id, 'daily', hour='00:01')
```
```php
$client = new TestingBot\Client($key, $secret);
$client->lab()->schedule($labTestId, ['cron' => '0 * * * *']);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
Map params = new HashMap<>();
params.put("type", "daily");
params.put("hour", "00:01");
boolean success = restApi.scheduleLabTest(labTestId, params);
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
const result = await api.scheduleCodelessTest(labTestId, { type: 'daily', hour: '00:01' });
```
Response
```json
{
"success": true
}
```
POST `/v1/lab/{id}/alert`
## Add an alert to a Codeless test
Adds a notification channel (email, SMS, or callback URL) that fires when a scheduled run fails. Use PUT to modify an existing alert.
### Arguments
- **`id` integer required:** Numeric Codeless test ID.
- **`kind` string required:** Alert channel.
- **`level` string required:** When to send (every failure vs. daily digest).
- **`content` string required:** Destination — email address, callback URL, or phone number.
POST `/v1/lab/{id}/alert`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl -X POST "https://api.testingbot.com/v1/lab/{id}/alert" \
-u key:secret \
-d "kind=EMAIL" -d "level=IMMEDIATELY" -d "content=alerts@example.com"
```
```csharp
var client = new TestingBotClient(key, secret);
await client.CodelessTests.AddAlertAsync(testId, new CodelessAlertInput
{
Kind = AlertKind.Email,
Level = AlertLevel.Immediately,
Content = "dev@example.com"
});
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.add_lab_test_alert(lab_test_id, { kind: 'EMAIL', level: 'IMMEDIATELY', content: 'alerts@example.com' })
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.lab.add_alert(lab_test_id, 'EMAIL', 'IMMEDIATELY', 'alerts@example.com')
```
```php
$client = new TestingBot\Client($key, $secret);
$client->lab()->addAlert($labTestId, ['content' => 'ops@example.com']);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
Map params = new HashMap<>();
params.put("kind", "EMAIL");
params.put("level", "IMMEDIATELY");
params.put("content", "alerts@example.com");
boolean success = restApi.addLabTestAlert(labTestId, params);
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
const result = await api.createCodelessAlert(labTestId, { kind: 'EMAIL', level: 'IMMEDIATELY', content: 'alerts@example.com' });
```
Response
```json
{
"success": true
}
```
POST `/v1/lab/{id}/report`
## Add a daily report config to a Codeless test
Configures a recurring email report summarising pass/fail rate for this Codeless test. Use PUT to update.
### Arguments
- **`id` integer required:** Numeric Codeless test ID.
- **`email` string required:** Email address that receives the report.
- **`cron` string:** Cron expression for when to send the report (defaults to daily).
POST `/v1/lab/{id}/report`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl -X POST "https://api.testingbot.com/v1/lab/{id}/report" \
-u key:secret \
-d "email=reports@example.com" -d "cron=0 9 * * *"
```
```csharp
var client = new TestingBotClient(key, secret);
await client.CodelessTests.AddReportAsync(testId, new CodelessReportInput { Email = "dev@example.com" });
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.create_lab_test_report(lab_test_id, { email: 'reports@example.com', cron: '0 9 * * *' })
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.lab.add_report(lab_test_id, 'reports@example.com', cron='0 9 * * *')
```
```php
$client = new TestingBot\Client($key, $secret);
$client->lab()->addReport($labTestId, ['email' => 'ops@example.com']);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
Map params = new HashMap<>();
params.put("email", "reports@example.com");
params.put("cron", "0 9 * * *");
boolean success = restApi.createLabTestReport(labTestId, params);
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
const result = await api.createCodelessReport(labTestId, { email: 'reports@example.com', cron: '0 9 * * *' });
```
Response
```json
{
"success": true
}
```
POST `/v1/lab/{id}/steps`
## Replace the Codeless test's steps
Deletes the test's existing steps and replaces them with the provided array. Useful for programmatic editing of recorded tests.
### Arguments
- **`id` integer required:** Numeric Codeless test ID.
- **`steps` array required:** Ordered list of steps; each is `{ order:, cmd:, locator:, value: }`.
POST `/v1/lab/{id}/steps`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl -X POST "https://api.testingbot.com/v1/lab/{id}/steps" \
-u key:secret \
-H 'Content-Type: application/json' \
-d '{ "steps": [{ "order": 0, "cmd": "open", "locator": "/", "value": "" }] }'
```
```csharp
var client = new TestingBotClient(key, secret);
await client.CodelessTests.SetStepsAsync(testId, new[]
{
new CodelessStepInput { Order = 1, Command = "open", Value = "/" }
});
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.set_lab_test_steps(lab_test_id, { steps: [{ order: 0, cmd: 'open', locator: '/', value: '' }] })
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.lab.set_steps(lab_test_id, [{'order': 0, 'cmd': 'open', 'locator': '/', 'value': ''}])
```
```php
$client = new TestingBot\Client($key, $secret);
$client->lab()->setSteps($labTestId, [
['order' => 0, 'cmd' => 'open', 'locator' => '/', 'value' => ''],
]);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
List steps = Arrays.asList("open", "click");
boolean success = restApi.setLabTestSteps(labTestId, steps);
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
const result = await api.addCodelessStep(labTestId, {
steps: [{ order: 0, cmd: 'open', locator: '/', value: '' }]
});
```
Response
```json
{
"success": true
}
```
GET `/v1/lab/{id}/steps`
## List Codeless test steps
Returns the recorded Selenium-IDE steps for a Codeless test, with pagination.
### Arguments
- **`id` integer required:** Numeric Codeless test ID.
- **`offset` integer:** Skip this many steps.
- **`count` integer:** Number of steps to return.
### Response fields
- **`data` array of lab test step objects:** —
- **`meta` meta object:** —
GET `/v1/lab/{id}/steps`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/lab/{id}/steps" \
-u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
var steps = await client.CodelessTests.GetStepsAsync(testId);
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.get_lab_test_steps(lab_test_id)
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.lab.get_steps(lab_test_id)
```
```php
$client = new TestingBot\Client($key, $secret);
$steps = $client->lab()->getSteps($labTestId, 0, 10);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
TestingbotLabTestStepCollection steps = restApi.getLabTestSteps(labTestId);
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
const steps = await api.getCodelessSteps(labTestId);
```
Response
```json
{
"data": [
{ "test_order": 0, "cmd": "open", "locator": "/", "value": "", "created_at": "2019-05-02T14:33:48.000Z", "updated_at": "2019-05-02T14:33:48.000Z" },
{ "test_order": 1, "cmd": "type", "locator": "id=username", "value": "test", "created_at": "2019-05-02T14:33:48.000Z", "updated_at": "2019-05-02T14:33:48.000Z" },
{ "test_order": 2, "cmd": "type", "locator": "id=password", "value": "test", "created_at": "2019-05-02T14:33:48.000Z", "updated_at": "2019-05-02T14:33:48.000Z" }
],
"meta": { "offset": 0, "count": 3, "total": 3 }
}
```
GET `/v1/lab/{id}/browsers`
## Get browsers for a Codeless test
Returns the list of browsers this Codeless test is configured to run on.
### Arguments
- **`id` integer required:** Numeric Codeless test ID.
### Response fields
- **`name` string:** Browser identifier (e.g. "firefox", "chrome").
- **`version` string:** Browser version the test runs against.
- **`os` string:** Operating system the test runs on (e.g. "WINDOWS", "MAC", "LINUX").
GET `/v1/lab/{id}/browsers`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/lab/{id}/browsers" \
-u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
var browsers = await client.CodelessTests.GetBrowsersAsync(testId);
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.get_lab_test_browsers(lab_test_id)
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.lab.get_browsers(lab_test_id)
```
```php
$client = new TestingBot\Client($key, $secret);
$browsers = $client->lab()->getBrowsers($labTestId);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
List browsers = restApi.getLabTestBrowsers(labTestId);
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
const browsers = await api.getCodelessBrowsers(labTestId);
```
Response
```json
[
{ "name": "firefox", "version": "41", "os": "VISTA" }
]
```
POST `/v1/lab/{id}/browsers`
## Update browsers for a Codeless test
Replaces the entire browser set attached to this Codeless test. Pass `browser_ids` as a comma-separated list of IDs from `/v1/browsers`.
### Arguments
- **`id` integer required:** Numeric Codeless test ID.
- **`browser_ids` string required:** Comma-separated list of browser\_ids the test should run on.
POST `/v1/lab/{id}/browsers`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl -X POST "https://api.testingbot.com/v1/lab/{id}/browsers" \
-u key:secret \
-d "browser_ids=1,5,12"
```
```csharp
var client = new TestingBotClient(key, secret);
await client.CodelessTests.SetBrowsersAsync(testId, new[] { 1, 22 });
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.set_lab_test_browsers(lab_test_id, { browser_ids: '1,5,12' })
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.lab.set_browsers(lab_test_id, '1,5,12')
```
```php
$client = new TestingBot\Client($key, $secret);
$client->lab()->setBrowsers($labTestId, [1, 2]);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
boolean success = restApi.setLabTestBrowsers(labTestId, "1,5,12");
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
const result = await api.setCodelessBrowsers(labTestId, { browser_ids: '1,5,12' });
```
Response
```json
{
"success": true
}
```
POST `/v1/lab/{id}/trigger`
## Run a specific Codeless test
Triggers an immediate run of a Codeless test on the browsers configured for it. Returns a `job_id` you can poll with `GET /v1/jobs/:id`.
### Arguments
- **`id` integer required:** Numeric Codeless test ID to run.
- **`url` string:** Override the test's base URL for this run only.
POST `/v1/lab/{id}/trigger`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl -X POST "https://api.testingbot.com/v1/lab/{id}/trigger" \
-u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
var result = await client.CodelessTests.TriggerAsync(testId);
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.trigger_lab_test(lab_test_id)
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.lab.trigger(lab_test_id)
```
```php
$client = new TestingBot\Client($key, $secret);
$run = $client->lab()->trigger($labTestId);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
TestingbotLabRunAck ack = restApi.triggerLabTest(labTestId);
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
const result = await api.triggerCodelessTest(labTestId);
```
Response
```json
{
"success": true,
"job_id": 14
}
```
PUT `/v1/lab/{id}/stop`
## Stop a running Codeless test
Force-stops an in-flight Codeless test run. Optional `browser_id` stops only the run on a specific browser.
### Arguments
- **`id` integer required:** Numeric Codeless test ID.
- **`browser_id` integer:** Only stop the run on this browser\_id (omit to stop all).
### Response fields
- **`success` boolean:** Whether the operation succeeded.
- **`errors` object:** Validation errors keyed by field name. Only present when `success` is false.
- **`error` string:** Single human-readable reason, used by the older endpoints in place of `errors`. Only present when `success` is false.
PUT `/v1/lab/{id}/stop`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl -X PUT "https://api.testingbot.com/v1/lab/{id}/stop" \
-u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
await client.CodelessTests.StopAsync(testId);
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.stop_lab_test(lab_test_id)
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.lab.stop(lab_test_id)
```
```php
$client = new TestingBot\Client($key, $secret);
$client->lab()->stop($labTestId);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
boolean success = restApi.stopLabTest(labTestId);
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
const result = await api.stopCodelessTest(labTestId);
```
Response
```json
{
"success": true
}
```
POST `/v1/lab/trigger_all`
## Run all Codeless tests
Queues a run of every Codeless test on the account. Returns a job\_id that aggregates the results — poll via `GET /v1/jobs/:id`. Optional `url` overrides each test's base URL for this run.
### Arguments
- **`url` string:** Override base URL for every queued test.
POST `/v1/lab/trigger_all`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl -X POST "https://api.testingbot.com/v1/lab/trigger_all" \
-u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
var result = await client.CodelessTests.TriggerAllAsync();
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.trigger_all_lab_tests
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.lab.trigger_all()
```
```php
$client = new TestingBot\Client($key, $secret);
$client->lab()->triggerAll();
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
TestingbotLabRunAck ack = restApi.triggerAllLabTests();
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
const result = await api.triggerAllCodelessTests();
```
Response
```json
{
"success": true,
"job_id": 14
}
```
PUT `/v1/lab/{id}/alert`
## Update an existing alert
Replaces the alert configuration on a Codeless test. POST to /alert creates one; PUT updates it.
### Arguments
- **`id` integer required:** Numeric Codeless test ID.
- **`kind` string:** Alert channel.
- **`level` string:** When to send.
- **`content` string:** Updated destination value.
### Response fields
- **`success` boolean:** Whether the operation succeeded.
- **`errors` object:** Validation errors keyed by field name. Only present when `success` is false.
- **`error` string:** Single human-readable reason, used by the older endpoints in place of `errors`. Only present when `success` is false.
PUT `/v1/lab/{id}/alert`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl -X PUT "https://api.testingbot.com/v1/lab/{id}/alert" \
-u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
await client.CodelessTests.UpdateAlertAsync(testId, new CodelessAlertInput { Content = "ops@example.com" });
```
```php
$client = new TestingBot\Client($key, $secret);
$client->lab()->updateAlert($labTestId, ['content' => 'ops@example.com']);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
Map alert = Map.of("content", "ops@example.com");
boolean success = restApi.updateLabTestAlert(labTestId, alert);
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
await api.updateCodelessAlert(testId, { content: 'ops@example.com' });
```
PUT `/v1/lab/{id}/report`
## Update a daily report config
Replaces the report destination/schedule for a Codeless test.
### Arguments
- **`id` integer required:** Numeric Codeless test ID.
- **`email` string:** Updated email address.
- **`cron` string:** Updated cron expression.
### Response fields
- **`success` boolean:** Whether the operation succeeded.
- **`errors` object:** Validation errors keyed by field name. Only present when `success` is false.
- **`error` string:** Single human-readable reason, used by the older endpoints in place of `errors`. Only present when `success` is false.
PUT `/v1/lab/{id}/report`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl -X PUT "https://api.testingbot.com/v1/lab/{id}/report" \
-u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
await client.CodelessTests.UpdateReportAsync(testId, new CodelessReportInput { Email = "ops@example.com" });
```
```php
$client = new TestingBot\Client($key, $secret);
$client->lab()->updateReport($labTestId, ['email' => 'ops@example.com']);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
Map report = Map.of("email", "ops@example.com");
boolean success = restApi.updateLabTestReport(labTestId, report);
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
await api.updateCodelessReport(testId, { email: 'ops@example.com' });
```
GET `/v1/jobs/{id}`
## Get a job's status
Returns the live status of an asynchronous job — typically used to poll Codeless test/suite runs started by `POST /v1/lab/:id/trigger`, `POST /v1/lab/trigger_all`, or `POST /v1/labsuites/:id/trigger`. Once `status` is FINISHED, the response includes `success` and per-test results.
### Arguments
- **`id` integer required:** Numeric job ID returned by a trigger endpoint.
### Response fields
- **`status` string:** Job state (QUEUED, RUNNING, FINISHED, FAILED).
- **`created_at` timestamp:** —
- **`updated_at` timestamp:** —
- **`success` boolean:** Aggregate pass/fail across all triggered tests; null until FINISHED.
- **`test_ids` array of integer:** Test IDs spawned by this job.
- **`errors` array of object:** Per-test failure detail (step, browser, time).
GET `/v1/jobs/{id}`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/jobs/{id}" \
-u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
var job = await client.Jobs.GetAsync(jobId);
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.get_job(job_id)
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.jobs.get_job(job_id)
```
```php
$client = new TestingBot\Client($key, $secret);
$job = $client->jobs()->get($jobId);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
TestingbotJob job = restApi.getJob(jobId);
```
Response
```json
{
"status": "FINISHED",
"created_at": "2016-04-15T13:47:39.000Z",
"updated_at": "2016-04-15T13:49:30.000Z",
"success": false,
"test_ids": [6620446],
"errors": [
{
"msg": "Actual value 'Google' did not match 'Goooogle'",
"step": "verifyTitle",
"browser": { "name": "firefox", "version": "43", "os": "VISTA" },
"time": "2016-04-15T13:48:25.839Z",
"test": 6620446
}
]
}
```
POST `(your callback URL)`
## Webhook callback
If you have configured a webhook URL, we will POST the result to the callback URL you configured in the Codeless alerts section. The body below is the JSON payload we send; respond with 2xx to acknowledge.
POST `(your callback URL)`
Response
```json
{
"success": false,
"errors": [
{
"msg": "word Not found",
"step": "verifyTextPresent",
"browser": {
"name": "iexplore",
"version": "8",
"os": "WINDOWS"
},
"time": "2012-03-13 20:34:59 UTC",
"test_id": "48586",
"job_id": 17,
"lab_id": 133
}
],
"job_id": 3,
"test_ids": [48586]
}
```
GET `/v1/labsuites`
## List your Codeless suites
Paginated list of every Codeless suite (a.k.a. Lab suite) on the account. A suite groups several Codeless tests so they can be scheduled, triggered, and reported on as a unit.
### Arguments
- **`offset` integer:** Skip this many suites from the start of the result set.
- **`count` integer max=`500`:** Number of suites to return .
### Response fields
- **`data` array of lab suite objects:** —
- **`meta` meta object:** —
GET `/v1/labsuites`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/labsuites" \
-u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
var suites = await client.CodelessSuites.ListAsync();
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.get_lab_suites(0, 10)
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.labsuites.get_suites(offset=0, limit=10)
```
```php
$client = new TestingBot\Client($key, $secret);
$suites = $client->labSuites()->list(0, 10);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
TestingbotLabSuiteCollection suites = restApi.getLabSuites();
```
Response
```json
{
"data": [
{
"id": 1051,
"enabled": true,
"name": "Example",
"created_at": "2017-03-07T19:11:41.000Z",
"updated_at": "2017-03-09T12:17:22.000Z",
"last_run": "2017-03-09T12:17:22.000Z",
"cron": null,
"test_count": 5,
"alerts": [
{ "type": "API", "value": "https://mysite.com/callback/", "level": "IMMEDIATELY" }
],
"browsers": [
{ "name": "firefox", "version": "41", "os": "VISTA" }
]
}
],
"meta": { "offset": 0, "count": 10, "total": 1 }
}
```
GET `/v1/labsuites/{id}`
## Get a specific Codeless suite
Returns a single Codeless suite's configuration: schedule, alerts, attached browsers, and number of tests inside.
### Arguments
- **`id` integer required:** Numeric suite ID.
### Response fields
- **`id` integer:** Unique numeric Codeless suite ID.
- **`name` string:** Suite name.
- **`enabled` boolean:** Whether scheduled runs are active.
- **`cron` string:** Cron expression for scheduled runs.
- **`test_count` integer:** Number of Codeless tests attached.
- **`created_at` timestamp:** —
- **`updated_at` timestamp:** —
- **`last_run` timestamp:** —
- **`alerts` array of lab alert objects:** —
- **`browsers` array of lab browser objects:** Browsers the suite runs against. Only present on GET /v1/labsuites; the single-suite endpoint omits it.
GET `/v1/labsuites/{id}`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/labsuites/{id}" \
-u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
var suite = await client.CodelessSuites.GetAsync(suiteId);
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.get_lab_suite(suite_id)
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.labsuites.get_suite(suite_id)
```
```php
$client = new TestingBot\Client($key, $secret);
$suite = $client->labSuites()->get($suiteId);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
TestingbotLabSuite suite = restApi.getLabSuite(suiteId);
```
Response
```json
{
"id": 41,
"enabled": true,
"name": "Example",
"cron": null,
"test_count": 5,
"created_at": "2017-03-07T19:11:41.000Z",
"updated_at": "2017-03-09T12:17:22.000Z",
"last_run": "2017-03-09T12:17:22.000Z",
"alerts": [
{ "type": "API", "value": "https://mysite.com/callback/", "level": "IMMEDIATELY" }
]
}
```
POST `/v1/labsuites/{id}/trigger`
## Run a Codeless suite
Queues every test in the suite for an immediate run. Returns a job\_id you can poll with `GET /v1/jobs/:id`.
### Arguments
- **`id` integer required:** Numeric suite ID to trigger.
POST `/v1/labsuites/{id}/trigger`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#)
Request
```bash
$ curl -X POST "https://api.testingbot.com/v1/labsuites/{id}/trigger" \
-u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
var result = await client.CodelessSuites.TriggerAsync(suiteId);
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.trigger_lab_suite(suite_id)
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.labsuites.trigger(suite_id)
```
```php
$client = new TestingBot\Client($key, $secret);
$run = $client->labSuites()->trigger($suiteId);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
TestingbotLabRunAck ack = restApi.triggerLabSuite(suiteId);
```
Response
```json
{
"success": true,
"job_id": 14
}
```
POST `/v1/labsuites`
## Create a Codeless suite
Creates a new Codeless suite. Attach Codeless tests with `POST /v1/labsuites/:id/tests` after creation.
### Arguments
- **`suite[name]` string required:** Suite name.
- **`suite[cron]` string:** Cron expression for scheduled suite runs.
- **`suite[screenshot]` boolean:** Take screenshots at every step in every test.
- **`suite[video]` boolean:** Record video for tests in this suite.
- **`suite[idletimeout]` integer:** Idle timeout in seconds before tests are aborted.
- **`suite[screenresolution]` string:** Browser viewport for every test in the suite.
POST `/v1/labsuites`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#)
Request
```bash
$ curl -X POST "https://api.testingbot.com/v1/labsuites" \
-u key:secret \
-d "suite[name]=My Suite"
```
```csharp
var client = new TestingBotClient(key, secret);
long suiteId = await client.CodelessSuites.CreateAsync(new CodelessSuiteCreate { Name = "Regression suite" });
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.create_lab_suite({ "suite[name]" => 'My Suite' })
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.labsuites.create_suite('My Suite')
```
```php
$client = new TestingBot\Client($key, $secret);
$suite = $client->labSuites()->create(['name' => 'regression']);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
Map fields = new HashMap<>();
fields.put("name", "My Suite");
TestingbotLabSuiteCreateAck ack = restApi.createLabSuite(fields);
```
Response
```json
{
"success": true,
"suite_id": 59391
}
```
DELETE `/v1/labsuites/{id}`
## Delete a Codeless suite
Deletes the suite. The Codeless tests attached to it are not deleted — only the suite grouping.
### Arguments
- **`id` integer required:** Numeric suite ID to delete.
DELETE `/v1/labsuites/{id}`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#)
Request
```bash
$ curl -X DELETE "https://api.testingbot.com/v1/labsuites/{id}" \
-u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
await client.CodelessSuites.DeleteAsync(suiteId);
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.delete_lab_suite(suite_id)
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.labsuites.delete_suite(suite_id)
```
```php
$client = new TestingBot\Client($key, $secret);
$client->labSuites()->delete($suiteId);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
boolean success = restApi.deleteLabSuite(suiteId);
```
Response
```json
{
"success": true
}
```
GET `/v1/labsuites/{id}/browsers`
## Get browsers for a Codeless suite
Returns the list of browsers the suite is configured to run on.
### Arguments
- **`id` integer required:** Numeric suite ID.
### Response fields
- **`name` string:** Browser identifier (e.g. "firefox", "chrome").
- **`version` string:** Browser version the test runs against.
- **`os` string:** Operating system the test runs on (e.g. "WINDOWS", "MAC", "LINUX").
GET `/v1/labsuites/{id}/browsers`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/labsuites/{id}/browsers" \
-u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
var browsers = await client.CodelessSuites.GetBrowsersAsync(suiteId);
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.get_lab_suite_browsers(suite_id)
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.labsuites.get_browsers(suite_id)
```
```php
$client = new TestingBot\Client($key, $secret);
$browsers = $client->labSuites()->getBrowsers($suiteId);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
List browsers = restApi.getLabSuiteBrowsers(suiteId);
```
Response
```json
[
{ "name": "firefox", "version": "41", "os": "VISTA" }
]
```
POST `/v1/labsuites/{id}/browsers`
## Update browsers for a Codeless suite
Replaces the browser set attached to a suite. Every test in the suite will run on the new browser list at next trigger.
### Arguments
- **`id` integer required:** Numeric suite ID.
- **`browser_ids` string required:** Comma-separated list of browser\_ids the suite should run on.
POST `/v1/labsuites/{id}/browsers`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#)
Request
```bash
$ curl -X POST "https://api.testingbot.com/v1/labsuites/{id}/browsers" \
-u key:secret \
-d "browser_ids=1,5,12"
```
```csharp
var client = new TestingBotClient(key, secret);
await client.CodelessSuites.SetBrowsersAsync(suiteId, new[] { 1, 22 });
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.set_lab_suite_browsers(suite_id, { browser_ids: '1,5,12' })
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.labsuites.set_browsers(suite_id, '1,5,12')
```
```php
$client = new TestingBot\Client($key, $secret);
$client->labSuites()->setBrowsers($suiteId, [1, 2]);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
boolean success = restApi.setLabSuiteBrowsers(suiteId, "1,5,12");
```
Response
```json
{
"success": true
}
```
GET `/v1/labsuites/{id}/tests`
## List tests in a Codeless suite
Returns the Codeless tests attached to a suite, with pagination.
### Arguments
- **`id` integer required:** Numeric suite ID.
- **`offset` integer:** Skip this many tests.
- **`count` integer:** Number of tests to return.
### Response fields
- **`data` array of lab test objects:** —
- **`meta` meta object:** —
GET `/v1/labsuites/{id}/tests`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/labsuites/{id}/tests" \
-u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
var tests = await client.CodelessSuites.GetTestsAsync(suiteId);
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.get_lab_suite_tests(suite_id)
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.labsuites.get_tests(suite_id)
```
```php
$client = new TestingBot\Client($key, $secret);
$tests = $client->labSuites()->getTests($suiteId, 0, 10);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
TestingbotLabTestCollection tests = restApi.getLabSuiteTests(suiteId);
```
Response
```json
{
"data": [
{
"id": 18666,
"enabled": true,
"name": "MyTest",
"url": "https://mysite.com/",
"created_at": "2017-03-09T12:14:48.000Z",
"updated_at": "2017-03-09T12:19:30.000Z",
"last_run": "2017-03-09T12:19:30.000Z",
"cron": null,
"browsers": [
{ "name": "firefox", "version": "41", "os": "VISTA" }
]
}
],
"meta": { "offset": 0, "count": 1, "total": 1 }
}
```
POST `/v1/labsuites/{id}/tests`
## Add tests to a Codeless suite
Attaches one or more existing Codeless tests to a suite. Pass `test_ids` as a comma-separated list.
### Arguments
- **`id` integer required:** Numeric suite ID.
- **`test_ids` string required:** Comma-separated list of Codeless test IDs to attach.
POST `/v1/labsuites/{id}/tests`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#)
Request
```bash
$ curl -X POST "https://api.testingbot.com/v1/labsuites/{id}/tests" \
-u key:secret \
-d "test_ids=215,228"
```
```csharp
var client = new TestingBotClient(key, secret);
await client.CodelessSuites.AddTestsAsync(suiteId, new long[] { 123, 456 });
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.add_lab_suite_tests(suite_id, { test_ids: '215,228' })
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.labsuites.add_tests(suite_id, '215,228')
```
```php
$client = new TestingBot\Client($key, $secret);
$client->labSuites()->addTests($suiteId, [$labTestId]);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
boolean success = restApi.addLabSuiteTests(suiteId, "215,228");
```
Response
```json
{
"success": true
}
```
DELETE `/v1/labsuites/{id}/tests/{testid}`
## Remove a test from a Codeless suite
Detaches a single Codeless test from a suite. The test itself is preserved.
### Arguments
- **`id` integer required:** Numeric suite ID.
- **`testid` integer required:** Numeric Codeless test ID to detach.
DELETE `/v1/labsuites/{id}/tests/{testid}`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#)
Request
```bash
$ curl -X DELETE "https://api.testingbot.com/v1/labsuites/{id}/tests/{testid}" \
-u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
await client.CodelessSuites.RemoveTestAsync(suiteId, testId);
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.remove_lab_suite_test(suite_id, test_id)
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.labsuites.remove_test(suite_id, test_id)
```
```php
$client = new TestingBot\Client($key, $secret);
$client->labSuites()->removeTest($suiteId, $labTestId);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
boolean success = restApi.removeLabSuiteTest(suiteId, testId);
```
Response
```json
{
"success": true
}
```
[Previous Tunnel](https://testingbot.com/support/api/tunnel) [Next TestingBot Storage](https://testingbot.com/support/api/storage)
---
URL: https://testingbot.com/support/api/storage
# TestingBot Storage
Upload app binaries and test bundles once, then reference them from your capabilities with a tb:// URL.
- **Endpoint:** api.testingbot.com
- **Version:** v1
- **Format:** JSON
- **Auth:** [HTTP Basic](https://testingbot.com/support/api#authentication)
POST `/v1/storage`
## Upload an app to storage
Uploads an APK or IPA to TestingBot storage. Either send the binary as multipart `file` or pass a public `url` that TestingBot will fetch. The returned `app_url` (`tb://`) can be set as the `app` capability on subsequent mobile sessions. Metadata (version, min\_device\_version, thumb) is extracted asynchronously: poll GET /storage/\ until its `state` is `DONE` before reading those fields.
### Arguments
- **`file` file:** Multipart binary upload of an .apk or .ipa.
- **`url` string:** Public HTTPS URL TestingBot will download the binary from.
POST `/v1/storage`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl -X POST "https://api.testingbot.com/v1/storage" \
-u key:secret \
-F "file=@/path/to/app/file/Application-debug.apk"
```
```csharp
var client = new TestingBotClient(key, secret);
var file = await client.Storage.UploadAsync(new FileInfo("/path/to/Application-debug.apk"));
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.upload_local_file(local_file_path)
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.storage.upload_local_file(local_file_path)
```
```php
$api = new TestingBot\TestingBotAPI($key, $secret);
$api->uploadLocalFileToStorage($pathToLocalFile);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
TestingbotStorageUploadResponse uploadResponse = restApi.uploadToStorage(file);
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
const { app_url } = await api.uploadFile(localFilePath);
```
Response
```json
{
"app_url": "tb://398eijf83i"
}
```
POST `/v1/storage/{app_url}`
## Update an app binary by appkey
Replaces the binary stored under an existing `appkey`. The `app_url` (`tb://`) stays the same so deployed CI configurations keep working without changes — useful for "always use the latest" CI flows. Metadata is re-extracted asynchronously: poll GET /storage/\ until its `state` is `DONE`.
### Arguments
- **`app_url` string required:** The app to operate on: its `appkey`, or the full `tb://` app\_url.
- **`file` file:** Multipart binary replacement.
- **`url` string:** Public HTTPS URL to fetch the new binary from.
POST `/v1/storage/{app_url}`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#)
Request
```bash
$ curl -X POST "https://api.testingbot.com/v1/storage/{app_url}" \
-u key:secret \
-F "file=@/path/to/app/file/Application-debug.apk"
```
```csharp
var client = new TestingBotClient(key, secret);
var file = await client.Storage.ReplaceAsync(appKey, new FileInfo("/path/to/Application-debug.apk"));
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.update_local_file(app_url, local_file_path)
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.storage.replace_local_file(app_url, local_file_path)
```
```php
$client = new TestingBot\Client($key, $secret);
$file = $client->storage()->replaceFile($appUrl, '/path/to/Application-debug.apk');
```
Response
```json
{
"app_url": "tb://398eijf83i"
}
```
GET `/v1/storage/{app_url}`
## Get a stored app by appkey
Returns metadata for a single uploaded app — its filename, type, version, icon, and download URL.
### Arguments
- **`app_url` string required:** The app to operate on: its `appkey`, or the full `tb://` app\_url.
### Response fields
- **`id` integer:** Unique numeric storage object ID.
- **`app_url` string:** TestingBot storage URL (`tb://`) — pass as a capability to reference this app in mobile tests.
- **`url` string:** Signed HTTPS URL to download the binary directly.
- **`filename` string:** Original uploaded filename.
- **`type` string:** Detected app type (ANDROID for .apk/.aab, IOS for .ipa/.zip, OTHER otherwise).
- **`version` string:** CFBundleShortVersionString / versionName extracted from the binary.
- **`min_device_version` string:** Minimum OS version required by the app.
- **`thumb` string:** App icon (signed PNG URL).
- **`created_at` timestamp:** Upload timestamp.
- **`state` string:** Metadata extraction state: PROCESSING right after upload, DONE once version / min\_device\_version / thumb are populated. Poll until DONE before relying on those fields.
- **`sim_only` boolean:** True for iOS simulator-only IPAs (no signing).
GET `/v1/storage/{app_url}`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/storage/{app_url}" \
-u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
var file = await client.Storage.GetAsync(appUrlOrKey);
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.get_uploaded_file(app_url)
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.storage.get_stored_file(app_url)
```
```php
$api = new TestingBot\TestingBotAPI($key, $secret);
$api->getStorageFile($appUrl);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
TestingBotStorageFile storedFile = restApi.getStorageFile(appUrl);
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
await api.getStorageFile(appUrl);
```
Response
```json
{
"app_url": "tb://0ec522702cd81ec1374e9b3c",
"url": "https://...",
"id": 261,
"type": "ANDROID",
"filename": "sample.apk",
"version": "1.0.1",
"min_device_version": "12.0",
"thumb": "https://...image.png",
"created_at": "2019-03-08T08:58:32.000Z",
"state": "DONE",
"sim_only": false
}
```
GET `/v1/storage`
## List your storage apps
Paginated list of every app you've uploaded to TestingBot storage, newest first.
### Arguments
- **`offset` integer:** Skip this many apps from the start of the result set.
- **`count` integer max=`500`:** Number of apps to return .
### Response fields
- **`data` array of storage file objects:** Storage objects for this page.
- **`meta` meta object:** —
GET `/v1/storage`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/storage/" \
-u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
var files = await client.Storage.ListAsync();
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.get_uploaded_files(0, 30)
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.storage.get_stored_files(offset=0, limit=30)
```
```php
$api = new TestingBot\TestingBotAPI($key, $secret);
$api->getStorageFiles();
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
TestingBotStorageFileCollection fileList = restApi.getStorageFiles(0, 30);
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
const files = await api.getStorageFiles(offset, count);
```
Response
```json
{
"data": [
{
"app_url": "tb://….",
"url": "…",
"id": 44,
"type": "ANDROID",
"filename": "sample.apk",
"version": "1.0.1",
"min_device_version": "23",
"thumb": "https://...image.png",
"created_at": "2019-01-08T13:42:42.000Z",
"state": "DONE",
"sim_only": false
}
],
"meta": {
"offset": 0,
"count": 10,
"total": 1
}
}
```
DELETE `/v1/storage/{id}`
## Delete a stored app
Permanently removes the binary and its metadata. Sessions referencing the `app_url` after deletion will fail at session-start with "app not found".
### Arguments
- **`id` string required:** Numeric storage ID or appkey to delete.
DELETE `/v1/storage/{id}`
[cURL](https://testingbot.com#) [.NET](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
Request
```bash
$ curl -X DELETE "https://api.testingbot.com/v1/storage/{app_url}" \
-u key:secret
```
```csharp
var client = new TestingBotClient(key, secret);
await client.Storage.DeleteAsync(idOrAppKey);
```
```ruby
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.delete_uploaded_file(file_identifier)
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.storage.remove_file(app_url)
```
```php
$api = new TestingBot\TestingBotAPI($key, $secret);
$api->deleteStorageFile($appUrl);
```
```java
TestingbotREST restApi = new TestingbotREST(key, secret);
boolean success = restApi.deleteStorageFile(appUrl);
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
const success = await api.deleteStorageFile(fileId);
```
Response
```json
{
"success": true
}
```
[Previous Codeless Automation & TestLab](https://testingbot.com/support/api/codeless) [Next Webhooks](https://testingbot.com/support/api/webhooks)
---
URL: https://testingbot.com/support/api/webhooks
# Webhooks
Manage the callbacks TestingBot posts when a test finishes, and verify them without waiting for a run.
- **Endpoint:** api.testingbot.com
- **Version:** v1
- **Format:** JSON
- **Auth:** [HTTP Basic](https://testingbot.com/support/api#authentication)
GET `/v1/webhooks`
## List your webhooks
Returns every webhook on the team. Webhooks fire when a test finishes; use `failure_only`, `alert_types` and `name_filter` to narrow which tests trigger a delivery. The signing secret is never returned here — it is shown once when the webhook is created or its secret is rotated.
### Arguments
- **`offset` integer:** Skip this many webhooks from the start of the result set.
- **`count` integer:** Number of webhooks to return.
### Response fields
- **`data` array of webhook objects:** Webhooks on this team.
- **`meta` meta object:** —
GET `/v1/webhooks`
[cURL](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/webhooks?offset=0&count=10" \
-u key:secret
```
Response
```json
{
"data": [
{
"id": 50,
"name": "CI notifier",
"url": "https://example.com/hooks/testingbot",
"http_method": "POST",
"failure_only": true,
"alert_types": "ALL",
"name_filter": null,
"auth_type": "NONE",
"headers": [{ "key": "X-Source", "value": "testingbot" }],
"params": [],
"payload_template": null,
"created_at": "2026-08-03T10:00:32.000Z",
"updated_at": "2026-08-03T10:00:32.000Z"
}
],
"meta": { "offset": 0, "count": 10, "total": 1 }
}
```
GET `/v1/webhooks/{id}`
## Get a specific webhook
Returns a single webhook by ID. Receiver credentials and the signing secret are never included.
### Arguments
- **`id` integer required:** Numeric webhook ID.
### Response fields
- **`id` integer:** Unique numeric webhook ID.
- **`name` string:** Label for the webhook, shown on the dashboard.
- **`url` string:** Destination URL. May contain `{{VARIABLES}}`, which are substituted at delivery time.
- **`http_method` string:** HTTP method used to deliver: POST, PUT or PATCH.
- **`failure_only` boolean:** When true, only failed tests trigger a delivery.
- **`alert_types` string:** Which grid the test ran on: ALL, VIRTUAL (VM) or REAL (physical device).
- **`name_filter` string:** Glob matched against the test name, falling back to the build identifier. Empty means every test.
- **`auth_type` string:** How the request authenticates against your endpoint: NONE, BASIC or BEARER. The credentials themselves are never returned.
- **`headers` array of object:** Custom request headers, each `{ "key": ..., "value": ... }`. Values support `{{VARIABLES}}`.
- **`params` array of object:** Custom query parameters appended to the URL, each `{ "key": ..., "value": ... }`.
- **`payload_template` string:** Custom JSON body template. Empty means the default TestingBot payload is sent.
- **`created_at` timestamp:** When the webhook was created.
- **`updated_at` timestamp:** When the webhook was last changed.
- **`signing_secret` string:** Secret used to verify the `X-TestingBot-Signature` header. Only present when the webhook is created or its secret is rotated; store it then, because it is never returned again.
GET `/v1/webhooks/{id}`
[cURL](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/webhooks/{id}" \
-u key:secret
```
Response
```json
{
"id": 50,
"name": "CI notifier",
"url": "https://example.com/hooks/testingbot",
"http_method": "POST",
"failure_only": true,
"alert_types": "ALL",
"name_filter": null,
"auth_type": "NONE",
"headers": [{ "key": "X-Source", "value": "testingbot" }],
"params": [],
"payload_template": null,
"created_at": "2026-08-03T10:00:32.000Z",
"updated_at": "2026-08-03T10:00:32.000Z"
}
```
POST `/v1/webhooks`
## Create a webhook
Creates a webhook. The response includes `signing_secret` — the only time it is ever returned — which you use to verify the `X-TestingBot-Signature` header on deliveries. A team may have at most 5 webhooks.
### Arguments
- **`url` string required:** Destination URL. Must be a public http(s) address; private, loopback and link-local hosts are rejected.
- **`name` string:** Label for the webhook. Required once the extended webhook columns are present.
- **`http_method` string:** HTTP method used to deliver. Defaults to POST.
- **`failure_only` boolean:** Deliver only for failed tests. Defaults to false.
- **`alert_types` string:** Which grid to report on: ALL, VIRTUAL or REAL.
- **`name_filter` string:** Glob matched against the test name, falling back to the build identifier.
- **`auth_type` string:** How to authenticate against your endpoint: NONE, BASIC or BEARER.
- **`auth_username` string:** Username for BASIC auth. Write-only, never returned.
- **`auth_password` string:** Password for BASIC auth. Write-only, never returned.
- **`auth_token` string:** Token for BEARER auth. Write-only, never returned.
- **`headers` array:** Custom request headers, each an object with `key` and `value`.
- **`params` array:** Custom query parameters appended to the URL, each an object with `key` and `value`.
- **`payload_template` string:** Custom JSON body template using `{{VARIABLES}}`. Omit to send the default TestingBot payload.
POST `/v1/webhooks`
[cURL](https://testingbot.com#)
Request
```bash
$ curl -X POST "https://api.testingbot.com/v1/webhooks" \
-u key:secret \
-H "Content-Type: application/json" \
-d '{
"name": "CI notifier",
"url": "https://example.com/hooks/testingbot",
"failure_only": true,
"alert_types": "ALL",
"headers": [{ "key": "X-Source", "value": "testingbot" }]
}'
```
Response
```json
{
"id": 50,
"name": "CI notifier",
"url": "https://example.com/hooks/testingbot",
"http_method": "POST",
"failure_only": true,
"alert_types": "ALL",
"name_filter": null,
"auth_type": "NONE",
"headers": [{ "key": "X-Source", "value": "testingbot" }],
"params": [],
"payload_template": null,
"created_at": "2026-08-03T10:00:32.000Z",
"updated_at": "2026-08-03T10:00:32.000Z",
"signing_secret": "whsec_a1b2c3d4e5f60718293a4b5c6d7e8f90"
}
```
PUT `/v1/webhooks/{id}`
## Update a webhook
Updates a webhook. Only the fields you send are changed. Sending `auth_password` or `auth_token` replaces the stored value; omitting them leaves it alone. The signing secret is not returned — rotate it if you have lost it.
### Arguments
- **`id` integer required:** Numeric webhook ID.
- **`url` string:** Destination URL. Must be a public http(s) address.
- **`name` string:** Label for the webhook.
- **`http_method` string:** HTTP method used to deliver.
- **`failure_only` boolean:** Deliver only for failed tests.
- **`alert_types` string:** Which grid to report on: ALL, VIRTUAL or REAL.
- **`name_filter` string:** Glob matched against the test name, falling back to the build identifier.
- **`auth_type` string:** How to authenticate against your endpoint.
- **`auth_username` string:** Username for BASIC auth. Write-only.
- **`auth_password` string:** Password for BASIC auth. Write-only.
- **`auth_token` string:** Token for BEARER auth. Write-only.
- **`headers` array:** Custom request headers, each an object with `key` and `value`. Replaces the existing set.
- **`params` array:** Custom query parameters, each an object with `key` and `value`. Replaces the existing set.
- **`payload_template` string:** Custom JSON body template. Send an empty string to go back to the default payload.
### Response fields
- **`id` integer:** Unique numeric webhook ID.
- **`name` string:** Label for the webhook, shown on the dashboard.
- **`url` string:** Destination URL. May contain `{{VARIABLES}}`, which are substituted at delivery time.
- **`http_method` string:** HTTP method used to deliver: POST, PUT or PATCH.
- **`failure_only` boolean:** When true, only failed tests trigger a delivery.
- **`alert_types` string:** Which grid the test ran on: ALL, VIRTUAL (VM) or REAL (physical device).
- **`name_filter` string:** Glob matched against the test name, falling back to the build identifier. Empty means every test.
- **`auth_type` string:** How the request authenticates against your endpoint: NONE, BASIC or BEARER. The credentials themselves are never returned.
- **`headers` array of object:** Custom request headers, each `{ "key": ..., "value": ... }`. Values support `{{VARIABLES}}`.
- **`params` array of object:** Custom query parameters appended to the URL, each `{ "key": ..., "value": ... }`.
- **`payload_template` string:** Custom JSON body template. Empty means the default TestingBot payload is sent.
- **`created_at` timestamp:** When the webhook was created.
- **`updated_at` timestamp:** When the webhook was last changed.
- **`signing_secret` string:** Secret used to verify the `X-TestingBot-Signature` header. Only present when the webhook is created or its secret is rotated; store it then, because it is never returned again.
PUT `/v1/webhooks/{id}`
[cURL](https://testingbot.com#)
Request
```bash
$ curl -X PUT "https://api.testingbot.com/v1/webhooks/{id}" \
-u key:secret \
-H "Content-Type: application/json" \
-d '{ "name": "CI notifier (failures only)", "failure_only": true }'
```
Response
```json
{
"id": 50,
"name": "CI notifier (failures only)",
"url": "https://example.com/hooks/testingbot",
"http_method": "POST",
"failure_only": true,
"alert_types": "ALL",
"name_filter": null,
"auth_type": "NONE",
"headers": [{ "key": "X-Source", "value": "testingbot" }],
"params": [],
"payload_template": null,
"created_at": "2026-08-03T10:00:32.000Z",
"updated_at": "2026-08-03T10:04:11.000Z"
}
```
DELETE `/v1/webhooks/{id}`
## Delete a webhook
Permanently deletes the webhook. Deliveries stop immediately; tests already in flight will not fire it.
### Arguments
- **`id` integer required:** Numeric webhook ID.
DELETE `/v1/webhooks/{id}`
[cURL](https://testingbot.com#)
Request
```bash
$ curl -X DELETE "https://api.testingbot.com/v1/webhooks/{id}" \
-u key:secret
```
Response
```json
{
"success": true,
"message": "Webhook \"https://example.com/hooks/testingbot\" deleted."
}
```
POST `/v1/webhooks/{id}/rotate-secret`
## Rotate a webhook signing secret
Issues a new signing secret and returns it. The previous secret stops verifying immediately, so update your receiver before rotating in production. This is the only other time the secret is returned.
### Arguments
- **`id` integer required:** Numeric webhook ID.
POST `/v1/webhooks/{id}/rotate-secret`
[cURL](https://testingbot.com#)
Request
```bash
$ curl -X POST "https://api.testingbot.com/v1/webhooks/{id}/rotate-secret" \
-u key:secret
```
Response
```json
{
"id": 50,
"signing_secret": "whsec_f0e9d8c7b6a5948372615f4e3d2c1b0a"
}
```
POST `/v1/webhooks/{id}/test`
## Send a test delivery
Delivers a sample payload to the webhook right now and returns what your endpoint responded, so you can verify the URL, authentication and signature verification without waiting for a test to finish. Sample data is used, not a real test result. Rate limited separately from the rest of the API.
### Arguments
- **`id` integer required:** Numeric webhook ID.
POST `/v1/webhooks/{id}/test`
[cURL](https://testingbot.com#)
Request
```bash
$ curl -X POST "https://api.testingbot.com/v1/webhooks/{id}/test" \
-u key:secret
```
Response
```json
{
"ok": true,
"status": 200,
"duration_ms": 148,
"headers": { "content-type": "application/json" },
"body": "{\"received\":true}",
"sent_payload": "{\"id\":1,\"status\":\"FAILED\",\"name\":\"MyTest::testTitle\"}"
}
```
[Previous TestingBot Storage](https://testingbot.com/support/api/storage) [Next App Automate](https://testingbot.com/support/api/app-automate)
---
URL: https://testingbot.com/support/api/app-automate
# App Automate
Everything for native mobile test runs: Maestro flows, Espresso and XCUITest suites, and their reports.
- **Endpoint:** api.testingbot.com
- **Version:** v1
- **Format:** JSON
- **Auth:** [HTTP Basic](https://testingbot.com/support/api#authentication)
## App Automate: Espresso
POST `/v1/app-automate/espresso/app`
## Upload an Android app for Espresso
Uploads the production `.apk` you want to test under Espresso. Returns the `app_url` to reference when starting a run.
POST `/v1/app-automate/espresso/app`
[cURL](https://testingbot.com#)
Request
```bash
$ curl -X POST "https://api.testingbot.com/v1/app-automate/espresso/app" \
-u key:secret
```
GET `/v1/app-automate/espresso/{id}`
## Get an Espresso project
Returns the metadata and current state of an Espresso project.
### Arguments
- **`id` integer required:** —
### Response fields
- **`runs` array of object:** Runs belonging to the project. For Maestro project status, entries match MaestroRunDetail except that the run-level and per-flow `report` fields are omitted; use the Maestro run detail, flow, junit\_report, or html\_report endpoints to retrieve reports.
- **`success` boolean:** True when every run in the project passed.
- **`completed` boolean:** True when every run has reached a terminal state.
GET `/v1/app-automate/espresso/{id}`
[cURL](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/app-automate/espresso/{id}" \
-u key:secret
```
GET `/v1/app-automate/espresso/{id}/report`
## Get the Espresso JUnit report
Returns the JUnit-format XML report for an Espresso run once it has completed.
### Arguments
- **`id` integer required:** —
GET `/v1/app-automate/espresso/{id}/report`
[cURL](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/app-automate/espresso/{id}/report" \
-u key:secret
```
POST `/v1/app-automate/espresso/{id}/run`
## Run an Espresso test suite
Triggers an asynchronous Espresso run across the requested device capabilities. Returns run metadata; poll status via the standard tests API.
### Arguments
- **`id` integer required:** —
POST `/v1/app-automate/espresso/{id}/run`
[cURL](https://testingbot.com#)
Request
```bash
$ curl -X POST "https://api.testingbot.com/v1/app-automate/espresso/{id}/run" \
-u key:secret
```
POST `/v1/app-automate/espresso/{id}/tests`
## Upload Espresso test APK
Attaches an Espresso test `.apk` (the instrumented test bundle) to a project. Pair with `/app` to provide both halves of the test.
### Arguments
- **`id` integer required:** —
POST `/v1/app-automate/espresso/{id}/tests`
[cURL](https://testingbot.com#)
Request
```bash
$ curl -X POST "https://api.testingbot.com/v1/app-automate/espresso/{id}/tests" \
-u key:secret
```
GET `/v1/app-automate/espresso/{project_id}/{id}`
## Get a specific Espresso run
Returns the detail record for a single Espresso run inside a project.
### Arguments
- **`project_id` integer required:** —
- **`id` integer required:** —
### Response fields
- **`id` integer:** Unique numeric Espresso run ID.
- **`created_at` timestamp:** —
- **`status` string:** WAITING, READY, DONE or FAILED.
- **`capabilities` object:** Capabilities the run was started with.
- **`success` boolean:** Whether the run passed. False until a session has completed.
- **`report` object:** Test counts and per-test results for the run.
- **`test` object:** Session detail. Only present once a session has been allocated.
GET `/v1/app-automate/espresso/{project_id}/{id}`
[cURL](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/app-automate/espresso/{project_id}/{id}" \
-u key:secret
```
## App Automate: Maestro
GET `/v1/app-automate/maestro`
## List your Maestro projects
Paginated list of Maestro app-automate projects (one per uploaded test bundle) on the account, newest first.
### Response fields
- **`data` array of maestro project objects:** Projects for this page, newest first.
- **`meta` meta object:** —
GET `/v1/app-automate/maestro`
[cURL](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/app-automate/maestro" \
-u key:secret
```
POST `/v1/app-automate/maestro/app`
## Upload an Android/iOS app for Maestro
Uploads an `.apk` / `.ipa` to attach to a Maestro project. Returns the `app_url` (`tb://`) you'll pass as a capability when running the project.
### Arguments
- **`file` string required:** —
- **`checksum` string:** MD5 checksum (base64) of the app file for deduplication
POST `/v1/app-automate/maestro/app`
[cURL](https://testingbot.com#)
Request
```bash
$ curl -X POST "https://api.testingbot.com/v1/app-automate/maestro/app" \
-u key:secret \
-F "file=@/path/to/app/file/Application-debug.apk"
```
POST `/v1/app-automate/maestro/app/checksum`
## Check whether an app needs uploading
Returns whether a binary identified by checksum already exists in TestingBot Storage. Used by the CLI to skip re-uploading unchanged apps.
### Arguments
- **`checksum` string required:** MD5 checksum (base64) of the app file
POST `/v1/app-automate/maestro/app/checksum`
[cURL](https://testingbot.com#)
Request
```bash
$ curl -X POST "https://api.testingbot.com/v1/app-automate/maestro/app/checksum" \
-u key:secret \
-d "checksum=$(openssl md5 -binary Application-debug.apk | base64)"
```
POST `/v1/app-automate/maestro/other-apps`
## Upload additional apps for a Maestro session
Uploads supplementary `.apk`/`.ipa` binaries that should be installed alongside the primary app under test (e.g. dependent apps, test runners).
### Arguments
- **`file` string required:** —
POST `/v1/app-automate/maestro/other-apps`
[cURL](https://testingbot.com#)
Request
```bash
$ curl -X POST "https://api.testingbot.com/v1/app-automate/maestro/other-apps" \
-u key:secret \
-F "file=@/path/to/app/file/DependencyApp-debug.apk"
```
GET `/v1/app-automate/maestro/runs`
## List your Maestro runs
Paginated list of Maestro app-automate runs across all projects on the account, newest first. Lightweight: no reports are inflated. Use GET /:project\_id/:id for a single run's full results.
### Response fields
- **`data` array of maestro run summary objects:** Runs for this page, newest first.
- **`meta` meta object:** —
GET `/v1/app-automate/maestro/runs`
[cURL](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/app-automate/maestro/runs" \
-u key:secret
```
GET `/v1/app-automate/maestro/runs/{build_name}`
## Look up Maestro runs by build name
Resolves a build/run name to every matching run on the account (a build can fan out to multiple device runs). Returns project\_id + run\_id without inflating reports. Use GET /:project\_id/:id for a single run's full results.
### Arguments
- **`build_name` string required:** —
### Response fields
- **`data` array of maestro run summary objects:** Every run submitted under this build name, newest first.
GET `/v1/app-automate/maestro/runs/{build_name}`
[cURL](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/app-automate/maestro/runs/{build_name}" \
-u key:secret
```
GET `/v1/app-automate/maestro/{id}`
## Get a Maestro project
Returns the metadata and current state of a single Maestro app-automate project.
### Arguments
- **`id` integer required:** —
### Response fields
- **`runs` array of object:** Runs belonging to the project. For Maestro project status, entries match MaestroRunDetail except that the run-level and per-flow `report` fields are omitted; use the Maestro run detail, flow, junit\_report, or html\_report endpoints to retrieve reports.
- **`success` boolean:** True when every run in the project passed.
- **`completed` boolean:** True when every run has reached a terminal state.
GET `/v1/app-automate/maestro/{id}`
[cURL](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/app-automate/maestro/{id}" \
-u key:secret
```
POST `/v1/app-automate/maestro/{id}/run`
## Run a Maestro app flow
Triggers an asynchronous Maestro run on the uploaded project against one or more device capabilities. Returns one run entry per capability.
### Arguments
- **`id` integer required:** —
POST `/v1/app-automate/maestro/{id}/run`
[cURL](https://testingbot.com#)
Request
```bash
$ curl -X POST "https://api.testingbot.com/v1/app-automate/maestro/{id}/run" \
-u key:secret
```
POST `/v1/app-automate/maestro/{id}/tests`
## Upload a Maestro test bundle
Attaches a Maestro flow archive (`.zip` or `.yaml`) to an existing Maestro project. Replaces any previously uploaded tests for that project.
### Arguments
- **`id` integer required:** —
POST `/v1/app-automate/maestro/{id}/tests`
[cURL](https://testingbot.com#)
Request
```bash
$ curl -X POST "https://api.testingbot.com/v1/app-automate/maestro/{id}/tests" \
-u key:secret
```
GET `/v1/app-automate/maestro/{project_id}/{id}`
## Get a specific Maestro run
Returns the detail record for a single Maestro run inside a project, including capability used, status, and per-flow results.
### Arguments
- **`project_id` integer required:** —
- **`id` integer required:** —
### Response fields
- **`id` integer:** Unique numeric Maestro run ID.
- **`status` string:** WAITING, READY, DONE, FAILED or CANCELLED, computed from the newest attempt of each flow.
- **`success` integer:** 1 when every flow passed, 0 otherwise.
- **`capabilities` object:** Capabilities the run was started with.
- **`options` object:** Maestro options the run was started with; `{}` when none were given. Variables passed as `secretEnv` are listed in `env` with the value `***` and named in `secretEnvKeys`; their values are never returned.
- **`created_at` timestamp:** —
- **`completed` boolean:** Whether every run in the parent project has reached a terminal state.
- **`assets_synced` boolean:** Whether asset processing has finished for every flow in the run.
- **`meta` object:** Metadata supplied at submission. Only present when the run carries any.
- **`report` string:** JUnit XML for the run. Only present once a report exists.
- **`test` object:** Session detail. Only present once a session has been allocated.
- **`assets` object:** Logs, video and screenshots. Only present once asset processing has finished.
- **`environment` object:** Device the run was allocated. Only present once one has been assigned.
- **`flows` array of maestro flow result objects:** Every flow attempt, including retries. Only present once the run has flows.
- **`error_messages` array of string:** Run-level failures from the newest attempt of each flow, so a failure a retry has since fixed does not linger. Only present when the latest attempts reported any.
GET `/v1/app-automate/maestro/{project_id}/{id}`
[cURL](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/app-automate/maestro/{project_id}/{id}" \
-u key:secret
```
POST `/v1/app-automate/maestro/{project_id}/{run_id}/cancel`
## Cancel a Maestro run
Immediately cancels a queued or running Maestro run. The run is marked CANCELLED so its workers stop requesting devices and stop retrying, every flow that has not finished yet is closed out as cancelled, any device session the run already acquired is released, and any request still queued on the grid is dropped. Flows that already completed keep their results and reports. Cancelling is idempotent only in the sense that it is safe to call once: a run that has already reached a terminal state (DONE, FAILED or CANCELLED) returns 409.
### Arguments
- **`project_id` integer required:** Numeric Maestro project ID the run belongs to.
- **`run_id` integer required:** Numeric Maestro run ID to cancel.
### Response fields
- **`success` boolean:** Whether the cancel was applied.
- **`run` object:** The run in its post-cancel state.
- **`cancelled_flow_count` integer:** How many flows were still queued or running and got closed out. 0 means the run held no unfinished flows (its device session, if any, is still released).
POST `/v1/app-automate/maestro/{project_id}/{run_id}/cancel`
[cURL](https://testingbot.com#)
Request
```bash
$ curl -X POST "https://api.testingbot.com/v1/app-automate/maestro/{project_id}/{run_id}/cancel" \
-u key:secret
```
GET `/v1/app-automate/maestro/{project_id}/{run_id}/flow/{flow_id}`
## Get a Maestro flow result
Returns the per-step results, logs, and screenshots for one flow inside a Maestro run.
### Arguments
- **`project_id` integer required:** —
- **`run_id` integer required:** —
- **`flow_id` integer required:** —
### Response fields
- **`id` integer:** Unique numeric flow-attempt ID.
- **`maestro_flow_id` integer:** The flow this attempt ran.
- **`status` string:** WAITING, READY, DONE, FAILED or CANCELLED.
- **`name` string:** Flow name(s) covered by this attempt.
- **`requested_at` timestamp:** —
- **`test_case_id` integer:** Test session backing this attempt. Only present once one has been allocated.
- **`completed_at` timestamp:** Only present once the attempt has finished.
- **`report` string:** JUnit XML for this attempt. Only present once a report exists, and omitted on the project status endpoint (GET /app-automate/maestro/:id); read it from the run detail endpoint (GET /app-automate/maestro/:project\_id/:id) or this flow endpoint.
- **`shard_index` integer:** Shard this attempt covered. Only present on a sharded run.
- **`flows` array of string:** Relative paths of the flow files this shard runs, in execution order. Only present on a sharded run.
- **`success` integer:** 1 when the flow passed. Only present on a terminal status.
- **`error_messages` array of string:** Failure detail. Only present on a terminal status.
- **`test` object:** Session detail. Only present once a session has been allocated.
- **`assets` object:** Logs, video and screenshots. Only present once asset processing has finished.
- **`assets_synced` boolean:** Whether this attempt's assets have finished processing.
GET `/v1/app-automate/maestro/{project_id}/{run_id}/flow/{flow_id}`
[cURL](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/app-automate/maestro/{project_id}/{run_id}/flow/{flow_id}" \
-u key:secret
```
GET `/v1/app-automate/maestro/{project_id}/{run_id}/html_report`
## Maestro run HTML report
Returns a hosted HTML report URL summarising the Maestro run results.
### Arguments
- **`project_id` integer required:** —
- **`run_id` integer required:** —
### Response fields
- **`html_report` string:** Signed URL to the rendered HTML report.
GET `/v1/app-automate/maestro/{project_id}/{run_id}/html_report`
[cURL](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/app-automate/maestro/{project_id}/{run_id}/html_report" \
-u key:secret
```
GET `/v1/app-automate/maestro/{project_id}/{run_id}/html_report_detailed`
## Maestro run detailed HTML report
Returns a detailed per-step HTML report URL for the Maestro run, including screenshots and logs.
### Arguments
- **`project_id` integer required:** —
- **`run_id` integer required:** —
### Response fields
- **`html_report_detailed` string:** Signed URL to the per-step HTML report.
GET `/v1/app-automate/maestro/{project_id}/{run_id}/html_report_detailed`
[cURL](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/app-automate/maestro/{project_id}/{run_id}/html_report_detailed" \
-u key:secret
```
GET `/v1/app-automate/maestro/{project_id}/{run_id}/junit_report`
## Maestro run JUnit report
Returns a JUnit-format XML report for a completed Maestro run. Suitable for ingestion by CI systems (Jenkins, GitHub Actions, etc.).
### Arguments
- **`project_id` integer required:** —
- **`run_id` integer required:** —
### Response fields
- **`junit_report` string:** JUnit XML for the run, as a string.
GET `/v1/app-automate/maestro/{project_id}/{run_id}/junit_report`
[cURL](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/app-automate/maestro/{project_id}/{run_id}/junit_report" \
-u key:secret
```
POST `/v1/app-automate/maestro/{project_id}/{run_id}/retry`
## Retry a Maestro run
Re-runs an entire Maestro run. Creates a new run on the same device configuration (capabilities) and options, re-scheduling every flow. Returns the newly created run; poll it for status.
### Arguments
- **`project_id` integer required:** —
- **`run_id` integer required:** —
POST `/v1/app-automate/maestro/{project_id}/{run_id}/retry`
[cURL](https://testingbot.com#)
Request
```bash
$ curl -X POST "https://api.testingbot.com/v1/app-automate/maestro/{project_id}/{run_id}/retry" \
-u key:secret
```
POST `/v1/app-automate/maestro/{project_id}/{run_id}/stop`
## Stop a Maestro run
Cancels a running Maestro run before it completes. Already-completed flows are preserved; in-flight flows are marked cancelled. Alias of the cancel endpoint, kept because testingbotctl calls it from its Ctrl-C handler; unlike cancel it always answers 200, so stopping an already-finished run is a no-op rather than a 409.
### Arguments
- **`project_id` integer required:** —
- **`run_id` integer required:** —
POST `/v1/app-automate/maestro/{project_id}/{run_id}/stop`
[cURL](https://testingbot.com#)
Request
```bash
$ curl -X POST "https://api.testingbot.com/v1/app-automate/maestro/{project_id}/{run_id}/stop" \
-u key:secret
```
POST `/v1/app-automate/maestro/{project_id}/{run_id}/{flow_id}/retry`
## Retry a Maestro flow
Re-runs a single flow from a Maestro run. Creates a new flow attempt inside the same run and schedules it. Returns the newly created flow record; poll the run for its status.
### Arguments
- **`project_id` integer required:** —
- **`run_id` integer required:** —
- **`flow_id` integer required:** —
POST `/v1/app-automate/maestro/{project_id}/{run_id}/{flow_id}/retry`
[cURL](https://testingbot.com#)
Request
```bash
$ curl -X POST "https://api.testingbot.com/v1/app-automate/maestro/{project_id}/{run_id}/{flow_id}/retry" \
-u key:secret
```
## App Automate: XCUITest
POST `/v1/app-automate/xcuitest/app`
## Upload an iOS app for XCUITest
Uploads the production iOS `.ipa` you want to test under XCUITest.
POST `/v1/app-automate/xcuitest/app`
[cURL](https://testingbot.com#)
Request
```bash
$ curl -X POST "https://api.testingbot.com/v1/app-automate/xcuitest/app" \
-u key:secret
```
GET `/v1/app-automate/xcuitest/{id}`
## Get an XCUITest project
Returns the metadata and current state of an XCUITest project.
### Arguments
- **`id` integer required:** —
### Response fields
- **`runs` array of object:** Runs belonging to the project. For Maestro project status, entries match MaestroRunDetail except that the run-level and per-flow `report` fields are omitted; use the Maestro run detail, flow, junit\_report, or html\_report endpoints to retrieve reports.
- **`success` boolean:** True when every run in the project passed.
- **`completed` boolean:** True when every run has reached a terminal state.
GET `/v1/app-automate/xcuitest/{id}`
[cURL](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/app-automate/xcuitest/{id}" \
-u key:secret
```
GET `/v1/app-automate/xcuitest/{id}/report`
## Get the XCUITest JUnit report
Returns the JUnit-format XML report for a completed XCUITest run.
### Arguments
- **`id` integer required:** —
GET `/v1/app-automate/xcuitest/{id}/report`
[cURL](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/app-automate/xcuitest/{id}/report" \
-u key:secret
```
POST `/v1/app-automate/xcuitest/{id}/run`
## Run an XCUITest suite
Triggers an asynchronous XCUITest run across the requested device capabilities.
### Arguments
- **`id` integer required:** —
POST `/v1/app-automate/xcuitest/{id}/run`
[cURL](https://testingbot.com#)
Request
```bash
$ curl -X POST "https://api.testingbot.com/v1/app-automate/xcuitest/{id}/run" \
-u key:secret
```
POST `/v1/app-automate/xcuitest/{id}/tests`
## Upload XCUITest test bundle
Attaches the XCUITest test bundle (.zip) to a project. Pair with `/app` to provide both halves of the test.
### Arguments
- **`id` integer required:** —
POST `/v1/app-automate/xcuitest/{id}/tests`
[cURL](https://testingbot.com#)
Request
```bash
$ curl -X POST "https://api.testingbot.com/v1/app-automate/xcuitest/{id}/tests" \
-u key:secret
```
GET `/v1/app-automate/xcuitest/{project_id}/{id}`
## Get a specific XCUITest run
Returns the detail record for a single XCUITest run inside a project.
### Arguments
- **`project_id` integer required:** —
- **`id` integer required:** —
### Response fields
- **`id` integer:** Unique numeric XCUITest run ID.
- **`created_at` timestamp:** —
- **`status` string:** WAITING, READY, DONE or FAILED.
- **`capabilities` object:** Capabilities the run was started with.
- **`success` boolean:** Whether the run passed. False until a session has completed.
- **`test` object:** Session detail. Only present once a session has been allocated.
GET `/v1/app-automate/xcuitest/{project_id}/{id}`
[cURL](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/app-automate/xcuitest/{project_id}/{id}" \
-u key:secret
```
GET `/v1/app-automate/xcuitest/{project_id}/{run_id}/html_report`
## XCUITest run HTML report
Returns a self-contained HTML report for a single XCUITest run.
### Arguments
- **`project_id` integer required:** —
- **`run_id` integer required:** —
### Response fields
- **`html_report` string:** Signed URL to the rendered HTML report.
GET `/v1/app-automate/xcuitest/{project_id}/{run_id}/html_report`
[cURL](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/app-automate/xcuitest/{project_id}/{run_id}/html_report" \
-u key:secret
```
GET `/v1/app-automate/xcuitest/{project_id}/{run_id}/junit_report`
## XCUITest run JUnit report
Returns a JUnit-format XML report for a single completed XCUITest run. Suitable for ingestion by CI systems (Jenkins, GitHub Actions, etc.).
### Arguments
- **`project_id` integer required:** —
- **`run_id` integer required:** —
### Response fields
- **`junit_report` string:** JUnit XML for the run, as a string.
GET `/v1/app-automate/xcuitest/{project_id}/{run_id}/junit_report`
[cURL](https://testingbot.com#)
Request
```bash
$ curl "https://api.testingbot.com/v1/app-automate/xcuitest/{project_id}/{run_id}/junit_report" \
-u key:secret
```
[Previous Webhooks](https://testingbot.com/support/api/webhooks)
---
URL: https://testingbot.com/support/k6-browser-testing
# k6 Browser Testing
k6 is an open-source tool designed to do easy load testing against websites. With the [k6 browser module](https://grafana.com/docs/k6/latest/using-k6-browser/), you can connect to a CDP-compatible browser grid such as TestingBot. Simply configure your k6 test to use the TestingBot grid, and instantly get access to a high capacity cloud of browsers.
With this integration, you can run tests in parallel, targeting different browsers, versions and operating systems. Each test comes with logs, screenshots and a video.
## Setup
To get started, please make sure you [install k6](https://k6.io/docs/get-started/installation/).
### macOS
```bash
brew install k6
```
### Windows
```bash
winget install k6 --source winget
```
### Linux
```bash
sudo gpg -k
sudo gpg --no-default-keyring --keyring /usr/share/keyrings/k6-archive-keyring.gpg --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys C5AD17C747E3415A3642D57D77C6C491D6AC1D69
echo "deb [signed-by=/usr/share/keyrings/k6-archive-keyring.gpg] https://dl.k6.io/deb stable main" | sudo tee /etc/apt/sources.list.d/k6.list
sudo apt-get update
sudo apt-get install k6
```
## Example
Next, let's run a sample browser test using k6 on the TestingBot browser grid. Please see the example below.
To connect to the TestingBot CDP grid, you will need to specify a `K6_BROWSER_WS_URL` environment variable. This needs to contain the URL to the TestingBot grid, together with the credentials and (url-encoded) capabilities.
```bash
K6_BROWSER_WS_URL="wss://cloud.testingbot.com/?key=API_KEY&secret=API_SECRET&capabilities=%7B%22browserName%22%3A%22chrome%22%2C%22browserVersion%22%3A%22latest%22%7D"
```
To run the example below, you can use the following command in your terminal:
```bash
K6_BROWSER_WS_URL="wss://cloud.testingbot.com/?key=API_KEY&secret=API_SECRET&capabilities=%7B%22browserName%22%3A%22chrome%22%2C%22browserVersion%22%3A%22latest%22%7D" K6_BROWSER_ENABLED=true k6 run k6_sample.js
```
**k6\_sample.js**
```javascript
import { browser } from 'k6/browser';
import { check } from 'k6';
export const options = {
scenarios: {
ui: {
executor: 'shared-iterations',
options: {
browser: {
type: 'chromium',
},
},
},
},
thresholds: {
checks: ['rate==1.0'],
},
};
export default async function () {
const context = await browser.newContext();
const page = await context.newPage();
try {
await page.goto('https://test.k6.io/my_messages.php');
await page.locator('input[name="login"]').type('admin');
await page.locator('input[name="password"]').type('123');
const submitButton = page.locator('input[type="submit"]');
await Promise.all([page.waitForNavigation(), submitButton.click()]);
const header = await page.locator('h2').textContent();
check(page, {
header: header == 'Welcome, admin!',
});
} finally {
await page.close();
}
}
```
### K6\_BROWSER\_WS\_URL
To construct the `K6_BROWSER_WS_URL`, you'll need to urlencode the capabilities:
```javascript
const capabilities = {
browserName: 'chrome',
browserVersion: 'latest',
platformName: 'VENTURA'
}
const url = `wss://cloud.testingbot.com/?key=API_KEY&secret=API_SECRET&capabilities=${encodeURIComponent(JSON.stringify(capabilities))}`
```
## Test Results
You can find the results of your k6 browser test in your terminal.
The TestingBot member area will contain a list of all k6 tests, together with logs, screenshots and a video of each test.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/app-automate
Documentation
# Mobile App Test Automation
TestingBot runs automated tests for native iOS, Android and cross-platform apps on real iPhones, iPads and Android phones and tablets in our EU datacenter, plus iOS simulators and Android emulators for fast iteration. Bring your existing Appium, XCUITest, Espresso, Maestro, Mobilewright or Electron tests and point them at the TestingBot grid.
## Pick a mobile test framework
TestingBot supports the major mobile and cross-platform test frameworks. Each card links to a complete setup guide with code examples in every supported language. Not sure which framework to pick? See the [framework comparison](https://testingbot.com#compare) below.
### [Appium](https://testingbot.com/support/app-automate/appium)
W3C WebDriver framework for cross-platform iOS and Android app automation.
- [Java](https://testingbot.com/support/app-automate/appium/java)
- [Python](https://testingbot.com/support/app-automate/appium/python)
- [Node.js](https://testingbot.com/support/app-automate/appium/nodejs)
- [Ruby](https://testingbot.com/support/app-automate/appium/ruby)
- [C#](https://testingbot.com/support/app-automate/appium/csharp)
- [PHP](https://testingbot.com/support/app-automate/appium/php)
- [Flutter](https://testingbot.com/support/app-automate/appium/flutter)
### [XCUITest](https://testingbot.com/support/app-automate/xcuitest)
Apple's native iOS UI testing framework, written in Swift or Objective-C.
- [XCUITest guide](https://testingbot.com/support/app-automate/xcuitest)
### [Espresso](https://testingbot.com/support/app-automate/espresso)
Google's native Android UI testing framework, written in Java or Kotlin.
- [Espresso guide](https://testingbot.com/support/app-automate/espresso)
### [Maestro](https://testingbot.com/support/app-automate/maestro)
YAML-based declarative test runner for iOS and Android.
- [Maestro guide](https://testingbot.com/support/app-automate/maestro)
### [Mobilewright](https://testingbot.com/support/app-automate/mobilewright)
Playwright-style mobile tests in TypeScript for iOS and Android.
- [Mobilewright guide](https://testingbot.com/support/app-automate/mobilewright)
### [Electron](https://testingbot.com/support/app-automate/electron)
Test cross-platform Electron desktop apps built with web technologies.
- [Electron guide](https://testingbot.com/support/app-automate/electron)
### [Smart TV](https://testingbot.com/support/app-automate/smart-tv)
Automate Smart TV and OTT applications on real connected TV hardware.
- [Smart TV guide](https://testingbot.com/support/app-automate/smart-tv)
## Quick start: run your first mobile app test
Four steps to point an existing Appium, XCUITest, Espresso or Maestro test at the TestingBot cloud:
1. **Sign up and grab your API key.** [Create a free account](https://testingbot.com/users/sign_up). Your API key and secret are in the dashboard under Account, then Keys and Authentication.
2. **Upload your app binary to TestingBot Storage.** Upload your `.ipa` (iOS), `.apk` (Android) or Maestro flow ZIP to [TestingBot Storage](https://testingbot.com/support/api/storage#upload). The response returns a `tb://` URL you reference in test capabilities.
3. **Point your framework at the TestingBot endpoint.** Connect to `https://hub.testingbot.com/wd/hub`. Pass your key and secret via `tb:options`, set `platformName`, `deviceName`, `platformVersion` and `app` (the `tb://` URL).
4. **Run the test and view results.** TestingBot streams a live video, screenshots, device logs and framework command logs into the dashboard while the test runs and after it finishes.
## Appium vs XCUITest vs Espresso vs Maestro
All four frameworks run on TestingBot. Pick the one that fits your codebase and team:
| | Appium | XCUITest | Espresso | Maestro |
| Platforms | iOS and Android | iOS only | Android only | iOS and Android |
| Languages | Java, Python, Ruby, JavaScript, C#, PHP, Flutter | Swift, Objective-C | Java, Kotlin | YAML (declarative) |
| Protocol | W3C WebDriver | Apple XCTest runner | Google AndroidJUnitRunner | Maestro CLI |
| Test speed | Slower (extra protocol layer) | Fast (native runner) | Fast (native runner) | Fast (direct UI driver) |
| Best for | One codebase across iOS and Android, language flexibility | iOS teams already using XCTest in Xcode | Android teams already using AndroidJUnit in Studio | QA teams that want readable, low-code flows |
## Frequently asked questions
What is mobile app test automation?
Mobile app test automation runs UI tests against native iOS and Android applications, exercising the app exactly like a user would. TestingBot runs these tests on real iPhones, iPads, Android phones and tablets in our EU datacenter, plus iOS simulators and Android emulators when you want faster iteration. See the [supported device list](https://testingbot.com/support/app-automate/devices) for current models.
Which mobile test frameworks does TestingBot support?
TestingBot supports Appium for cross-platform iOS and Android, Apple's XCUITest for iOS, Google's Espresso for Android, Maestro for declarative cross-platform flows, Mobilewright for Playwright-style tests in TypeScript, and Electron for cross-platform desktop apps. See the dedicated guides for [Appium](https://testingbot.com/support/app-automate/appium), [XCUITest](https://testingbot.com/support/app-automate/xcuitest), [Espresso](https://testingbot.com/support/app-automate/espresso), [Maestro](https://testingbot.com/support/app-automate/maestro), [Mobilewright](https://testingbot.com/support/app-automate/mobilewright) and [Electron](https://testingbot.com/support/app-automate/electron).
Can I run tests on real iPhones and Android devices, not simulators?
Yes. TestingBot runs automated tests on physical iPhones, iPads, and Android phones and tablets housed in our EU datacenter, alongside iOS simulators and Android emulators. Use `realDevice: true` in your capabilities to target real hardware. See [real device testing](https://testingbot.com/real-device-testing) for the full real-hardware overview.
How do I upload my app binary to TestingBot?
Upload your `.ipa`, `.apk` or Maestro flow ZIP to [TestingBot Storage](https://testingbot.com/support/api/storage#upload) with your API key and secret. The response returns a `tb://` URL that you reference in your test capabilities. Apps are retained for 60 days and can be deleted via the REST API at any time.
Can I run mobile app tests in parallel?
Yes. Each plan includes a number of concurrent device sessions. Most test runners (Appium, XCUITest, Espresso, Maestro) launch parallel sessions natively, and each session runs on a separate, isolated device. See [our pricing plans](https://testingbot.com/pricing) for the device-concurrency limits.
What is the difference between Appium, XCUITest, Espresso and Maestro?
Appium is a cross-platform W3C WebDriver implementation for iOS and Android, similar to Selenium. XCUITest is Apple's native iOS UI testing framework. Espresso is Google's native Android UI testing framework. Maestro is a YAML-based declarative test runner for both platforms. Appium gives the broadest reuse for teams that need one codebase across iOS and Android. XCUITest and Espresso are tightly coupled to their respective platforms and typically run faster. Maestro is the simplest to write and read. See the [framework comparison](https://testingbot.com#frameworks) on this page for details.
Can I test my app on a real device behind a firewall or staging environment?
Yes. Use the [TestingBot Tunnel](https://testingbot.com/support/tunnel) to expose internal services to the device under test. Start the tunnel with an identifier, then pass tunnelIdentifier in your tb:options capabilities. The tunnel encrypts traffic between your network and the cloud device.
How are mobile app tests billed on TestingBot?
Mobile app tests are billed by the minute (the time the device was allocated to your test) and capped by your plan's concurrency limit. Simulator, emulator and real-device sessions all consume the same minutes at the same rate. See [our pricing plans](https://testingbot.com/pricing) for current rates.
## Next steps
- [ **Supported devices**
Real iPhones, iPads and Android devices on the grid](https://testingbot.com/support/app-automate/devices)
- [ **Appium guide**
Cross-platform iOS and Android automation](https://testingbot.com/support/app-automate/appium)
- [ **XCUITest guide**
Native iOS UI testing](https://testingbot.com/support/app-automate/xcuitest)
- [ **Espresso guide**
Native Android UI testing](https://testingbot.com/support/app-automate/espresso)
- [ **Maestro guide**
YAML-based declarative flows](https://testingbot.com/support/app-automate/maestro)
- [ **Mobilewright guide**
Playwright-style tests in TypeScript](https://testingbot.com/support/app-automate/mobilewright)
- [ **Real device testing**
How the real-hardware cloud works](https://testingbot.com/real-device-testing)
- [ **TestingBot Tunnel**
Test against internal or staging APIs](https://testingbot.com/support/tunnel)
- [ **Pricing**
Real-device plans and concurrency](https://testingbot.com/pricing)
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/app-automate/devices
- [Desktop Browsers & Emulators](https://testingbot.com/support/web-automate/browsers)
- [Physical Mobile Devices](https://testingbot.com#)
# Physical Mobile Devices
Run Appium, Espresso and XCUITest automation on our farm of real Android and iOS devices. Pick a device to see the exact capabilities for your test framework.
Platform
- All Platforms
- Apple iOS
- Android
Android Version
- All Versions
- Android 17.0
- Android 16.0
- Android 15.0
- Android 14.0
- Android 13.0
- Android 12.0
- Android 11.0
- Android 10.0
- Android 9.0
- Android 7.1
- Android 6.0
iOS Version
- All Versions
- iOS 27.0
- iOS 26.6
- iOS 26.5
- iOS 26.4
- iOS 26.2
- iOS 26.0
- iOS 18.7
- iOS 18.5
- iOS 18.0
- iOS 17.6
- iOS 17.5
- iOS 16.7
- iOS 16.5
- iOS 15.8
- iOS 15.0
- iOS 14.8
- iOS 14.7
- iOS 14.5
- iOS 14.0
- iOS 13.5
- iOS 12.2
Resolution
- All Resolutions
- 1440x3200
- 1440x3040
- 1440x2960
- 1320x2868
- 1600x2560
- 1260x2736
- 1206x2622
- 1179x2556
- 1170x2532
- 1080x2424
- 1080x2400
- 1080x2340
- 1080x1920
- 828x1792
- 720x1600
- 750x1334
- 810x1080
- 768x1024
- 
Galaxy S8
Android 9.0 1440x2960
- **CPU::** Octa-core (4x2.3 GHz Mongoose M2 & 4x1.7 GHz Cortex-A53)
- **Model::** SM-G950F
Capabilities
- 
iPhone 6s
iOS 15.8 750x1334
- **CPU::** Apple A9
- **Model::** MKQN2ZD/A
Capabilities
- 
iPad (6th generation)
iOS 13.5 768x1024
- **CPU::** Apple A10
- **Model::** MR7G2NF/A
Capabilities
- 
iPhone XR
iOS 18.7 828x1792
- **CPU::** Apple A12 Bionic
- **Model::** MRY42B/A
Capabilities
- 
Pixel
Android 7.1 1080x1920
- **CPU::** ARM - Quad-core - 1593Mhz
- **Model::** PQ1A.190105.004
Capabilities
- 
Galaxy S10
Android 10.0 1440x3040
- **CPU::** Octa-core
- **Model::** SM-G973F
Capabilities
- 
Galaxy Tab S11
Android 16.0 1600x2560
- **CPU::** MediaTek Dimensity 9400+
- **Model::** SM-X730
Capabilities
- 
iPhone 8
iOS 16.7 750x1334
- **CPU::** Apple A11 Bionic
- **Model::** MQ6H2ZD/A
Capabilities
- 
iPhone 11
iOS 14.0 828x1792
- **CPU::**
- **Model::** MWLT2
Capabilities
- 
Galaxy S20
Android 10.0 1440x3200
- **CPU::** Exynos 990
- **Model::** SM-G980F
Capabilities
- 
iPhone 12
iOS 14.5 1170x2532
- **CPU::** Apple A14 Bionic
- **Model::** MGJ83ZD/A
Capabilities
- 
Galaxy S21
Android 11.0 1080x2400
- **CPU::** Exynos 2100
- **Model::** SM-G991B
Capabilities
- 
Mi 10
Android 12.0 1080x2400
- **CPU::** Qualcomm Snapdragon 865
- **Model::** M2007J3SY
Capabilities
- 
Galaxy A12
Android 11.0 720x1600
- **CPU::**
- **Model::** SM-A125F
Capabilities
- 
iPad (8th generation)
iOS 15.0 810x1080
- **CPU::** Apple A12 Bionic
- **Model::** MYL92NF/A
Capabilities
- 
iPhone XR
iOS 14.8 828x1792
- **CPU::** Apple A12 Bionic
- **Model::** MRY42ZD/A
Capabilities
- 
iPhone 13
iOS 18.5 1170x2532
- **CPU::** Apple A15 Bionic
- **Model::** MLPF3ZD/A
Capabilities
- 
Galaxy S23
Android 13.0 1080x2340
- **CPU::** ARM
- **Model::** SM-S911B
Capabilities
- 
iPhone 14
iOS 17.6 1170x2532
- **CPU::** Apple A15 Bionic
- **Model::** MPUF3ZD/A
Capabilities
- 
iPhone 11
iOS 14.8 828x1792
- **CPU::**
- **Model::** MWLT2
Capabilities
- 
iPad (8th generation)
iOS 14.7 810x1080
- **CPU::** Apple A12 Bionic
- **Model::** MYLA2NF/A
Capabilities
- 
iPhone 15
iOS 18.0 1179x2556
- **CPU::** Apple A16 Bionic
- **Model::** MTP03ZD/A
Capabilities
- 
Pixel 8
Android 14.0 1080x2400
- **CPU::** Google Tensor G3 (4 nm)
- **Model::** GKWS6
Capabilities
- 
Galaxy S24
Android 14.0 1080x2340
- **CPU::** ARM
- **Model::** SM-S921B
Capabilities
- 
Galaxy A55
Android 14.0 1080x2340
- **CPU::** Exynos 1480
- **Model::** SM-A556B
Capabilities
- 
iPhone SE 2022
iOS 17.5 750x1334
- **CPU::** A15 Bionic Chip
- **Model::** MMX53LL/A
Capabilities
- 
Redmi Note 13
Android 14.0 1080x2400
- **CPU::** Octa-core (2x2.4 GHz Cortex-A76)
- **Model::** 23124RA7EO
Capabilities
- 
iPhone 16
iOS 26.5 1179x2556
- **CPU::** Apple A18
- **Model::** MYE73ZD/A
Capabilities
- 
Galaxy S25
Android 15.0 1080x2340
- **CPU::** ARM
- **Model::** SM-S931B
Capabilities
- 
Pixel 9
Android 16.0 1080x2424
- **CPU::** Google Tensor G4
- **Model::** GUR25
Capabilities
- 
iPhone 17
iOS 26.0 1179x2556
- **CPU::** Apple A19
- **Model::** MG6N4ZD/A
Capabilities
- 
Pixel 10
Android 16.0 1080x2424
- **CPU::** Google Tensor G5
- **Model::** GUR25
Capabilities
- 
iPhone Air
iOS 26.2 1260x2736
- **CPU::** Apple A19 Pro
- **Model::** MG2L4ZD/A
Capabilities
- 
Huawei P40
Android 12.0 1080x2340
- **CPU::** Cortex-A76
- **Model::** ANA-AN00
Capabilities
- 
Galaxy S26
Android 16.0 1080x2340
- **CPU::** ARM
- **Model::** SM-S942B
Capabilities
- 
iPad (9th generation)
iOS 26.6 810x1080
- **CPU::** Apple A13 Bionic
- **Model::** MK2K3NF/A
Capabilities
- 
iPhone 16 Pro
iOS 26.4 1206x2622
- **CPU::** Apple A18 Pro
- **Model::** MYND3QN/A
Capabilities
- eSIM
iPhone 17 Pro Max
iOS 26.6 1320x2868
- **CPU::** Apple A19 Pro
- **Model::** MFYM4ZD/A
Capabilities
- 
iPhone 17 Pro
iOS 26.5 1206x2622
- **CPU::** Apple A19 Pro
- **Model::** MG8G4ZD/A
Capabilities
- 
Pixel 11
Android 17.0 1080x2424
- **CPU::** Google Tensor G6
- **Model::** GUJ0N
Capabilities
- 
Pixel 8
Android 17.0 1080x2400
- **CPU::** Google Tensor G3 (4 nm)
- **Model::** GKWS6
Capabilities
- 
iPhone 13 Pro
iOS 27.0 1170x2532
- **CPU::** Apple A15 Bionic
- **Model::** MLVP3ZD/A
Capabilities
No devices match your filters
Try removing a filter or clearing your search.
## Capabilities
Use these Appium capabilities to run your test on this device.
---
URL: https://testingbot.com/support/app-automate/electron
# Electron Testing
[Electron](https://electronjs.org/) is a framework that allows you to create cross-platform native applications by using Javascript code. Electron will embed a Chromium engine in a native app format, allowing web developers to create desktop applications by writing HTML and Javascript code.
TestingBot supports running automated WebDriver tests against Electron-based apps on these platforms. Prefer Playwright? See [Electron testing with Playwright](https://testingbot.com/support/app-automate/electron/playwright).
- Windows 11
- Windows 10
- macOS Tahoe
- macOS Sequoia
- macOS Sonoma
- macOS Ventura
- macOS Monterey
All Electron versions from version 16 to the most recent version are supported.
TestingBot has created a [demo Electron app](https://github.com/testingbot/testingbot-electron-demo-app), which acts as a simple calculator, to showcase automated Electron app testing.
## Run your first Electron test
### App management
To test an Electron app, you'll first need to upload a zip file of your Electron application to a URL where TestingBot can download it from. You can choose to upload this yourself, for example to AWS S3 or use [TestingBot Storage](https://testingbot.com/support/api/storage#upload) to upload your `.zip` file.
Note that files uploaded to TestingBot Storage are automatically pruned after 60 days.
[cURL](https://testingbot.com#) [Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#)
```bash
$ curl -X POST "https://api.testingbot.com/v1/storage" \
-u key:secret -F "file=@/path/to/app/file/electron-app.zip"
```
```ruby
require 'testingbot'
api = TestingBot::Api.new("key", "secret")
api.upload_local_file('/path/to/electron-app.zip')
```
```java
TestingbotREST restApi = new TestingbotREST("key", "secret");
TestingbotStorageUploadResponse uploadResponse = restApi.uploadToStorage(new File("/path/to/electron-app.zip"));
```
```php
$api = new TestingBot\TestingBotAPI("key", "secret");
$api->uploadLocalFileToStorage("/path/to/electron-app.zip");
```
```python
import testingbotclient
tb = testingbotclient.TestingBotClient("key", "secret")
tb.storage.upload_local_file("/path/to/electron-app.zip")
```
```javascript
const TestingBot = require('testingbot-api');
const api = new TestingBot({
api_key: "your-tb-key",
api_secret: "your-tb-secret"
});
api.uploadFile('/path/to/electron-app.zip', function(error, response) {
// response.app_url
});
```
TestingBot Storage will return a `app_url` which you can use in the `tb:app` capability of your test.
This is how the capabilities in your test should currently look like:
[Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#)
```ruby
caps = {
"browserName" => "electron",
"browserVersion" => "42",
"tb:app" => "tb://the-app-url-you-received" # or https://s3.amazonaws.com/...
}
```
```java
ChromeOptions options = new ChromeOptions();
options.setCapability("browserName", "electron");
options.setCapability("browserVersion", "42");
options.setCapability("tb:app", "tb://the-app-url-you-received");
Map tbOptions = new HashMap<>();
tbOptions.put("key", "api_key");
tbOptions.put("secret", "api_secret");
options.setCapability("tb:options", tbOptions);
```
```python
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
options.set_capability("browserName", "electron")
options.set_capability("browserVersion", "42")
options.set_capability("tb:app", "tb://the-app-url-you-received")
options.set_capability("tb:options", {
"name": "Electron Sample"
})
driver = webdriver.Remote("https://hub.testingbot.com/wd/hub", options=options)
```
```javascript
driver = await new webdriver.Builder().withCapabilities({
"browserName": 'electron',
"browserVersion": "42",
"tb:options": {
"key": "api_key",
"secret": "api_secret"
},
'tb:app': 'tb://the-app-url-you-received' // or https://s3.amazonaws.com/...
}).usingServer("https://hub.testingbot.com/wd/hub").build();
```
```csharp
var chromeOptions = new ChromeOptions();
chromeOptions.BrowserName = "electron";
chromeOptions.BrowserVersion = "42";
chromeOptions.PlatformName = "Windows 10";
var tbOptions = new Dictionary
{
["key"] = "api_key",
["secret"] = "api_secret",
["name"] = TestContext.CurrentContext.Test.Name
};
// Use AddAdditionalOption (Selenium 4) so these vendor caps land at the top level,
// not nested under goog:chromeOptions like AddAdditionalChromeOption would.
chromeOptions.AddAdditionalOption("tb:options", tbOptions);
chromeOptions.AddAdditionalOption("tb:app", "tb://the-app-url-you-received");
driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"), chromeOptions);
```
### Binary location
Next to the required `app` capability, you will also need to specify a `tb:binary_location` capability. This is the path to the actual Electron app, so that TestingBot knows which file in the `zip` file is the Electron app.
For example, if your uploaded zip file is structured like this:
```yaml
TestingBot-Electron-App.zip
[testingbot-electron-demo-app.app]
[Contents]
[MacOS]
-- testingbot-electron-demo-app
```
The `binary_location` capability in this case would be:
```bash
tb:binary_location = "testingbot-electron-demo-app.app/Contents/MacOS/testingbot-electron-demo-app"
```
Now the capabilities in your test should look like the example below.
[Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#)
```ruby
require 'selenium-webdriver'
options = Selenium::WebDriver::Options.chrome
options.browser_name = "electron"
options.platform_name = "TAHOE"
options.browser_version = "31"
options.add_option("tb:binary_location", "testingbot-electron-demo-app.app/Contents/MacOS/testingbot-electron-demo-app")
options.add_option("tb:app", "https://github.com/testingbot/testingbot-electron-demo-app/releases/download/v1.0.0/testingbot-electron-demo-app-darwin-arm64-1.0.0.zip")
http_client = Selenium::WebDriver::Remote::Http::Default.new
http_client.open_timeout = 200
http_client.read_timeout = 200
driver = Selenium::WebDriver.for(
:remote,
url: "https://key:secret@hub.testingbot.com/wd/hub",
options: options,
http_client: http_client)
# Generate two random numbers between 0 and 9
num1 = rand(0..9)
num2 = rand(0..9)
# Find elements on the page
first_number = driver.find_element(id: "btn-#{num1}")
second_number = driver.find_element(id: "btn-#{num2}")
plus = driver.find_element(id: 'btn-plus')
equal = driver.find_element(id: 'btn-equal')
# Perform operations
first_number.click
plus.click
second_number.click
equal.click
# Get the result
result_element = driver.find_element(id: 'calc-display')
result_text = result_element.text
driver.quit
raise "Test failed" unless (num1 + num2) == result_text.to_i
```
```java
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeOptions;
import org.openqa.selenium.remote.RemoteWebDriver;
import java.net.URL;
import java.util.Random;
public class SeleniumTest {
public static void main(String[] args) {
try {
ChromeOptions options = new ChromeOptions();
options.setCapability("browserName", "electron");
options.setCapability("platformName", "TAHOE");
options.setCapability("browserVersion", "31");
options.setCapability("tb:binary_location", "testingbot-electron-demo-app.app/Contents/MacOS/testingbot-electron-demo-app");
options.setCapability("tb:app", "https://github.com/testingbot/testingbot-electron-demo-app/releases/download/v1.0.0/testingbot-electron-demo-app-darwin-arm64-1.0.0.zip");
WebDriver driver = new RemoteWebDriver(
new URL("https://key:secret@hub.testingbot.com/wd/hub"),
options
);
// Generate two random numbers between 0 and 9
Random random = new Random();
int num1 = random.nextInt(10);
int num2 = random.nextInt(10);
WebElement firstNumber = driver.findElement(By.id("btn-" + num1));
WebElement secondNumber = driver.findElement(By.id("btn-" + num2));
WebElement plus = driver.findElement(By.id("btn-plus"));
WebElement equal = driver.findElement(By.id("btn-equal"));
firstNumber.click();
plus.click();
secondNumber.click();
equal.click();
WebElement resultElement = driver.findElement(By.id("calc-display"));
String resultText = resultElement.getText();
// Verify the result
if ((num1 + num2) != Integer.parseInt(resultText)) {
throw new AssertionError("Test failed");
}
// Quit the driver
driver.quit();
} catch (Exception e) {
e.printStackTrace();
}
}
}
```
```python
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.options import Options
import random
# Set capabilities
options = Options()
options.set_capability("browserName", "electron")
options.set_capability("platformName", "TAHOE")
options.set_capability("browserVersion", "31")
options.set_capability("tb:binary_location", "testingbot-electron-demo-app.app/Contents/MacOS/testingbot-electron-demo-app")
options.set_capability("tb:app", "https://github.com/testingbot/testingbot-electron-demo-app/releases/download/v1.0.0/testingbot-electron-demo-app-darwin-arm64-1.0.0.zip")
# Initialize the WebDriver connection to TestingBot
driver = webdriver.Remote(
command_executor="https://key:secret@hub.testingbot.com/wd/hub",
options=options
)
try:
# Generate two random numbers between 0 and 9
num1 = random.randint(0, 9)
num2 = random.randint(0, 9)
# Find elements on the page
first_number = driver.find_element(By.ID, f"btn-{num1}")
second_number = driver.find_element(By.ID, f"btn-{num2}")
plus = driver.find_element(By.ID, "btn-plus")
equal = driver.find_element(By.ID, "btn-equal")
# Perform operations
first_number.click()
plus.click()
second_number.click()
equal.click()
# Get the result
result_element = driver.find_element(By.ID, "calc-display")
result_text = result_element.text
# Verify the result
assert (num1 + num2) == int(result_text), "Test failed"
finally:
driver.quit()
```
```javascript
const { Builder, By } = require('selenium-webdriver');
const { Options } = require('selenium-webdriver/chrome');
(async function example() {
let driver;
try {
// Set capabilities
let caps = {
browserName: 'electron',
platformName: 'TAHOE',
browserVersion: '31',
'tb:binary_location': 'testingbot-electron-demo-app.app/Contents/MacOS/testingbot-electron-demo-app',
'tb:app': 'https://github.com/testingbot/testingbot-electron-demo-app/releases/download/v1.0.0/testingbot-electron-demo-app-darwin-arm64-1.0.0.zip'
};
driver = await new Builder()
.usingServer('https://key:secret@hub.testingbot.com/wd/hub')
.withCapabilities(caps)
.build();
// Generate two random numbers between 0 and 9
const num1 = Math.floor(Math.random() * 10);
const num2 = Math.floor(Math.random() * 10);
const firstNumber = await driver.findElement(By.id(`btn-${num1}`));
const secondNumber = await driver.findElement(By.id(`btn-${num2}`));
const plus = await driver.findElement(By.id('btn-plus'));
const equal = await driver.findElement(By.id('btn-equal'));
// Perform operations
await firstNumber.click();
await plus.click();
await secondNumber.click();
await equal.click();
// Get the result
const resultElement = await driver.findElement(By.id('calc-display'));
const resultText = await resultElement.getText();
// Verify the result
if ((num1 + num2) !== parseInt(resultText)) {
throw new Error('Test failed');
}
} catch (error) {
console.error('Error during test execution:', error);
} finally {
if (driver) {
await driver.quit();
}
}
})();
```
```csharp
using OpenQA.Selenium;
using OpenQA.Selenium.Chrome;
using OpenQA.Selenium.Remote;
using System;
class SeleniumTest
{
static void Main(string[] args)
{
try
{
// Set capabilities
var options = new ChromeOptions();
options.BrowserName = "electron";
options.PlatformName = "TAHOE";
options.BrowserVersion = "31";
// Use AddAdditionalOption (Selenium 4) so these vendor caps land at the top level,
// not nested under goog:chromeOptions like AddAdditionalChromeOption would.
options.AddAdditionalOption("tb:binary_location", "testingbot-electron-demo-app.app/Contents/MacOS/testingbot-electron-demo-app");
options.AddAdditionalOption("tb:app", "https://github.com/testingbot/testingbot-electron-demo-app/releases/download/v1.0.0/testingbot-electron-demo-app-darwin-arm64-1.0.0.zip");
IWebDriver driver = new RemoteWebDriver(
new Uri("https://key:secret@hub.testingbot.com/wd/hub"),
options
);
// Generate two random numbers between 0 and 9
Random random = new Random();
int num1 = random.Next(0, 10);
int num2 = random.Next(0, 10);
IWebElement firstNumber = driver.FindElement(By.Id($"btn-{num1}"));
IWebElement secondNumber = driver.FindElement(By.Id($"btn-{num2}"));
IWebElement plus = driver.FindElement(By.Id("btn-plus"));
IWebElement equal = driver.FindElement(By.Id("btn-equal"));
// Perform operations
firstNumber.Click();
plus.Click();
secondNumber.Click();
equal.Click();
// Get the result
IWebElement resultElement = driver.FindElement(By.Id("calc-display"));
string resultText = resultElement.Text;
// Verify the result
if ((num1 + num2) != int.Parse(resultText))
{
throw new Exception("Test failed");
}
// Quit the driver
driver.Quit();
}
catch (Exception ex)
{
Console.WriteLine($"Error during test execution: {ex.Message}");
}
}
}
```
The example above will run the TestingBot Demo Electron app on a remote macOS machine. You can also choose to run the demo app on a remote Windows machine inside TestingBot's grid, please see the capabilities below.
```ruby
options = Selenium::WebDriver::Options.chrome
options.browser_name = "electron"
options.platform_name = "WIN10"
options.browser_version = "31"
options.add_option("tb:binary_location", "testingbot-electron-demo-app.exe")
options.add_option("tb:app", "https://github.com/testingbot/testingbot-electron-demo-app/releases/download/v1.0.0/testingbot-electron-demo-app-win32-x64-1.0.0.zip")
```
## Configuring Electron tests
To configure your Electron test with TestingBot, you need to supply 3 capabilities:
- [app capability](https://testingbot.com#capability-app)
- [binary\_location capability](https://testingbot.com#capability-binary)
- Electron version capability
Please use the `version` or `browserVersion` capability to indicate to TestingBot which version of Electron was used to create the final app that you are testing.
TestingBot needs to know which version was used, so that we can map it to the Chromium version that is being used for that particular Electron version. For example, if you created your Electron app with Electron 42, you would need to provide `browserVersion: 42` as a capability.
## Electron Test Results
Test results for each Electron test will be available in the TestingBot member dashboard. For each Electron test, you will get access to a video recording, generated log files and additional meta-data.
These results are also available through the TestingBot [REST-API](https://testingbot.com/support/api) or by using one of the [TestingBot CI/CD plugins](https://testingbot.com/support/integrations/ci-cd).
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/app-automate/electron/playwright
# Electron App Testing with Playwright
[Electron](https://electronjs.org/) apps embed a Chromium engine, which means [Playwright](https://playwright.dev/) can automate them through the Chrome DevTools Protocol (CDP). TestingBot starts your Electron app on a remote Windows or macOS machine and gives you a WebSocket endpoint that Playwright connects to with `chromium.connectOverCDP()`.
Your Playwright test code runs on your own machine or CI server, while the Electron app itself runs in the TestingBot cloud, on these platforms:
- Windows 11
- Windows 10
- macOS Tahoe
- macOS Sequoia
- macOS Sonoma
- macOS Ventura
- macOS Monterey
All Electron versions from version 16 to the most recent version are supported. If you prefer WebDriver (Selenium) over Playwright, please see [Electron testing with WebDriver](https://testingbot.com/support/app-automate/electron).
TestingBot has created a [demo Electron app](https://github.com/testingbot/testingbot-electron-demo-app), which acts as a simple calculator, to showcase automated Electron app testing. The examples below run against this demo app.
## Prepare your Electron app
TestingBot needs to be able to download your Electron app as a `zip` file. You can host this zip file yourself, for example on AWS S3 or GitHub Releases, or upload it to [TestingBot Storage](https://testingbot.com/support/api/storage#upload), which returns a `tb://` URL you can use instead.
Next to the URL of the zip file, you also need to know the path of the actual Electron binary inside the zip file. This is the `binary_location` parameter, so TestingBot knows which file to launch. For the demo app on macOS this is:
```bash
testingbot-electron-demo-app.app/Contents/MacOS/testingbot-electron-demo-app
```
On Windows, it is simply the executable in the zip file:
```bash
testingbot-electron-demo-app.exe
```
See [App management](https://testingbot.com/support/app-automate/electron#capability-app) on the WebDriver Electron page for more details about uploading your app.
## Run your first test
Install `playwright-core`, connect with `chromium.connectOverCDP()` and pass the Electron parameters in the WebSocket URL. Once connected, your Electron app's window is available as the first page of the first browser context.
The `browserVersion` parameter must be the major Electron version your app was built with. The demo app was built with Electron 31, so the examples below use `browserVersion=31`.
```bash
npm i playwright-core
```
[macOS](https://testingbot.com#) [Windows](https://testingbot.com#)
```javascript
const { chromium } = require('playwright-core');
const app = 'https://github.com/testingbot/testingbot-electron-demo-app/releases/download/v1.0.0/testingbot-electron-demo-app-darwin-arm64-1.0.0.zip';
const binaryLocation = 'testingbot-electron-demo-app.app/Contents/MacOS/testingbot-electron-demo-app';
const wsEndpoint = 'wss://cloud.testingbot.com'
+ '?key=api_key&secret=api_secret'
+ '&browserName=electron'
+ '&browserVersion=31'
+ '&platform=TAHOE'
+ `&app=${encodeURIComponent(app)}`
+ `&binary_location=${encodeURIComponent(binaryLocation)}`
+ `&name=${encodeURIComponent('My First Electron Test')}`;
(async () => {
// Starting the remote machine and downloading your app can take a while
const browser = await chromium.connectOverCDP(wsEndpoint, { timeout: 300000 });
const context = browser.contexts()[0];
const page = context.pages()[0]; // the Electron app window
// Calculate 3 + 4 in the demo calculator app
await page.click('#btn-3');
await page.click('#btn-plus');
await page.click('#btn-4');
await page.click('#btn-equal');
const result = await page.textContent('#calc-display');
console.log(`3 + 4 = ${result}`);
await browser.close();
})();
```
```javascript
const { chromium } = require('playwright-core');
const app = 'https://github.com/testingbot/testingbot-electron-demo-app/releases/download/v1.0.0/testingbot-electron-demo-app-win32-x64-1.0.0.zip';
const binaryLocation = 'testingbot-electron-demo-app.exe';
const wsEndpoint = 'wss://cloud.testingbot.com'
+ '?key=api_key&secret=api_secret'
+ '&browserName=electron'
+ '&browserVersion=31'
+ '&platform=WIN10'
+ `&app=${encodeURIComponent(app)}`
+ `&binary_location=${encodeURIComponent(binaryLocation)}`
+ `&name=${encodeURIComponent('My First Electron Test')}`;
(async () => {
// Starting the remote machine and downloading your app can take a while
const browser = await chromium.connectOverCDP(wsEndpoint, { timeout: 300000 });
const context = browser.contexts()[0];
const page = context.pages()[0]; // the Electron app window
// Calculate 3 + 4 in the demo calculator app
await page.click('#btn-3');
await page.click('#btn-plus');
await page.click('#btn-4');
await page.click('#btn-equal');
const result = await page.textContent('#calc-display');
console.log(`3 + 4 = ${result}`);
await browser.close();
})();
```
Make sure to use `chromium.connectOverCDP()` and the `wss://cloud.testingbot.com` endpoint for Electron tests. The `wss://cloud.testingbot.com/playwright` endpoint used for [Playwright browser testing](https://testingbot.com/support/web-automate/playwright) does not support Electron apps.
## Playwright Test
You can use the same connection with [Playwright Test](https://testingbot.com/support/web-automate/playwright/playwright-test), Playwright's own test runner. Connect in a `beforeAll` hook or fixture and run your suite with `npx playwright test`.
```javascript
const { test, expect, chromium } = require('@playwright/test');
const app = 'https://github.com/testingbot/testingbot-electron-demo-app/releases/download/v1.0.0/testingbot-electron-demo-app-darwin-arm64-1.0.0.zip';
const binaryLocation = 'testingbot-electron-demo-app.app/Contents/MacOS/testingbot-electron-demo-app';
const wsEndpoint = 'wss://cloud.testingbot.com'
+ '?key=api_key&secret=api_secret'
+ '&browserName=electron'
+ '&browserVersion=31'
+ '&platform=TAHOE'
+ `&app=${encodeURIComponent(app)}`
+ `&binary_location=${encodeURIComponent(binaryLocation)}`
+ `&name=${encodeURIComponent('Electron Playwright Test')}`;
test('calculator adds two numbers', async () => {
test.setTimeout(600000); // remote machine startup can take a few minutes
const browser = await chromium.connectOverCDP(wsEndpoint, { timeout: 300000 });
const page = browser.contexts()[0].pages()[0];
await page.click('#btn-3');
await page.click('#btn-plus');
await page.click('#btn-4');
await page.click('#btn-equal');
await expect(page.locator('#calc-display')).toHaveText('7');
await browser.close();
});
```
## Python
Playwright for Python offers the same `connect_over_cdp()` API, so you can test your Electron app from Python as well.
```bash
pip install playwright
```
```python
from urllib.parse import quote
from playwright.sync_api import sync_playwright
app = "https://github.com/testingbot/testingbot-electron-demo-app/releases/download/v1.0.0/testingbot-electron-demo-app-darwin-arm64-1.0.0.zip"
binary_location = "testingbot-electron-demo-app.app/Contents/MacOS/testingbot-electron-demo-app"
ws_endpoint = (
"wss://cloud.testingbot.com"
"?key=api_key&secret=api_secret"
"&browserName=electron"
"&browserVersion=31"
"&platform=TAHOE"
f"&app={quote(app, safe='')}"
f"&binary_location={quote(binary_location, safe='')}"
"&name=Electron%20Playwright%20Python"
)
with sync_playwright() as p:
# Starting the remote machine and downloading your app can take a while
browser = p.chromium.connect_over_cdp(ws_endpoint, timeout=300000)
page = browser.contexts[0].pages[0] # the Electron app window
page.click("#btn-3")
page.click("#btn-plus")
page.click("#btn-4")
page.click("#btn-equal")
result = page.text_content("#calc-display")
print(f"3 + 4 = {result}")
browser.close()
```
## Connection parameters
All parameters are passed as query parameters in the WebSocket URL. Make sure to URL-encode the values, especially the `app` URL and `binary_location` path.
Parameter | Description || `key` and `secret` | Your TestingBot [API credentials](https://testingbot.com/members/user/edit). |
| `browserName` | Must be `electron`. |
| `browserVersion` | The major Electron version your app was built with, for example `42`. TestingBot uses this to match the correct Chromium version. |
| `platform` | The operating system to run your app on, for example `TAHOE`, `SEQUOIA`, `WIN10` or `WIN11`. |
| `app` | URL of the zip file containing your Electron app, or a `tb://` URL from [TestingBot Storage](https://testingbot.com/support/api/storage#upload). |
| `binary_location` | Path of the Electron binary inside the zip file. |
| `name` | Optional name for the test, shown in the [TestingBot dashboard](https://testingbot.com/members). |
| `build` | Optional build identifier to group multiple tests together. |
## Mark tests as passed or failed
Playwright does not report test success or failure to TestingBot by itself. You can use the `testingbot_executor` custom command from within your test to mark the session as passed or failed in the TestingBot dashboard.
```javascript
try {
await expect(page.locator('#calc-display')).toHaveText('7');
// Mark the test as passed on TestingBot
await page.evaluate(_ => {}, `testingbot_executor: ${JSON.stringify({
action: 'setSessionStatus',
arguments: { passed: true, reason: 'Calculator sum matched' }
})}`);
} catch (err) {
// Mark the test as failed on TestingBot
await page.evaluate(_ => {}, `testingbot_executor: ${JSON.stringify({
action: 'setSessionStatus',
arguments: { passed: false, reason: 'Calculator sum did not match' }
})}`);
throw err;
}
```
## Supported Playwright features
Playwright talks to your Electron app through the Chrome DevTools Protocol, which supports everything that operates on browser contexts and pages:
- Locators, clicking, typing and all other page interactions
- Assertions with `expect()`
- Screenshots
- Evaluating JavaScript in the renderer process with `page.evaluate()`
- Network inspection of requests made by your app's windows
Because your app runs on a remote machine, the Electron-specific APIs from Playwright's local `_electron.launch()` mode are not available. This means no `electronApp.evaluate()` access to the Electron main process, such as `app` or `BrowserWindow` modules. If you need to assert on main process behavior, expose that state to the renderer process, for example via IPC, and read it with `page.evaluate()`.
## Test Results
Every Electron Playwright test appears in the TestingBot member dashboard. For each test you get access to a video recording, log files and additional meta-data.
These results are also available through the TestingBot [REST-API](https://testingbot.com/support/api) or by using one of the [TestingBot CI/CD plugins](https://testingbot.com/support/integrations/ci-cd).
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/app-automate/earlgrey
# EarlGrey Testing
TestingBot will release testing with EarlGrey in Q2 of 2026.
Run your EarlGrey tests on physical Android devices in the cloud, on Samsung Galaxy devices, Pixels, Xiaomi and other Android devices.
If you are interested in participating in our beta program, please contact us.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/app-automate/maestro
# Maestro Cloud Testing
TestingBot provides a grid of real iOS and Android devices, plus iOS simulators and Android emulators, ready to run your Maestro flows. Connect your existing Maestro YAML flows to the TestingBot cloud and run them in parallel from your machine or your CI pipeline.
Maestro is a mobile UI testing framework, which supports testing on both iOS and Android. You write flows, which are story-like tests, where you describe a specific flow in your mobile application. For example, you could have a login flow, or a checkout flow.
Maestro is a relatively new test framework and offers several nice features, such as:
- Built-in tolerance to flakiness: various mechanisms are added to try and prevent flaky tests, for example locators that have changed or moved positions.
- Automatic waits: Maestro will automatically wait for certain conditions, which means you do not have to worry about delays, or adding sleep commands to your tests.
- Declarative syntax, which is defined in Flow (`yaml`) files.
## Why TestingBot Cloud?
TestingBot has been offering cloud-based testing since 2012 and has built a robust solution for Maestro cloud testing. Compared to other vendors and Maestro cloud, we offer the following improved features:
- Run on any iOS version, both physical iOS devices and iOS simulators.
- Test on **physical Android devices** , not just emulators (though that's also certainly possible)
- No execution limit: your tests can run as long as they need
- Apply various [options](https://testingbot.com/support/app-automate/maestro/options) such as setting orientation, geolocation options and more
**New to TestingBot?** Maestro users get a 30-day free trial. [Start your trial](https://testingbot.com/users/sign_up) or see the [Maestro feature overview](https://testingbot.com/features/automation/maestro).
## TestingBot CLI
The easiest way to run Maestro tests on TestingBot is using our CLI tool. It handles uploading your app, flows and running tests in a single command.
https://www.youtube.com/embed/MMWCkBgXFOU
### Installation
```bash
npm install -g @testingbot/cli
```
### Authentication
Authenticate using this command:
```bash
testingbot login
```
This opens your browser for authentication. After logging in, your credentials are saved to `~/.testingbot` and you can start using the CLI.
#### Other Authentication Methods
- **Command-line options** : `--api-key` and `--api-secret`
- **Environment variables** : `TB_KEY` and `TB_SECRET`
- **Config file** : Create `~/.testingbot` with content `key:secret`
### Quick Start
Run Maestro tests with a single command:
```bash
testingbot maestro app.apk flow.yml --device "Pixel 8" --deviceVersion "14"
```
```bash
testingbot maestro app.ipa flow.yml --device "iPhone 16" --real-device
```
The CLI will:
1. Upload your app to TestingBot, from a local file or a download URL such as an EAS Build link (`--app-url`), or reuse an earlier upload with `--app-binary-id`
2. Upload and zip your flow files
3. Start the test run, on one device or a whole [device matrix](https://testingbot.com/support/app-automate/maestro/options#device-matrix)
4. Show real-time progress and results, and exit `0`, `2` (flows failed) or `1` (error)
For pipelines there is `--json` output, `--async` with `testingbot status --id --wait` and `testingbot artifacts` to come back for results, a [GitHub Action](https://testingbot.com/support/app-automate/maestro/ci-cd/github-actions), [GitHub PR status checks](https://testingbot.com/support/app-automate/maestro/ci-cd/github-pr-checks) and [Expo / EAS](https://testingbot.com/support/app-automate/maestro/ci-cd/expo-eas) support. See [CI/CD integration](https://testingbot.com/support/app-automate/maestro/ci-cd).
Coming from Maestro Cloud? The [migration guide](https://testingbot.com/support/app-automate/maestro/migrate-from-maestro-cloud) maps every `maestro cloud` flag to its TestingBot equivalent.
## Upload Mobile App
You will need to upload a mobile app to run tests with Maestro. The format of this file depends on which device type you'd like to test:
- **iOS physical devices/simulators** :
You can upload your `.ipa` or `.zip` file (for simulators, see [here how to prepare this zip file](https://testingbot.com/support/app-automate/help/prepare#simulator)).
- **Android physical devices/emulators** :
Upload the `.apk` file, which TestingBot will automatically install on the Android device.
[CLI](https://testingbot.com#) [cURL](https://testingbot.com#)
When using the CLI, the app upload is handled automatically as part of the `testingbot maestro` command. Simply specify your app file as the first argument:
```bash
testingbot maestro /path/to/app/file/sample-debug.apk ./flows
```
Supported app formats:
- **Android** : `.apk`
- **iOS Physical Device** : `.ipa`
- **iOS Simulator** : `.app`, `.zip` (zipped .app bundle)
```bash
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/maestro/app" \
-F "file=@/path/to/app/file/sample-debug.zip"
```
The API response will include an `id` response, which you can use in the other API calls.
```json
{
"id": 4012
}
```
## Upload Maestro Flows
To upload the Flows zip file, please follow the example below.
[CLI](https://testingbot.com#) [cURL](https://testingbot.com#)
When using the CLI, flow files are automatically zipped and uploaded. You can specify:
- A directory containing flow files
- Individual `.yaml` or `.yml` files
- A pre-zipped `.zip` file
- Glob patterns (e.g., `./flows/**/*.yaml`)
```bash
# Single directory
testingbot maestro app.apk ./flows
# Multiple directories
testingbot maestro app.apk ./flows/smoke ./flows/regression
# Individual flow files
testingbot maestro app.apk login.yaml checkout.yaml
# Pre-zipped flows
testingbot maestro app.apk flows.zip
```
Maestro Flow files are `.yaml` files which contain the actions that need to be performed. You can zip these together and upload to TestingBot by using the following command:
```bash
zip -r flows.zip *.yaml
```
Make sure all the necessary flows and sub flows are included in the zip file.
```bash
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/maestro/:id/tests" \
-F "file=@/path/to/app/file/flows.zip"
```
Replace the `:id` with the identifier you received during the app upload call. (In this example 4012)
Calling this multiple times with the same `:id` will overwrite the previously uploaded Flows zip file.
### Zip structure
Maestro needs to know which flows it should run. If there are flow files (`.yaml` files) in the top-level directory of the zip file, it will run these flow files.
You can also include a [config.yaml](https://testingbot.com/support/app-automate/maestro/options#config) file in the top-level directory of the zip file, to indicate which flows should be tested. For example, you can use the `**` annotation to indicate you want to run all flows, even if they are in separate sub-directories.
```yaml
flows:
- "**"
```
More information is available in the [Maestro Docs](https://docs.maestro.dev/cli/test-suites-and-reports#controlling-what-tests-to-include).
## Run Maestro tests
Once you've uploaded both your app and Flows zip file, you can run the Maestro tests on the TestingBot cloud.
Specify one or more capabilities, to indicate on which iOS simulators or Android devices you want to run your tests on.
[CLI](https://testingbot.com#) [cURL](https://testingbot.com#)
Use the CLI to run tests with device selection options:
```bash
# Basic usage with device selection
testingbot maestro app.apk flow.yml --device "Pixel 8" --deviceVersion "14"
# iOS simulator
testingbot maestro app.zip ./flows --platform iOS --device "iPhone 16" --deviceVersion "26.0"
# iOS physical device
testingbot maestro app.ipa flow.yml --device "iPhone 16" --real-device
# Android real device
testingbot maestro app.apk flow.yml --device "Galaxy S24" --real-device
# With additional options
testingbot maestro app.apk ./flows \
--device "Pixel 8" \
--deviceVersion "14" \
--name "Smoke Tests" \
--build "v2.1.0"
# With CI/CD integration (Git metadata)
testingbot maestro app.apk ./flows \
--device "Pixel 8" \
--commit-sha "$COMMIT_SHA" \
--pull-request-id "$PR_NUMBER" \
--repo-owner "myorg" \
--repo-name "myapp"
```
### Device Options
| Option | Description |
| --- | --- |
| `--device ` | Device name (e.g., "Pixel 9", "iPhone 16") |
| `--platform ` | Platform: Android or iOS |
| `--deviceVersion ` | OS version (e.g., "14", "18.0") |
| `--real-device` | Use a real device instead of emulator/simulator |
See the [Options page](https://testingbot.com/support/app-automate/maestro/options) for more configuration options.
Select a device configuration below, then use the generated cURL command:
 iOS 18.6 › iPhone 16
Loading environments...
Please wait while we load the available browsers and platforms.
Replace the `:id` with the identifier you received during the app upload call. (In this example 4012)
When running Maestro tests on physical devices, a device could be occupied at times. You might want to use regex/wildcard patterns to target a broad range of devices.
We offer special parameters which you can use to allocate a device:
Regex Input | Result || `".*Galaxy.*"` | This will allocate any of the available Galaxy devices (phone or tablet) |
| `"Pixel.*"` | This will allocate any of the available Pixel devices. |
| `".*Galaxy.*\|.*Pixel.*"` | This will allocate any available Galaxy or Pixel device. Use `\|` to combine patterns; a comma is matched literally and will find no device. |
| `"*"` | This will allocate a random available device, you can use this with the `deviceName` or `version`. |
The `version` capability accepts the same patterns, and additionally a comparison against an OS version. This is useful when you do not want to pin a run to a version that may be retired.
Version Input | Result || `"14"` | Any 14.x version. |
| `"18.6"` | Exactly this version. |
| `">=14"` | The newest available version of 14 or higher. `>`, `<`, `<=` and `=` work too. |
| `"1[34].*"` | Any 13.x or 14.x version. |
| `"*"` | Any available version. |
## View Maestro Results
The results of your Maestro tests will be available in the [TestingBot dashboard](https://testingbot.com/members).
Each test result contains a video, test logs and other meta data generated during the test run. Maestro flows will be displayed, together with the steps that were taken during each flow.
You can also use the CLI or an API call to get back the result from the run(s):
[CLI](https://testingbot.com#) [cURL](https://testingbot.com#)
By default, the CLI waits for test completion and displays real-time progress:
```bash
testingbot maestro app.apk ./flows --device "Pixel 8"
```
The CLI will show:
- Upload progress for app and flows
- Device allocation status
- Live output from Maestro flows
- Final pass/fail status
### Async Mode
Use `--async` to start tests without waiting for results, then come back for them with `status` and `artifacts`:
```bash
testingbot maestro app.apk ./flows --device "Pixel 8" --async
# Tests started in async mode. Project ID: 1234
testingbot status --id 1234 --wait
testingbot artifacts --id 1234 --report junit --report-output-dir ./reports
```
### JSON Output
Use `--json` to get every run and flow result as one JSON document on stdout, or `--json-file` to write it next to the normal output:
```bash
testingbot maestro app.apk ./flows --device "Pixel 8" --json | jq .outcome
```
### Download Artifacts
Download test artifacts (logs, screenshots, video) after completion:
```bash
testingbot maestro app.apk ./flows --device "Pixel 8" \
--download-artifacts \
--artifacts-output-dir ./artifacts
```
```bash
curl -u api_key:api_secret \
"https://api.testingbot.com/v1/app-automate/maestro/:project_id"
```
Replace the `:project_id` with the identifier you received during the app upload call. (In this example 4012)
The response will contain a `success` boolean, indicating whether the test passed or failed.
**Example response:**
```json
{"runs":[{"id":14019,"status":"DONE","capabilities":{"version":"15","deviceName":"Pixel 9","platformName":"Android"},"success":0,"created_at":"2025-08-26T10:30:42.000Z","test":{"sessionId":"0255832ee646e7620dc7d44917293ccf","environment":{"name":"android","os":"Pixel 9 - 15.0","version":"15.0"}}}],"success":false,"version":"1.0","completed":true}
```
You can poll this request to check if the tests have finished running. The `completed` boolean will be set to true when all tests have finished.
### JUnit Report
All Maestro tests on TestingBot will by default run with a JUnit reporter. This report is accessible programmatically:
[CLI](https://testingbot.com#) [cURL](https://testingbot.com#)
Download the JUnit report automatically after test completion:
```bash
# Download JUnit XML report
testingbot maestro app.apk ./flows --device "Pixel 8" \
--report junit \
--report-output-dir ./reports
```
The report will be saved to the specified directory with a filename based on the test run. `--report` also accepts `html`, `html-detailed` and `allure`.
```bash
curl -u api_key:api_secret \
"https://api.testingbot.com/v1/app-automate/maestro/:project_id/:run_id/junit_report"
```
Replace the `:project_id` with the identifier you received during the app upload call. (In this example 4012)
Replace the `:run_id` with the identifier for this run.
The response will contain a `junit_report` string value that contains the XML formatted JUnit report.
### Webhooks
Because the Maestro tests run asynchronously, you might want to get notified when a test run has finished. You can poll the [results API](https://testingbot.com#results) or use the [Webhook integration](https://testingbot.com/support/integrations/webhooks) or [Slack integration](https://testingbot.com/support/integrations/slack) to get notified when a test run has finished.
## Example Maestro Flow
Below is a simple example Flow, where we input two values in the TestingBot example app:
- [Android Sample .apk](https://testingbot.com/appium/sample.apk)
- [iOS Simulator Sample .zip](https://testingbot.com/appium/sample.zip)
- [iOS Physical Sample .ipa](https://testingbot.com/appium/sample.ipa)
```yaml
appId: org.reactjs.native.example.SampleApp
---
- launchApp
- tapOn:
id: inputA
- longPressOn:
id: inputA
- tapOn:
text: "Select All"
- eraseText
- inputText: "10"
- tapOn:
id: inputB
- longPressOn:
id: inputB
- tapOn:
text: "Select All"
- eraseText
- inputText: "5"
- tapOn:
id: sum
- assertVisible:
text: "15"
```
## Maestro Parallel Testing
The TestingBot physical and virtual device grid is optimised to run many tests concurrently. You can run multiple Maestro flows simultaneously (in parallel), which will drastically shorten your total test duration.
Depending on your subscription plan, you will be able to run 1 or more Maestro flows at the same time. If you exceed the threshold, additional Maestro flows will be queued until a slot becomes available.
By default, each flow runs in a separate session. Make sure each flow can run isolated. If you don't want this, you can use the [--shard-split 1](https://testingbot.com#shard-split) option to run all flows in the same session.
For example, let's assume you have an Automation Pro plan with 2 concurrent tests. And you want to run 10 Maestro tests, or run the same Maestro test on 10 different device configurations:
- 2 Maestro tests will run simultaneously
- The other 8 are waiting (queued) until one of the first two finishes
- When a slot frees up, the next Maestro test in the queue will start to run
- Finally, when all tests have run, you will see the results in the TestingBot dashboard
Currently there is no limit to the amount of Maestro tests you can add to the queue.
## Grouping Flows
When you have a large number of flows, you might want to split these flows across multiple test runs. This is also known as sharding. By default, each test runs separately in parallel on TestingBot.
You can group flows together that should run in the same session by using the `--shard-split` option.
[CLI](https://testingbot.com#) [cURL](https://testingbot.com#)
```bash
# Shard flows across 3 parallel sessions
testingbot maestro app.apk ./flows \
--device "Pixel 8" \
--deviceVersion "14" \
--shard-split 3
```
You can also decide to run all flows in the same session by using the `--shard-split 1` option.
```bash
# Run all flows in a single session
testingbot maestro app.apk ./flows \
--device "Pixel 8" \
--deviceVersion "14" \
--shard-split 1
```
Use the `shardSplit` parameter in your API request to control how flows are grouped.
```bash
# Shard flows across 3 parallel sessions
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/maestro/:id/run" \
-d '{"capabilities":[{"version":"14","deviceName":"Pixel 8","platformName":"Android"}], "shardSplit": 3}' \
-H "Content-Type: application/json"
```
To run all flows in a single session, set `shardSplit` to `1`:
```bash
# Run all flows in a single session
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/maestro/:id/run" \
-d '{"capabilities":[{"version":"14","deviceName":"Pixel 8","platformName":"Android"}], "shardSplit": 1}' \
-H "Content-Type: application/json"
```
Replace the `:id` with the identifier you received during the app upload call.
By default TestingBot balances the shards by expected duration, so they finish at roughly the same time instead of one shard getting all the slow flows. Durations come from your own previous runs of each flow file; a flow that has never passed is estimated from the steps in its YAML. Within a shard, flows still run in file order.
To split in plain file order instead (with 29 flows and 4 shards: the first 8 files on shard 1, the next 8 on shard 2, and so on), pass `"shardStrategy": "contiguous"` in the API request. The default is `balanced`.
Each shard's assignment is fixed when the run is created and is returned in the run response: every entry in `flows` carries a `shard_index` and a `flows` list with the file paths that shard runs, in order. Retrying a failed shard (`--retry`, the dashboard, or the flow retry API) re-runs exactly the same files, and retrying a whole run keeps the same shard count and strategy.
Shards run concurrently on separate devices, so a flow cannot rely on another shard having run first. If two flows share state (a test account, data one creates and the other checks), combine those steps into one flow or give each flow its own accounts. With the default `balanced` strategy there is no way to predict which flows share a shard. With `contiguous` the split points are fixed, but two neighbouring files can still fall either side of one, so check the returned `flows` lists rather than relying on file order alone.
## Integrations
TestingBot offers various [integrations](https://testingbot.com/support/integrations) which allow for connecting your Maestro tests with external services. For example; you can receive automated Slack messages each time a Maestro test runs or fails, or you can integrate Maestro testing with any CI/CD system.
- [App Center](https://testingbot.com/support/integrations/appcenter)
- [Bitrise](https://testingbot.com/support/integrations/ci-cd/bitrise)
- [GitHub Actions (Maestro action)](https://testingbot.com/support/app-automate/maestro/ci-cd/github-actions)
- [GitHub PR Checks](https://testingbot.com/support/app-automate/maestro/ci-cd/github-pr-checks)
- [Expo / EAS Workflows](https://testingbot.com/support/app-automate/maestro/ci-cd/expo-eas)
- [Jenkins CI](https://testingbot.com/support/integrations/ci-cd/jenkins)
- [Jira](https://testingbot.com/support/integrations/jira)
- [Slack](https://testingbot.com/support/integrations/slack)
- [TeamCity CI](https://testingbot.com/support/integrations/ci-cd/teamcity)
- [Webhooks](https://testingbot.com/support/integrations/webhooks)
## API
TestingBot provides a [REST-API](https://testingbot.com/support/app-automate/maestro/api) to retrieve information about your Maestro projects and runs.
## Frequently asked questions
What is Maestro and how does it differ from Appium?
Maestro is a declarative mobile UI test framework built by mobile.dev. You describe user flows in YAML files (login, checkout, etc.) and Maestro runs them on iOS and Android. Compared to Appium, Maestro has automatic waits, built-in flakiness tolerance and a much simpler syntax, but it does not implement the W3C WebDriver protocol so Appium tooling does not apply.
How do I run Maestro tests on TestingBot?
Install the [TestingBot CLI](https://testingbot.com#cli), upload your app and Maestro flow ZIP to [TestingBot Storage](https://testingbot.com/support/api/storage#upload), then run the flows with a single CLI command or via the [REST API](https://testingbot.com/support/app-automate/maestro/api). Flows run in parallel on real iPhones, iPads, Android phones and tablets, plus iOS simulators and Android emulators.
Can I run Maestro on real iOS and Android devices?
Yes. TestingBot supports Maestro on real physical iPhones, iPads, Android phones and tablets in our EU datacenter, in addition to iOS simulators and Android emulators. Target a specific device name or use wildcards like `".*Galaxy.*"` to allocate any available matching device.
Why are my Maestro tests slow on real devices?
By default Maestro flows run in parallel on TestingBot. When you pin a specific device, each session waits for that device to free up and complete its post-test clean cycle. Two fixes: (1) target a broader device pattern with wildcards or regex; (2) run all flows in one session with `--shard-split 1` (CLI) or `shardSplit: 1` (API). See the [Grouping Flows section](https://testingbot.com#shard-split) for details.
Can I run Maestro tests from CI/CD?
Yes. TestingBot has dedicated [CI/CD integration docs](https://testingbot.com/support/app-automate/maestro/ci-cd) with examples for GitHub Actions, GitLab CI, Jenkins and CircleCI, plus exit codes, environment variables and build metadata.
Can I pin a specific Maestro version?
Yes. Pass the `maestroVersion` option to pin to an exact Maestro release. See the [Maestro version option](https://testingbot.com/support/app-automate/maestro/options#version) for current supported versions.
How do I fetch Maestro results, video and logs programmatically?
Use the [Maestro REST API](https://testingbot.com/support/app-automate/maestro/api) to list runs, fetch per-flow results, download video and screenshot artifacts, and stream Maestro step logs. The dashboard exposes the same data interactively.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/app-automate/maestro/api
# Maestro API Overview
TestingBot provides a REST API to upload apps, run Maestro flows, and retrieve information about your [Maestro projects and runs](https://testingbot.com/support/app-automate/maestro).
In the documentation below, a Maestro project using a `:project_id` consists of a mobile application that was uploaded together with a zipped set of Maestro flow files.
A Maestro project can have multiple test runs, depending on how many device configurations (capabilities) you specified when you [started the tests](https://testingbot.com/support/app-automate/maestro/api#run).
All API endpoints require authentication using your TestingBot API key and secret, which you can obtain from the Member area.
| Base URL | Authentication |
| --- | --- |
| `https://api.testingbot.com/v1/app-automate/maestro` | HTTP Basic Auth (API key:secret) |
## CLI Alternative
For most use cases, we recommend using the [TestingBot CLI](https://testingbot.com/support/app-automate/maestro#cli) instead of the REST API directly. The CLI handles uploading your app, flows and running tests in a single command:
```bash
# Install the CLI
npm install -g @testingbot/cli
# Authenticate
testingbot login
# Run Maestro tests (handles upload + run automatically)
testingbot maestro app.apk ./flows --device "Pixel 8" --deviceVersion "14"
```
The CLI provides:
- Automatic app and flow uploads
- Real-time progress and streaming output
- Automatic report downloads
- Simpler authentication
Use the REST API directly when you need programmatic access or custom integration workflows.
## Upload App
Upload your Android (.apk/.aab) or iOS (.ipa/.app) application to TestingBot. This creates a new Maestro project and returns a project ID.
| Method | Endpoint |
| --- | --- |
| `POST` | `/app-automate/maestro/app` |
### Parameters
| Name | Type | Required | Description |
| --- | --- | --- | --- |
| `file` | file | Yes | The app file to upload (.apk, .aab, .ipa, .app or .zip) |
[cURL](https://testingbot.com#)
```bash
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/maestro/app" \
-F "file=@/path/to/your/app.apk"
```
### Response
```json
{
"id": 17876,
"app": {
"app_url": "https://...",
"icon_url": "https://...",
"app_version": "1.0.0",
"bundle_id": "com.example.app"
}
}
```
Save the returned `id` - you'll need it for uploading flows and running tests.
## Reuse App
Create a new Maestro project that shares the app binary of a project uploaded earlier, so you can run new flows against the same build without uploading it again. The source project is left untouched. This is what the CLI's `--app-binary-id` flag calls.
| Method | Endpoint |
| --- | --- |
| `POST` | `/app-automate/maestro/app/:id/reuse` |
### Parameters
| Name | Type | Required | Description |
| --- | --- | --- | --- |
| `id` | integer | Yes | The project ID whose app should be reused |
[cURL](https://testingbot.com#)
```bash
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/maestro/app/17876/reuse"
```
### Response
```json
{
"id": 17901,
"source_id": 17876,
"platform": "Android",
"app": {
"app_url": "https://...",
"icon_url": "https://...",
"app_version": "1.0.0",
"bundle_id": "com.example.app"
}
}
```
### Error Responses
| Status Code | Description |
| --- | --- |
| `404` | No project with this ID on your account |
| `410` | The app of this project is no longer in storage. Upload the app again. |
Use the returned `id` as the project for [Upload Maestro Flows](https://testingbot.com#uploadFlows) and [Run Maestro Tests](https://testingbot.com#run). `platform` is derived from the stored app so you know which device capabilities apply.
## Upload Maestro Flows
Upload your Maestro flow files (as a .zip archive) to an existing project. This replaces any previously uploaded flows.
| Method | Endpoint |
| --- | --- |
| `POST` | `/app-automate/maestro/:id/tests` |
### Parameters
| Name | Type | Required | Description |
| --- | --- | --- | --- |
| `id` | integer | Yes | The project ID returned from the app upload |
| `file` | file | Yes | A .zip file containing your Maestro flow YAML files |
[cURL](https://testingbot.com#)
```bash
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/maestro/:id/tests" \
-F "file=@/path/to/flows.zip"
```
### Response
```json
{
"id": 17876,
"flows": [
{
"id": 123,
"name": "login_flow"
}
]
}
```
## Run Maestro Tests
Start running your Maestro flows on one or more devices. You can specify multiple device configurations to run tests in parallel; every capability becomes its own run (this is how the CLI's `--device-matrix` works). All capabilities are validated first: if any one is not a valid device and OS combination, the request is rejected as a whole and no run is created.
| Method | Endpoint |
| --- | --- |
| `POST` | `/app-automate/maestro/:id/run` |
### Parameters
| Name | Type | Required | Description |
| --- | --- | --- | --- |
| `id` | integer | Yes | The project ID |
| `capabilities` | array | Yes | Array of device configurations to run tests on |
| `maestroOptions` | object | No | Additional Maestro options (version, env, secretEnv, flows, tags). `secretEnv` holds credentials: its values are stored encrypted, never returned by the API (the run's `options.env` shows them as `***`) and masked in the reports. See [Test Options](https://testingbot.com/support/app-automate/maestro/options#secret-env). |
| `shardSplit` | integer | No | Group flows into N chunks, each chunk running on a separate device. When omitted, each flow runs in its own session on the same device. |
| `metadata` | object | No | CI/CD metadata stored with the run (`commitSha`, `pullRequestId`, `pullRequestUrl`, `repoOwner`, `repoName`, `branch`, plus a free-form `custom` object). Displayed in the TestingBot dashboard. When `repoOwner`, `repoName` and a full 40-character `commitSha` are present and the [TestingBot GitHub App](https://testingbot.com/support/integrations/ci-cd/github-pr-checks#maestro) is installed for that repository, a status check is posted on the commit and the response carries a `ci_check` object. |
[cURL](https://testingbot.com#)
```bash
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/maestro/:id/run" \
-d '{"capabilities":[{"platformName":"Android","version":"14","deviceName":"Pixel 8"}]}' \
-H "Content-Type: application/json"
```
### Response
```json
{
"success": true,
"id": 17876,
"runs": [
{
"id": 18809,
"capabilities": {
"platformName": "Android",
"version": "14",
"deviceName": "Pixel 8"
},
"flows": [
{
"id": 123,
"name": "login_flow",
"report": "...",
"requested_at": "2025-11-28T09:15:00.000Z",
"completed_at": "2025-11-28T09:16:00.000Z",
"status": "DONE",
"test_case_id": 456
}
]
}
]
}
```
When a GitHub check was registered for the run, the response also includes:
```json
"ci_check": { "provider": "github", "repo": "acme/app", "commit_sha": "<40-hex sha>" }
```
### Error Responses
| Status Code | Description |
| --- | --- |
| `200` with `success: false` | One or more capabilities are not a valid device and OS combination. `errors` lists them and `runs` is empty; nothing was started. |
| `402` | Insufficient credits to run this test. Please upgrade your account to continue running Maestro tests. |
## List Maestro Projects
Retrieve a paginated list of all your Maestro projects.
| Method | Endpoint |
| --- | --- |
| `GET` | `/app-automate/maestro` |
### Query Parameters
| Name | Type | Default | Description |
| --- | --- | --- | --- |
| `offset` | integer | 0 | Number of records to paginate |
| `count` | integer | 10 | Number of records to return |
[cURL](https://testingbot.com#)
```bash
curl -u api_key:api_secret \
"https://api.testingbot.com/v1/app-automate/maestro"
```
### Response
```json
{
"data": [
{
"id": 17876,
"name": "Maestro Project 17876",
"created_at": "2025-11-28T08:12:31.000Z",
"updated_at": "2025-11-28T08:12:31.000Z",
"completed": false,
"app": {
"app_url": "https://...",
"icon_url": "https://...",
"app_version": "2.7.50420",
"bundle_id": "org.wikipedia"
},
"flows_url": "https://...",
"flows": [
{
"id": 123,
"name": "login_flow",
}
],
"runs": [18809, 18854, 18863, 18866]
}
],
"meta": {
"offset": 0,
"count": 10,
"total": 25
}
}
```
## List Maestro Runs
Retrieve a paginated list of all your Maestro runs across every project, newest first. This is a lightweight endpoint: it returns run metadata only and does not inflate the (potentially large) flow reports, so it is well suited to dashboards and CI lookups. For a single run's full results and reports, use [Get Run Info](https://testingbot.com#runInfo).
| Method | Endpoint |
| --- | --- |
| `GET` | `/app-automate/maestro/runs` |
### Query Parameters
| Name | Type | Default | Description |
| --- | --- | --- | --- |
| `offset` | integer | 0 | Number of records to paginate |
| `count` | integer | 10 | Number of records to return |
[cURL](https://testingbot.com#)
```bash
curl -u api_key:api_secret \
"https://api.testingbot.com/v1/app-automate/maestro/runs?count=10&offset=0"
```
### Response
```json
{
"data": [
{
"id": 18809,
"project_id": 17876,
"name": "maestro-android-PR12730-26648200708",
"status": "DONE",
"success": 1,
"capabilities": {
"platformName": "Android",
"version": "14",
"deviceName": "Pixel 8"
},
"created_at": "2025-11-28T09:15:00.000Z",
"completed_at": "2025-11-28T09:16:00.000Z"
}
],
"meta": {
"offset": 0,
"count": 10,
"total": 25
}
}
```
The `status` and `success` fields reflect the run's last-known state. A run is finished once `status` is `DONE` or `FAILED`.
## Find Run by Build Name
Look up runs by their build name (the `name` capability you set when [starting the tests](https://testingbot.com#run)). This lets you resolve a build name straight to its `project_id` and `run_id` in a single request, instead of listing projects and scanning each one.
A single build can fan out to multiple runs (for example one run per device configuration), so this endpoint always returns an array of every matching run.
| Method | Endpoint |
| --- | --- |
| `GET` | `/app-automate/maestro/runs/:build_name` |
### Parameters
| Name | Type | Required | Description |
| --- | --- | --- | --- |
| `build_name` | string | Yes | The build name to look up, matched against the run's `name`. Remember to URL-encode names that contain spaces or special characters. |
[cURL](https://testingbot.com#)
```bash
curl -u api_key:api_secret \
"https://api.testingbot.com/v1/app-automate/maestro/runs/maestro-android-PR12730-26648200708"
```
### Response
```json
{
"data": [
{
"id": 18809,
"project_id": 17876,
"name": "maestro-android-PR12730-26648200708",
"status": "DONE",
"success": 1,
"capabilities": {
"platformName": "Android",
"version": "14",
"deviceName": "Pixel 8"
},
"created_at": "2025-11-28T09:15:00.000Z",
"completed_at": "2025-11-28T09:16:00.000Z"
}
]
}
```
### Error Responses
| Status Code | Description |
| --- | --- |
| `404` | No run matches the supplied build name on your account. |
Use the returned `project_id` and `id` (run ID) to fetch full results from [Get Run Info](https://testingbot.com#runInfo) or download the [JUnit Report](https://testingbot.com#report).
## Get Maestro Project Info
Retrieve details about a specific Maestro project, including all its runs.
| Method | Endpoint |
| --- | --- |
| `GET` | `/app-automate/maestro/:id` |
[cURL](https://testingbot.com#)
```bash
curl -u api_key:api_secret \
"https://api.testingbot.com/v1/app-automate/maestro/:id"
```
### Response
```json
{
"runs": [
{
"id": 18809,
"status": "DONE",
"capabilities": {
"version": "16",
"deviceName": "Pixel 9",
"platformName": "Android"
},
"success": 1,
"options": {},
"flows": [
{
"id": 123,
"name": "login_flow",
"requested_at": "2025-11-28T09:15:00.000Z",
"completed_at": "2025-11-28T09:16:00.000Z",
"status": "DONE",
"success": 1,
"test_case_id": 456
}
]
}
],
"success": true,
"completed": true
}
```
This endpoint is meant to be polled, so it does not carry any JUnit XML: neither the run nor its flows have a `report` field here. Fetch a report through [Get Run Info](https://testingbot.com#runInfo) or [JUnit Report](https://testingbot.com#report) once `completed` is true.
### Run Status Values
| Status | Description |
| --- | --- |
| `WAITING` | Run is queued and waiting for a device |
| `READY` | The test is running on the device |
| `DONE` | Run has completed |
| `FAILED` | Run failed to complete |
| `CANCELLED` | Run was [cancelled](https://testingbot.com#cancel) before it finished, either through the API or from the dashboard |
## Get Run Info
Retrieve detailed information about a specific Maestro run. Use this to poll for completion status.
| Method | Endpoint |
| --- | --- |
| `GET` | `/app-automate/maestro/:project_id/:run_id` |
[cURL](https://testingbot.com#)
```bash
curl -u api_key:api_secret \
"https://api.testingbot.com/v1/app-automate/maestro/:project_id/:run_id"
```
Replace `:project_id` with your project ID and `:run_id` with the run ID.
### Response
```json
{
"id": 18809,
"status": "DONE",
"capabilities": {
"version": "16",
"deviceName": "Pixel 9",
"platformName": "Android"
},
"flows": [
{
"id": 123,
"name": "login_flow",
"report": "...",
"requested_at": "2025-11-28T09:15:00.000Z",
"completed_at": "2025-11-28T09:16:00.000Z",
"status": "DONE",
"test_case_id": 456
}
],
"success": 1,
"report": "",
"options": {},
"completed": true
}
```
Poll this endpoint to check if the test has finished. The `completed` field will be `true` when the run has completed.
## JUnit Report
Retrieve the JUnit XML report for a completed Maestro run. This is useful for CI/CD integration.
| Method | Endpoint |
| --- | --- |
| `GET` | `/app-automate/maestro/:project_id/:run_id/junit_report` |
[cURL](https://testingbot.com#)
```bash
curl -u api_key:api_secret \
"https://api.testingbot.com/v1/app-automate/maestro/:project_id/:run_id/junit_report"
```
### Response
```json
{
"junit_report": "..."
}
```
The `junit_report` field contains the XML-formatted JUnit report generated by Maestro.
## Cancel Maestro Run
Immediately cancel a queued or running Maestro run. Cancelling stops the run from requesting any more devices, closes out every flow that has not finished yet, releases the device session the run is currently holding, and drops any request the run still has queued on the device grid, so you stop being billed for it right away. Flows that already completed keep their results and reports.
| Method | Endpoint |
| --- | --- |
| `POST` | `/app-automate/maestro/:project_id/:run_id/cancel` |
### Parameters
| Name | Type | Required | Description |
| --- | --- | --- | --- |
| `project_id` | integer | Yes | The project ID |
| `run_id` | integer | Yes | The ID of the run to cancel |
[cURL](https://testingbot.com#)
```bash
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/maestro/:project_id/:run_id/cancel"
```
### Response
```json
{
"success": true,
"run": {
"id": 18809,
"project_id": 4211,
"name": "nightly-regression",
"status": "CANCELLED",
"success": 0,
"capabilities": {
"platformName": "Android",
"version": "16",
"deviceName": "Pixel 9"
},
"created_at": "2025-11-28T09:15:00.000Z",
"completed_at": "2025-11-28T09:15:42.000Z"
},
"cancelled_flow_count": 3
}
```
`cancelled_flow_count` is how many flows were still queued or running when the cancel landed. A `0` means the run held no unfinished flows; its device session, if it had one, is still released.
### Error Responses
| Status Code | Description |
| --- | --- |
| `403` | Your account is read-only and cannot start or stop tests. |
| `404` | No project or run with those IDs exists on your account. |
| `409` | The run had already reached a terminal state (`DONE`, `FAILED` or `CANCELLED`), so there was nothing to cancel. |
A cancelled run cannot be resumed, and its individual flows cannot be retried with [Retry Maestro Flow](https://testingbot.com#retry). Use [Retry Maestro Run](https://testingbot.com#retryRun) to start the whole run again on the same capabilities.
## Stop Maestro Run
Alias of [Cancel Maestro Run](https://testingbot.com#cancel), kept for existing integrations. It does exactly the same thing, with one difference: it always answers `200`, so stopping a run that has already finished is a no-op rather than a `409`. It returns only a success flag, not the run. New integrations should use the cancel endpoint.
| Method | Endpoint |
| --- | --- |
| `POST` | `/app-automate/maestro/:project_id/:run_id/stop` |
[cURL](https://testingbot.com#)
```bash
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/maestro/:project_id/:run_id/stop"
```
### Response
```json
{
"success": true
}
```
## Retry Maestro Flow
Re-run a single flow from a Maestro run. This creates a new flow attempt inside the same run and schedules it on the same device configuration, which is useful for retrying a flaky or failed flow without starting a whole new run.
| Method | Endpoint |
| --- | --- |
| `POST` | `/app-automate/maestro/:project_id/:run_id/:flow_id/retry` |
### Parameters
| Name | Type | Required | Description |
| --- | --- | --- | --- |
| `project_id` | integer | Yes | The project ID |
| `run_id` | integer | Yes | The run ID |
| `flow_id` | integer | Yes | The ID of the flow (from the run's `flows` array) to retry |
[cURL](https://testingbot.com#)
```bash
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/maestro/:project_id/:run_id/:flow_id/retry"
```
### Response
```json
{
"success": true,
"flow": {
"id": 124,
"name": "login_flow",
"status": "WAITING",
"requested_at": "2025-11-28T09:20:00.000Z"
}
}
```
The retry runs as a new flow attempt inside the same run. Poll the [Get Run Info](https://testingbot.com#runInfo) endpoint to track the new flow's status via its returned `id`.
## Retry Maestro Run
Re-run an entire Maestro run. This creates a brand new run on the same device configuration (capabilities) and options as the original, re-scheduling every flow. Use this when you want to repeat a full run rather than a single [flow](https://testingbot.com#retry).
| Method | Endpoint |
| --- | --- |
| `POST` | `/app-automate/maestro/:project_id/:run_id/retry` |
### Parameters
| Name | Type | Required | Description |
| --- | --- | --- | --- |
| `project_id` | integer | Yes | The project ID |
| `run_id` | integer | Yes | The ID of the run to retry |
[cURL](https://testingbot.com#)
```bash
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/maestro/:project_id/:run_id/retry"
```
### Response
```json
{
"success": true,
"run": {
"id": 18920,
"capabilities": {
"platformName": "Android",
"version": "14",
"deviceName": "Pixel 8"
},
"flows": [
{
"id": 125,
"name": "login_flow",
"status": "WAITING",
"requested_at": "2025-11-28T09:25:00.000Z"
}
]
}
}
```
A new run is created with its own `id`. Poll the [Get Run Info](https://testingbot.com#runInfo) endpoint with that ID to track its status.
## Frequently asked questions
Do I have to use the REST API or can I just use the TestingBot CLI?
For most workflows the [TestingBot CLI](https://testingbot.com/support/app-automate/maestro) is simpler since it handles upload, run and result polling in one command. The REST API is the right choice when you want fine-grained control, are integrating from a non-Node environment, or need to fetch artifacts asynchronously.
How do I upload my app and Maestro flows via the API?
POST your binary to the `/maestro/app` endpoint and your flow ZIP to `/maestro/:id/tests` with HTTP basic auth (API key and secret). The response returns IDs you use when starting the run. See the [Upload App](https://testingbot.com#uploadApp) and [Upload Maestro Flows](https://testingbot.com#uploadFlows) sections.
How do I start a Maestro test run via the API?
POST to the run endpoint with the app and flow IDs from the upload step, plus your device target and any options. See the [Run Maestro Tests section](https://testingbot.com#run) for the request body and response shape.
How do I fetch JUnit-style reports for CI?
GET the run's JUnit endpoint to receive a JUnit XML file with per-flow pass and fail results. Most CI runners can consume this format directly. See the [JUnit Report section](https://testingbot.com#report).
How do I find a run by its build name?
GET `/maestro/runs/:build_name` to resolve a build name straight to its project ID and run ID(s) in one request, instead of listing projects and scanning each one. A build can map to multiple runs (one per device), so the response is always an array. See the [Find Run by Build Name section](https://testingbot.com#findRun).
How do I cancel a running Maestro test?
POST to the cancel endpoint with the project ID and run ID. The run stops requesting devices, its unfinished flows are closed out, and the device session it holds is released, so billing stops right away. Flows that already finished keep their results. See the [Cancel Maestro Run section](https://testingbot.com#cancel) for details.
Can I retry a single flow from a cancelled run?
No. A cancelled run cannot be reopened, so flow-level retry returns a `409` for it. Use [Retry Maestro Run](https://testingbot.com#retryRun) to run the whole thing again on the same capabilities.
How do I retry a single failed flow?
POST to the retry endpoint with the project ID, run ID and flow ID. This re-runs just that flow inside the same run instead of starting a whole new run. See the [Retry Maestro Flow section](https://testingbot.com#retry).
How do I retry an entire run?
POST to the run-level retry endpoint with the project ID and run ID. This creates a new run on the same capabilities and options, re-scheduling every flow. See the [Retry Maestro Run section](https://testingbot.com#retryRun).
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/app-automate/maestro/options
# Maestro Test Options
When running Maestro tests on TestingBot, you can customize the test execution using various options. These options are passed when [starting a test run](https://testingbot.com/support/app-automate/maestro/api#run) via the API.
Options are organized into two categories:
- **maestroOptions** : Options specific to Maestro execution (version, environment variables, flows, tags)
- **capabilities** : Device and test configuration options (device, orientation, locale, network, etc.)
Moving an existing suite across? See [migrate from Maestro Cloud](https://testingbot.com/support/app-automate/maestro/migrate-from-maestro-cloud) for the full flag mapping.
## Quick Reference
All available options at a glance:
### Maestro Options
| Option | Type | Description |
| --- | --- | --- |
| `version` | string | Maestro CLI version to use (e.g., "2.0.10") |
| `flows` | array | Only run these flows, by path, file name or directory (e.g., ["login.yml", "checkout.yml"]) |
| `env` | object | Environment variables to pass to flows |
| `includeTags` | array | Only run flows carrying at least one of these tags. Untagged flows are skipped |
| `excludeTags` | array | Skip flows carrying any of these tags. Untagged flows are kept |
| `shardSplit` | number | Split flows across N parallel sessions. Top-level run parameter, not inside `maestroOptions` |
| `shardStrategy` | string | `balanced` (default) evens out shard durations; `contiguous` splits in file order. Top-level run parameter, not inside `maestroOptions` (REST API only) |
| `showKeyboard` | boolean | Show the soft keyboard on Android (default: false) |
| `commitSha` | string | Git commit SHA for CI/CD tracking |
| `pullRequestId` | string | Pull request ID for CI/CD tracking |
| `repoName` | string | Repository name (e.g., GitHub repo slug) |
| `repoOwner` | string | Repository owner (e.g., GitHub org or username) |
### Capability Options
| Option | Type | Description |
| --- | --- | --- |
| `platformName` | string | Platform: "Android" or "iOS" (required) |
| `version` | string | OS version (required) |
| `deviceName` | string | Device name (required) |
| `name` | string | Test name shown in dashboard |
| `build` | string | Build name for grouping tests |
| `groups` | comma-separated strings | Groups to which the test belongs, visible in the dashboard |
| `orientation` | string | "PORTRAIT" (default) or "LANDSCAPE" |
| `locale` | string | Device locale (e.g., "en\_US", "de\_DE") |
| `timeZone` | string | Device timezone (e.g., "America/New\_York") |
| `googlePlayStore` | boolean | Use Google Play Store Android emulator. |
| `throttle_network` | string | Network preset: "Edge", "3G", "4G", "airplane" |
| `testingbot.geoCountryCode` | string | Route traffic through specific country (ISO code) |
## Set Maestro Version
Unless you pin one, TestingBot runs your flows with Maestro **2.6.0**. This is a known-good default rather than the newest release, so pin a version explicitly if you need a specific one.
These versions can be pinned:
- `2.12.0`,`2.11.0`,`2.10.0`, `2.9.0`, `2.8.0`, `2.7.0`
- `2.6.1`, `2.6.0` (default), `2.5.0`, `2.4.0`
- `2.3.0`, `2.0.10`
**Startup time.** A few versions are pre-installed on our devices and start immediately. Any other version is fetched when the session starts, which adds to your startup time. If a run is slow to start after pinning a version, that is why.
[CLI](https://testingbot.com#) [cURL](https://testingbot.com#)
```bash
testingbot maestro app.apk ./flows \
--device "Pixel 8" \
--deviceVersion "14" \
--maestro-version "2.0.10"
```
If you need a specific version for compatibility, specify the `version` option in `maestroOptions`.
| Option | Type | Example |
| --- | --- | --- |
| `maestroOptions.version` | string | "2.0.10" |
```bash
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/maestro/:id/run" \
-d '{
"maestroOptions": {
"version": "2.0.10"
},
"capabilities": [{
"platformName": "Android",
"version": "14",
"deviceName": "Pixel 8"
}]
}' \
-H "Content-Type: application/json"
```
Replace `:id` with your project ID from the [app upload](https://testingbot.com/support/app-automate/maestro/api#uploadApp).
## Select Specific Flows
By default, TestingBot runs every flow file in your zip archive, one device session per flow. To run only some of them, use the option below. Flows you filter out are not scheduled at all, so they do not consume a session.
Each entry is matched against the path of the flow inside your archive, and can be:
- the full path, for example `flows/smoke/login.yml`
- just the file name, for example `login.yml`
- a directory, which selects every flow inside it, for example `flows/smoke`
### Use Cases
- Run a subset of flows for faster feedback during development
- Execute specific test scenarios without modifying your test suite
- Re-run failed flows without running the entire suite
[CLI](https://testingbot.com#) [cURL](https://testingbot.com#)
With the CLI, you specify flows directly as arguments:
```bash
# Run specific flow files
testingbot maestro app.apk login.yml checkout.yml --device "Pixel 8"
# Run flows from a specific directory
testingbot maestro app.apk ./flows/smoke --device "Pixel 8"
# Mix directories and individual files
testingbot maestro app.apk ./flows/smoke login.yml --device "Pixel 8"
```
| Option | Type | Example |
| --- | --- | --- |
| `maestroOptions.flows` | array of strings | `["login.yml", "checkout.yml"]` |
```bash
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/maestro/:id/run" \
-d '{
"maestroOptions": {
"flows": ["login.yml", "checkout.yml"]
},
"capabilities": [{
"platformName": "Android",
"version": "14",
"deviceName": "Pixel 8"
}]
}' \
-H "Content-Type: application/json"
```
**A filter that matches nothing is rejected.** If `flows`, `includeTags` or `excludeTags` leave no flows to run, the run request comes back with a `400` listing the flows the project does have, and nothing is scheduled. Check the paths and tags against the archive you uploaded.
## Environment Variables
Pass environment variables to your Maestro flows using the `env` option. This is useful for configuration that varies between environments (staging, production): backend URLs, feature flags, package names. Values passed this way are shown as given in the run's options, in the TestingBot dashboard and in the JUnit and HTML reports, so for credentials use [`secretEnv`](https://testingbot.com#secret-env) instead.
### Use Cases
- **Environment-specific URLs** : Point to staging or production backends
- **Feature flags** : Enable or disable features during tests
- **App identifiers** : Specify package names or bundle IDs dynamically
- **Test credentials** : Pass them as [secrets](https://testingbot.com#secret-env), not as plain env
[CLI](https://testingbot.com#) [cURL](https://testingbot.com#)
Use the `-e` or `--env` flag (can be repeated):
```bash
testingbot maestro app.apk ./flows --device "Pixel 8" \
-e API_URL=https://staging.example.com \
-e TEST_USER=testuser@example.com \
-e FEATURE_CHECKOUT_V2=true
```
| Option | Type | Example |
| --- | --- | --- |
| `maestroOptions.env` | object | `{"API_URL": "https://staging.example.com"}` |
```bash
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/maestro/:id/run" \
-d '{
"maestroOptions": {
"env": {
"API_URL": "https://staging.example.com",
"TEST_USER": "testuser@example.com",
"FEATURE_CHECKOUT_V2": "true"
}
},
"capabilities": [{
"platformName": "Android",
"version": "14",
"deviceName": "Pixel 8"
}]
}' \
-H "Content-Type: application/json"
```
### Using Variables in Your Flows
Reference environment variables in your Maestro flow files using the `${VARIABLE_NAME}` syntax:
```yaml
appId: com.example.app
---
- launchApp
- inputText:
text: "${TEST_USER}"
- tapOn: "Next"
- inputText:
text: "${TEST_PASSWORD}"
- tapOn: "Login"
```
### Secrets
Passwords, API keys and tokens your flows need go in `secretEnv`. A secret works exactly like an `env` variable inside the flow (`${TEST_PASSWORD}` above), but TestingBot treats its value differently:
- It is stored encrypted and is never returned by the API: the run's `options.env` lists the variable with the value `***`, and `options.secretEnvKeys` names the variables that were passed as secrets.
- It is masked as `***` wherever it would otherwise appear in the run's output: the JUnit report (an `inputText` step records the text it typed), the HTML reports, the Maestro console log and the TestingBot dashboard.
- A whole-run retry re-uses the secrets of the run it retries, masked the same way.
A variable name can be given either as a secret or as plain env, not both; a run request that lists the same name in `env` and `secretEnv` is rejected with a `400`.
[CLI](https://testingbot.com#) [cURL](https://testingbot.com#)
Use the `--secret-env` flag (can be repeated) next to `-e`:
```bash
testingbot maestro app.apk ./flows --device "Pixel 8" \
-e API_URL=https://staging.example.com \
-e TEST_USER=testuser@example.com \
--secret-env TEST_PASSWORD="$TEST_PASSWORD"
```
| Option | Type | Example |
| --- | --- | --- |
| `maestroOptions.secretEnv` | object | `{"TEST_PASSWORD": "..."}` |
```bash
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/maestro/:id/run" \
-d '{
"maestroOptions": {
"env": {
"API_URL": "https://staging.example.com",
"TEST_USER": "testuser@example.com"
},
"secretEnv": {
"TEST_PASSWORD": "..."
}
},
"capabilities": [{
"platformName": "Android",
"version": "14",
"deviceName": "Pixel 8"
}]
}' \
-H "Content-Type: application/json"
```
**Best Practice:** Never commit passwords or API keys to your repository, and never pass them with `-e` / `env`: those values are visible in the run's options and reports. Keep them in your CI/CD provider's secret store and pass them with `--secret-env` / `secretEnv`.
## Include Tags
Run only flows that have specific tags. This is equivalent to Maestro's `--include-tags` option.
**Untagged flows are skipped.** When you set `includeTags`, a flow has to carry at least one of the tags you listed to run. Flows with no tags at all are left out, which is how Maestro's own `--include-tags` behaves. Tags are matched without regard to case.
### Adding Tags to Flows
Add tags to your flow files using the `tags` property:
```yaml
appId: com.example.app
tags:
- smoke
- login
---
- launchApp
- tapOn: "Login"
```
[CLI](https://testingbot.com#) [cURL](https://testingbot.com#)
Use the `--include-tags` flag with comma-separated tags:
```bash
testingbot maestro app.apk ./flows --device "Pixel 8" \
--include-tags "smoke,critical"
```
| Option | Type | Example |
| --- | --- | --- |
| `maestroOptions.includeTags` | array of strings | `["smoke", "critical"]` |
```bash
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/maestro/:id/run" \
-d '{
"maestroOptions": {
"includeTags": ["smoke", "critical"]
},
"capabilities": [{
"platformName": "Android",
"version": "14",
"deviceName": "Pixel 8"
}]
}' \
-H "Content-Type: application/json"
```
## Exclude Tags
Skip flows that have specific tags. This is equivalent to Maestro's `--exclude-tags` option. Unlike [includeTags](https://testingbot.com#include-tags), flows with no tags are kept.
You can combine the two. `includeTags` is applied first and `excludeTags` second, so a flow tagged both `smoke` and `flaky` is skipped by `"includeTags": ["smoke"], "excludeTags": ["flaky"]`.
### Use Cases
- Skip slow tests during development for faster feedback
- Exclude known flaky tests from CI pipelines
- Skip platform-specific tests (e.g., exclude "ios-only" when running on Android)
[CLI](https://testingbot.com#) [cURL](https://testingbot.com#)
Use the `--exclude-tags` flag with comma-separated tags:
```bash
testingbot maestro app.apk ./flows --device "Pixel 8" \
--exclude-tags "slow,flaky"
```
| Option | Type | Example |
| --- | --- | --- |
| `maestroOptions.excludeTags` | array of strings | `["slow", "flaky"]` |
```bash
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/maestro/:id/run" \
-d '{
"maestroOptions": {
"excludeTags": ["slow", "flaky"]
},
"capabilities": [{
"platformName": "Android",
"version": "14",
"deviceName": "Pixel 8"
}]
}' \
-H "Content-Type: application/json"
```
## IP Geolocation
Test your app from different geographic locations using TestingBot's GeoIP feature. Traffic from the device will route through an IP address in the specified country.
### Use Cases
- Test geo-restricted content or features
- Verify location-based pricing or currency display
- Test localized content delivery
[CLI](https://testingbot.com#) [cURL](https://testingbot.com#)
```bash
testingbot maestro app.zip ./flows \
--platform iOS \
--device "iPhone 15" \
--deviceVersion "17.2" \
--geo-country-code "DE"
```
| Option | Type | Example |
| --- | --- | --- |
| `testingbot.geoCountryCode` | string (ISO code) | `"DE", "US", "JP"` |
```bash
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/maestro/:id/run" \
-d '{
"capabilities": [{
"platformName": "iOS",
"version": "17.2",
"deviceName": "iPhone 15",
"testingbot.geoCountryCode": "DE"
}]
}' \
-H "Content-Type: application/json"
```
See the full list of [available country codes](https://testingbot.com/support/web-automate/selenium/test-options#geo).
## Network Throttling
Simulate different network conditions to test how your app performs under various connectivity scenarios.
| Option | Type | Example |
| --- | --- | --- |
| `throttle_network` | string | "4G", "3G", "Edge", "airplane" |
### Network Presets
| Preset | Download | Upload | Latency | Use Case |
| --- | --- | --- | --- | --- |
| `4G` | 18 Mbps | 9 Mbps | 100ms | Standard mobile connection |
| `3G` | 400 Kbps | 100 Kbps | 100ms | Slower mobile network |
| `Edge` | 250 Kbps | 150 Kbps | 300ms | Poor connectivity areas |
| `airplane` | 0 | 0 | - | No connectivity (offline mode) |
| `disable` | - | - | - | Restore full connectivity |
[CLI](https://testingbot.com#) [cURL](https://testingbot.com#)
```bash
testingbot maestro app.zip ./flows \
--platform iOS \
--device "iPhone 15" \
--deviceVersion "17.2" \
--throttle-network "3G"
```
```bash
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/maestro/:id/run" \
-d '{
"capabilities": [{
"platformName": "iOS",
"version": "17.2",
"deviceName": "iPhone 15",
"throttle_network": "3G"
}]
}' \
-H "Content-Type: application/json"
```
## Test Name
Set a custom name for your test that appears in the TestingBot dashboard. This makes it easier to identify specific test runs.
| Option | Type | Example |
| --- | --- | --- |
| `name` | string | "Login Flow - Smoke Test" |
[CLI](https://testingbot.com#) [cURL](https://testingbot.com#)
```bash
testingbot maestro app.zip ./flows \
--platform iOS \
--device "iPhone 15" \
--deviceVersion "17.2" \
--name "Login Flow - Smoke Test"
```
```bash
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/maestro/:id/run" \
-d '{
"capabilities": [{
"platformName": "iOS",
"version": "17.2",
"deviceName": "iPhone 15",
"name": "Login Flow - Smoke Test"
}]
}' \
-H "Content-Type: application/json"
```
## Build Grouping
Group multiple test runs under a single build name. This is useful for organizing tests that belong to the same CI/CD pipeline run or release version.
| Option | Type | Example |
| --- | --- | --- |
| `build` | string | "Release 2.5.0" |
### Benefits
- View aggregated pass/fail statistics for all tests in a build
- Track test results across different device configurations
- Easily identify which builds introduced failures
[CLI](https://testingbot.com#) [cURL](https://testingbot.com#)
```bash
# The Maestro CLI groups runs by the --name you pass; the API exposes this as the "build" field
testingbot maestro app.ipa ./flows \
--platform iOS \
--device "iPhone 17" \
--real-device \
--name "Release 2.5.0"
```
```bash
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/maestro/:id/run" \
-d '{
"capabilities": [{
"platformName": "iOS",
"version": "26.0",
"deviceName": "iPhone 17",
"realDevice": true,
"build": "Release 2.5.0"
}]
}' \
-H "Content-Type: application/json"
```
## Shard Split
Group your Maestro flows into chunks, which will run across multiple parallel sessions for faster execution. For example you have 3 parallel slots and 9 tests, but you want to split this test suite into 3 chunks of tests and run them in parallel on connected devices, then you would use `shardSplit` set to 3.
By default the shards are **balanced** by expected duration, taken from your previous runs of each flow file (or estimated from the flow's steps when it has not passed before), so they finish at about the same time. Set `shardStrategy` to `contiguous` to split in plain file order instead (REST API only for now; the CLI does not have a flag for it yet, so CLI runs always use `balanced`). Within a shard, flows run in file order either way, and shards never share state: flows that depend on each other belong in the same shard, which only `contiguous` lets you control.
| Option | Type | Example |
| --- | --- | --- |
| `shardSplit` | number | 1 |
| `shardStrategy` | string | `balanced` (default) or `contiguous` |
[CLI](https://testingbot.com#) [cURL](https://testingbot.com#)
```bash
# Split flows across 3 parallel sessions - without this each flow would run in its own session
testingbot maestro app.apk ./flows \
--device "Pixel 8" \
--deviceVersion "14" \
--shard-split 3
```
```bash
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/maestro/:id/run" \
-d '{
"shardSplit": 3,
"capabilities": [{
"platformName": "Android",
"version": "14",
"deviceName": "Pixel 8"
}]
}' \
-H "Content-Type: application/json"
```
## Device Matrix
Run the same flows across several devices in one command with `--device-matrix`. Each cell names exactly one device as `[:][:real]`; cells are comma-separated or the flag is repeated. There is no cross-product, because not every device exists in every OS version.
```bash
testingbot maestro app.apk ./flows \
--device-matrix "Pixel 9:14" \
--device-matrix "Samsung Galaxy S24:14:real" \
--device-matrix "Pixel 8"
```
Every flow runs once per device, so the cost is devices × flows; the CLI prints that summary before submitting. Each device becomes its own run with its own results, live table rows and dashboard link, `--json` lists the device per run, and `--retry` re-runs only the flow that failed on the device it failed on. If any cell is not a valid device and OS combination the whole request is rejected and nothing runs, so a matrix never partially submits.
`--real-device`, or an `.ipa` app which implies it, applies to every cell. `--device` and `--deviceVersion` cannot be combined with a matrix; list every device as a cell instead.
## Reuse an Uploaded App
Every run prints its project ID after the app upload. Pass that ID to a later run with `--app-binary-id` and the upload is skipped; each run still gets its own project and results. `testingbot upload` uploads an app on its own and prints the ID, which is handy for pipelines that build once and run several suites.
```bash
testingbot upload app.apk
# App ready: app.apk. Project ID: 4321
# Run flows against it with: testingbot maestro --app-binary-id 4321 ./flows
testingbot maestro --app-binary-id 4321 ./flows/smoke
testingbot maestro --app-binary-id 4321 ./flows/regression --device "Pixel 9"
```
The platform is taken from the stored app unless `--platform` is given. Unchanged binaries are also deduplicated by checksum on every upload; pass `--ignore-checksum-check` to force a fresh upload.
## Expo / EAS Build
`eas build` hands back a download URL rather than a local file, and for iOS simulator builds the artifact is a `.tar.gz` that contains the `.app`. Pass the URL with `--app-url` and the CLI downloads it, extracts the bundle when needed, and uploads it like any other app. Android EAS builds (`.apk`) and plain `.zip` / `.ipa` URLs work the same way, and a local `.tar.gz` can be passed as the app file directly.
```bash
# iOS simulator build
URL=$(eas build --platform ios --profile preview --json --non-interactive | jq -r '.[0].artifacts.buildUrl')
testingbot maestro --app-url "$URL" ./flows --device "iPhone 16"
# Android build
URL=$(eas build --platform android --profile preview --json --non-interactive | jq -r '.[0].artifacts.buildUrl')
testingbot maestro --app-url "$URL" ./flows --device "Pixel 9"
```
Inside an EAS Build or EAS Workflows job the run is tagged automatically with the EAS build id, profile and platform, and the job's commit hash is used as the commit SHA unless `--commit-sha` is given. EAS download links are signed and expire after about an hour; a rejected link is reported as such so you know to request a fresh one. `testingbot upload ` accepts URLs too, which is handy to [upload once and reuse](https://testingbot.com#app-binary-id) across several suites.
## JSON Output and Exit Codes
`--json` prints one JSON document on stdout with the outcome, project ID, dashboard URL and every flow attempt per run, and moves all log lines to stderr. `--json-file` writes the same document to `_testingbot.json` (or `--json-file-name`) while keeping the console output. Exit codes are `0` passed, `2` flows failed, `1` CLI or infrastructure error. Examples and the document layout are on the [CI/CD page](https://testingbot.com/support/app-automate/maestro/ci-cd#json).
## CI/CD Integration
Pass Git and repository metadata to associate test runs with specific commits and pull requests. This information appears in the TestingBot dashboard and can be used for tracking and debugging.
| Option | Type | Description |
| --- | --- | --- |
| `commitSha` | string | Git commit SHA associated with this test run |
| `pullRequestId` | string | Pull request ID this test run originated from |
| `repoName` | string | Repository name (e.g., GitHub repo slug) |
| `repoOwner` | string | Repository owner (e.g., GitHub organization or username) |
[CLI](https://testingbot.com#) [cURL](https://testingbot.com#)
```bash
# Pass Git metadata for CI/CD tracking
testingbot maestro app.apk ./flows \
--device "Pixel 8" \
--deviceVersion "14" \
--commit-sha "abc123def456" \
--pull-request-id "42" \
--repo-owner "myorg" \
--repo-name "myapp"
```
### GitHub Actions Example
The [TestingBot Maestro GitHub Action](https://github.com/testingbot/maestro-action) fills in the branch, commit, repository and pull request from the workflow context, so no metadata flags are needed:
```yaml
- uses: testingbot/maestro-action@v1
with:
api-key: ${{ secrets.TB_KEY }}
api-secret: ${{ secrets.TB_SECRET }}
app-file: app.apk
workspace: ./flows
device: Pixel 8
```
See [GitHub Actions](https://testingbot.com/support/app-automate/maestro/ci-cd/github-actions) for the full workflow and the action's outputs.
```bash
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/maestro/:id/run" \
-d '{
"metadata": {
"commitSha": "abc123def456",
"pullRequestId": "42",
"repoName": "myapp",
"repoOwner": "myorg"
},
"capabilities": [{
"platformName": "Android",
"version": "14",
"deviceName": "Pixel 8"
}]
}' \
-H "Content-Type: application/json"
```
## Device Orientation
Set the device orientation at the start of your test.
| Option | Type | Values | Default |
| --- | --- | --- | --- |
| `orientation` | string | `"PORTRAIT", "LANDSCAPE"` | PORTRAIT |
[CLI](https://testingbot.com#) [cURL](https://testingbot.com#)
```bash
testingbot maestro app.apk ./flows \
--device "Pixel 8" \
--deviceVersion "14" \
--orientation LANDSCAPE
```
```bash
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/maestro/:id/run" \
-d '{
"capabilities": [{
"platformName": "Android",
"version": "14",
"deviceName": "Pixel 8",
"orientation": "LANDSCAPE"
}]
}' \
-H "Content-Type: application/json"
```
## Soft Keyboard (Android)
By default, the Android soft keyboard is hidden during Maestro tests. Use the `showKeyboard` option to display the soft keyboard on Android devices.
| Option | Type | Default | Platform |
| --- | --- | --- | --- |
| `maestroOptions.showKeyboard` | boolean | false | Android only |
```bash
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/maestro/:id/run" \
-d '{
"maestroOptions": {
"showKeyboard": true
},
"capabilities": [{
"platformName": "Android",
"version": "14",
"deviceName": "Pixel 8"
}]
}' \
-H "Content-Type: application/json"
```
## Platform Configuration
Platform-specific settings allow you to optimize the environment for Android or iOS. These are configured in your [config.yaml](https://testingbot.com#config) file under the `platform` key.
### iOS specifics
| Key | Description |
| --- | --- |
| `disableAnimations` | Enables Reduce Motion on the iOS Simulator to prevent flakiness caused by system-level animations. |
### Android specifics
| Key | Description |
| --- | --- |
| `disableAnimations` | Disables system-level window, transition and animator animations on the Android Emulator. |
### Example config.yaml
```yaml
platform:
ios:
disableAnimations: true
android:
disableAnimations: true
```
## Device Timezone
Set the device timezone for your test. Useful for testing time-sensitive features.
| Option | Type | Example |
| --- | --- | --- |
| `timeZone` | string (IANA) | `"America/New_York", "Europe/London", "Asia/Tokyo"` |
[CLI](https://testingbot.com#) [cURL](https://testingbot.com#)
```bash
testingbot maestro app.zip ./flows \
--platform iOS \
--device "iPhone 15" \
--deviceVersion "17.2" \
--timezone "America/New_York"
```
```bash
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/maestro/:id/run" \
-d '{
"capabilities": [{
"platformName": "iOS",
"version": "17.2",
"deviceName": "iPhone 15",
"timeZone": "America/New_York"
}]
}' \
-H "Content-Type: application/json"
```
See the [full list of timezone identifiers](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones).
## Device Locale
Set the device locale to test localized content and formatting. The locale affects language, date formats, number formats, and currency display.
| Option | Type | Format | Example |
| --- | --- | --- | --- |
| `locale` | string | language\_COUNTRY | `"de_DE", "fr_CA", "ja_JP"` |
[CLI](https://testingbot.com#) [cURL](https://testingbot.com#)
```bash
testingbot maestro app.zip ./flows \
--platform iOS \
--device "iPhone 15" \
--deviceVersion "17.2" \
--device-locale "de_DE"
```
```bash
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/maestro/:id/run" \
-d '{
"capabilities": [{
"platformName": "iOS",
"version": "17.2",
"deviceName": "iPhone 15",
"locale": "de_DE"
}]
}' \
-H "Content-Type: application/json"
```
### Common Locale Codes
| Region | Locale Code |
| --- | --- |
| English (US) | `en_US` |
| English (UK) | `en_GB` |
| German (Germany) | `de_DE` |
| French (France) | `fr_FR` |
| French (Canada) | `fr_CA` |
| Spanish (Spain) | `es_ES` |
| Spanish (Mexico) | `es_MX` |
| Japanese | `ja_JP` |
| Chinese (Simplified) | `zh_CN` |
| Chinese (Traditional) | `zh_TW` |
| Korean | `ko_KR` |
| Portuguese (Brazil) | `pt_BR` |
| Italian | `it_IT` |
| Dutch | `nl_NL` |
| Russian | `ru_RU` |
| Arabic | `ar_SA` |
| Hindi | `hi_IN` |
## Maestro config.yaml
You can include a `config.yaml` file in your [Maestro flows zip file](https://testingbot.com/support/app-automate/maestro#uploadSuite). This configuration file lets you define options that apply to all your flows.
### Supported config.yaml Options
| Option | Description |
| --- | --- |
| `includeTags` | Only run flows with these tags |
| `excludeTags` | Skip flows with these tags |
| `env` | Environment variables available in all flows |
| `platform` | Platform-specific settings for iOS and Android (see [Platform Configuration](https://testingbot.com#platform-configuration)) |
### Example config.yaml
```yaml
env:
API_URL: https://api.example.com
DEBUG: "true"
includeTags:
- smoke
- critical
```
## TestingBot Tunnel
Use [TestingBot Tunnel](https://testingbot.com/support/tunnel) to allow your app to access private or staging backend servers during tests.
### Use Cases
- Test against staging APIs not accessible from the public internet
- Access localhost services running on your machine
- Test with private databases or internal services
When a TestingBot Tunnel is active on your account, Maestro tests automatically route traffic through it. No additional configuration is needed.
To start a tunnel, download and run the [TestingBot Tunnel](https://testingbot.com/support/tunnel) client before starting your tests.
You can also use the built-in [tunnel](https://testingbot.com#tunnel-options) option to automatically start a tunnel for the duration of your test run.
See the dedicated [Maestro Tunnel guide](https://testingbot.com/support/app-automate/maestro/tunnel) for use cases, a staging token helper and CI/CD examples.
## CLI Reference
When using the [TestingBot CLI](https://testingbot.com/support/app-automate/maestro#cli), these options are available as command-line flags:
### App & Flows
The application and flows are normally passed as positional arguments (`testingbot maestro app.apk ./flows`), but you can also set them explicitly:
| CLI Option | Description |
| --- | --- |
| `--app ` | Path to the application under test (.apk, .ipa, .app, or .zip) |
| `--other-app ` | Additional app to install alongside `--app`. Can be repeated, up to 4 times. |
| `--app-url ` | Download the app from an http(s) URL instead of a local file: `.apk`, `.ipa`, `.zip` or an EAS Build iOS `.tar.gz` (the `.app` inside is extracted automatically). Every positional argument is then a flow. Signed URLs such as EAS links expire after about an hour. See [Expo / EAS Build](https://testingbot.com#expo). |
| `--app-binary-id ` | Reuse the app of a project uploaded earlier (`testingbot upload` or any previous run's project ID) instead of uploading one. Every positional argument is then a flow. See [Reuse an Uploaded App](https://testingbot.com#app-binary-id). |
### Device Options
| CLI Option | Description |
| --- | --- |
| `--device ` | Device name (e.g., "Pixel 9", "iPhone 17") |
| `--platform ` | Platform: Android or iOS |
| `--deviceVersion ` | OS version (e.g., "14", "17.2") |
| `--real-device` | Use a real device instead of emulator/simulator |
| `--device-matrix ` | Run every flow on each listed device: `[:][:real]`, comma-separated or repeatable. Cannot be combined with `--device` or `--deviceVersion`. See [Device Matrix](https://testingbot.com#device-matrix). |
| `--google-play` | Use the Google Play Store-enabled version (Android emulator only) |
| `--orientation ` | Screen orientation: PORTRAIT or LANDSCAPE |
| `--device-locale ` | Device locale (e.g., "en\_US", "de\_DE") |
| `--timezone ` | Timezone (e.g., "America/New\_York", "Europe/London") |
### Test Configuration
| CLI Option | Description |
| --- | --- |
| `--name ` | Name for this Maestro run, used for dashboard identification and build grouping |
| `--groups ` | Tag the test session with one or more groups (comma-separated) |
| `--include-tags ` | Only run flows with these tags (comma-separated) |
| `--exclude-tags ` | Exclude flows with these tags (comma-separated) |
| `--exclude-flows ` | Flow files, directories or glob patterns to leave out of the run (comma-separated, repeatable). An excluded flow that another flow still invokes via `runFlow` is bundled as a subflow but never runs on its own. |
| `-e, --env ` | Environment variable for flows (can be repeated) |
| `--maestro-version ` | Maestro version to use (e.g., "2.0.10") |
| `--config ` | Path to a custom Maestro config file (default: config.yaml in project root) |
### Execution Options
| CLI Option | Description |
| --- | --- |
| `--retry ` | Retry failed flows up to N times (0-2, default 0). Stops as soon as a flow passes. |
| `--async` | Start tests and exit immediately without waiting for results |
| `--dry-run` | Validate and prepare everything but skip the HTTP calls. Shows what would be sent. |
| `-q, --quiet` | Quieter console output without progress updates |
| `--json` | Print results as a single JSON document on stdout (logs move to stderr). Implies `--quiet`. Exit code 2 when flows fail. See [JSON Output](https://testingbot.com#json). |
| `--json-file` | Write the JSON results to a file (default `_testingbot.json`). Exit code stays 0 when flows fail so the pipeline can gate on the file. |
| `--json-file-name ` | Custom path for the JSON results file (requires `--json-file`) |
### CI/CD Integration
| CLI Option | Description |
| --- | --- |
| `--commit-sha ` | Git commit SHA associated with this test run |
| `--pull-request-id ` | Pull request ID this test run originated from |
| `--repo-name ` | Repository name (e.g., GitHub repo slug) |
| `--repo-owner ` | Repository owner (e.g., GitHub organization or username) |
| `--branch ` | Git branch this test run was built from |
| `--pr-url ` | Pull request URL this test run originated from |
| `-m, --metadata ` | Free-form metadata attached to the run and shown in the dashboard (repeatable) |
| `--check-name ` | Name the GitHub pull request check `TestingBot / `, so several runs on one commit (iOS, Android) post separate checks. See [GitHub PR Checks](https://testingbot.com/support/app-automate/maestro/ci-cd/github-pr-checks#one-check). |
With the [TestingBot GitHub App](https://testingbot.com/support/app-automate/maestro/ci-cd/github-pr-checks) installed, a run carrying the repository and full commit SHA also posts a status check on the pull request.
### Network & Location
| CLI Option | Description |
| --- | --- |
| `--throttle-network ` | Network throttling: 4G, 3G, Edge, airplane, or disable |
| `--geo-country-code ` | Geographic IP location (ISO country code, e.g., "US", "DE") |
### Report & Artifacts
| CLI Option | Description |
| --- | --- |
| `--report ` | Download report after completion: html, html-detailed, junit, or allure (Allure result files under `allure-results/`) |
| `--report-output-dir ` | Directory to save reports (required when `--report` is used) |
| `--download-artifacts [mode]` | Download test artifacts (logs, screenshots, video) after completion. Mode: `all` (default) or `failed`. |
| `--artifacts-output-dir ` | Directory to save artifacts zip (defaults to the current directory) |
### Tunnel Options
| CLI Option | Description |
| --- | --- |
| `-t, --tunnel` | Start a [TestingBot Tunnel](https://testingbot.com/support/tunnel) for this test run (cannot be combined with `--async`) |
| `--tunnel-identifier ` | Identifier for the tunnel, allowing multiple tunnels in parallel |
### Advanced Options
| CLI Option | Description |
| --- | --- |
| `--ignore-checksum-check` | Skip checksum verification and always upload the app |
| `--shard-split ` | Split flows into N parallel sessions for faster execution. More information in the [Shard Splitting](https://testingbot.com#shard-split) section. |
### Authentication & Debugging
Credentials are normally resolved from `testingbot login`, the `TB_KEY` / `TB_SECRET` environment variables, or a `~/.testingbot` file. You can also pass them explicitly:
| CLI Option | Description |
| --- | --- |
| `--api-key ` | TestingBot API key |
| `--api-secret ` | TestingBot API secret |
| `--debug` | Enable debug logging of API responses |
### Other Commands
Besides `maestro`, the CLI has commands for working with projects after they were started. All of them accept the authentication and `--json` flags above.
| Command | Description |
| --- | --- |
| `testingbot upload ` | Upload an app once and print a project ID to reuse with `--app-binary-id` |
| `testingbot status --id [--wait]` | Show every run and flow of a project; `--wait` blocks with the live flow table until it finishes (Ctrl-C detaches without cancelling) |
| `testingbot artifacts --id ` | Download `--report` and/or `--download-artifacts` for a finished project |
| `testingbot list [--count] [--offset]` | List recent Maestro projects, newest first |
| `testingbot login` | Authenticate via the browser and store the credentials in `~/.testingbot` |
See [Async runs, status and artifacts](https://testingbot.com/support/app-automate/maestro/ci-cd#async) for a pipeline example.
## Frequently asked questions
How do I pin a specific Maestro version on TestingBot?
Pass the `maestroVersion` option in your run configuration. See the [Set Maestro Version section](https://testingbot.com#version) for the list of supported versions and CLI / API examples.
How do I pass environment variables to Maestro flows?
Use the `env` option to inject environment variables into the Maestro run, and `secretEnv` for credentials, whose values are stored encrypted and masked in the API and the reports. The flows read both with the `${VARIABLE_NAME}` syntax inside the YAML. See the [Environment Variables section](https://testingbot.com#environment) for examples.
Can I run only a subset of my Maestro flows?
Yes. Use `flows` to whitelist specific files, or `includeTags` and `excludeTags` to filter by Maestro tag. Only the flows that survive the filter are scheduled, so the rest do not consume a device session, and a filter that matches nothing is rejected rather than run. See the [Select Specific Flows](https://testingbot.com#flows), [Include Tags](https://testingbot.com#include-tags) and [Exclude Tags](https://testingbot.com#exclude-tags) sections.
How do I split Maestro flows into parallel shards?
Pass `shardSplit` with the number of parallel sessions you want. Each shard runs a subset of the flows on its own device session. See the [Shard Split section](https://testingbot.com#shard-split) for details.
Can I change the device timezone, locale or geolocation for Maestro tests?
Yes. Use the `timezone`, `locale` and `geolocation` options to control the device environment. See the [Device Timezone](https://testingbot.com#timezone), [Device Locale](https://testingbot.com#locale) and [IP Geolocation](https://testingbot.com#geolocation) sections.
Can I test a private or staging server with Maestro on TestingBot?
Yes. Start the [TestingBot Tunnel](https://testingbot.com/support/tunnel) and pass `tunnelIdentifier` in your Maestro run config. The device will route traffic through your tunnel so it can reach internal APIs. See the [TestingBot Tunnel section](https://testingbot.com#tunnel).
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/app-automate/maestro/ci-cd
# CI/CD Integration
The [TestingBot CLI](https://testingbot.com/support/app-automate/maestro#cli) integrates seamlessly with any CI/CD environment. It runs all your Maestro flows, streams the output in real-time and exits with the appropriate exit code for pipeline automation.
We recommend using `npx` to run the CLI without installing it globally. This ensures you're always using the latest version.
## Quick Start
Run Maestro tests in any CI/CD service with a single command:
```bash
npx --yes @testingbot/cli@latest maestro app.apk ./flows \
--device "Pixel 8" \
--deviceVersion "14" \
--api-key $TB_KEY \
--api-secret $TB_SECRET
```
This command will:
- Upload your mobile app (`.apk`, `.aab`, `.ipa` or `.app`)
- Upload and run all Maestro flow files from the `./flows` directory
- Stream test progress to your CI logs
- Exit with code `0` on success, non-zero on failure
You can pass credentials as arguments (`--api-key` and `--api-secret`) or set them as environment variables. Environment variables are the recommended approach for CI/CD.
## Environment Variables
Instead of passing credentials as command arguments, you can set them as environment variables. This is the most secure approach for CI/CD pipelines since credentials are never exposed in logs or command history.
| Variable | Description |
| --- | --- |
| `TB_KEY` | Your TestingBot API key |
| `TB_SECRET` | Your TestingBot API secret |
When these environment variables are set, you can omit the credential arguments entirely:
```bash
export TB_KEY="your_api_key"
export TB_SECRET="your_api_secret"
npx --yes @testingbot/cli@latest maestro app.apk ./flows \
--device "Pixel 8" \
--deviceVersion "14"
```
Never hardcode credentials in your CI configuration files. Always use your CI provider's secrets management feature to store `TB_KEY` and `TB_SECRET`.
## Exit Codes
The CLI distinguishes a red test run from a broken pipeline, so a job can retry on infrastructure errors but fail the build on real test failures:
| Exit Code | Description |
| --- | --- |
| `0` | All flows passed. Also returned for `--async`, `--dry-run`, and for failed flows when `--json-file` is used (the file is the contract then) |
| `1` | CLI or infrastructure error: invalid arguments, missing credentials, upload failure, timeout |
| `2` | One or more flows failed |
`testingbot status` follows the same rule and additionally exits `0` while a project is still running.
## Test Reports
Generate test reports for CI/CD result decoration and test analytics. The CLI supports JUnit XML, HTML and Allure report formats, plus a machine-readable JSON summary.
### JUnit Report
Generate JUnit XML reports for integration with CI/CD test result visualization:
```bash
npx --yes @testingbot/cli@latest maestro app.apk ./flows \
--device "Pixel 8" \
--deviceVersion "14" \
--report junit \
--report-output-dir ./test-results
```
| Option | Description |
| --- | --- |
| `--report ` | Report format: `junit` |
| `--report-output-dir ` | Directory to save test reports (required when `--report` is used) |
JUnit reports can be used with most CI/CD platforms for:
- Test result visualization in your pipeline
- Tracking test trends over time
- Annotating pull requests with test results
- Integration with test management tools
You can also fetch reports via the [REST API](https://testingbot.com/support/app-automate/maestro/api#report) if you need to retrieve them separately.
### Allure Report
`--report allure` converts each run's results into Allure result files under `/allure-results/`, one JSON per flow with its steps, timings, status and failure details. Render them with the [Allure CLI](https://allurereport.org/docs/v2/install/); results from several runs or shards accumulate in the same directory.
```bash
npx --yes @testingbot/cli@latest maestro app.apk ./flows --report allure --report-output-dir ./reports
allure serve ./reports/allure-results
```
### JSON Output
`--json` prints a single JSON document on stdout and moves all log lines to stderr, so it pipes straight into `jq`. `--json-file` writes the same document to `_testingbot.json` (or the path given with `--json-file-name`) while keeping the normal console output. Both imply `--quiet`.
```bash
npx --yes @testingbot/cli@latest maestro app.apk ./flows --json | jq '.runs[].flows[] | {name, passed, durationSeconds}'
```
```json
{
"provider": "maestro",
"outcome": "failed",
"success": false,
"appId": 1234,
"url": "https://testingbot.com/members/maestro/1234",
"runs": [
{
"id": 5678,
"status": "DONE",
"passed": false,
"device": { "name": "Pixel 9", "platform": "Android", "version": "14" },
"url": "https://testingbot.com/members/maestro/1234/runs/5678",
"flows": [
{ "id": 1, "runId": 5678, "name": "login", "status": "DONE", "passed": true, "attempt": 1, "latest": true, "durationSeconds": 30, "errors": [] }
]
}
]
}
```
`outcome` is one of `passed`, `failed`, `started` (async), `dry-run` or `error`. `flows` lists every attempt, including `--retry` re-runs; `latest` marks the attempt whose verdict counts. When the CLI fails before a verdict, the document still arrives with `outcome: "error"` and an `error` message, so a consumer never has to handle missing output.
## CI/CD Metadata
Pass repository and commit information to link test runs with your source control. This enables better traceability and integration with your development workflow.
| Option | Description |
| --- | --- |
| `--commit-sha ` | The commit SHA of this upload |
| `--pull-request-id ` | The ID of the pull request this upload originated from |
| `--repo-name ` | Repository name (e.g., GitHub repo slug) |
| `--repo-owner ` | Repository owner (e.g., GitHub organization or user) |
| `--branch ` | Git branch this run was built from |
| `--pr-url ` | URL of the pull request this run originated from |
| `-m, --metadata ` | Free-form metadata attached to the run (repeatable), for example `-m team=mobile -m env=staging` |
Example with metadata:
```bash
npx --yes @testingbot/cli@latest maestro app.apk ./flows \
--device "Pixel 8" \
--deviceVersion "14" \
--commit-sha "$COMMIT_SHA" \
--branch "$BRANCH" \
--pull-request-id "$PR_NUMBER" \
--repo-owner "your-org" \
--repo-name "your-repo" \
-m team=mobile
```
With the [TestingBot GitHub App](https://testingbot.com/support/app-automate/maestro/ci-cd/github-pr-checks) installed, a run that carries the repository and the full head commit SHA also posts a `TestingBot / tests` status check on the pull request, so you can require green Maestro flows before merging. No extra step is needed.
## Async runs, status and artifacts
Long suites do not have to hold a CI runner. Start the run with `--async`, capture the project ID, and come back for the result in a later job or from a different machine.
```bash
# Start and capture the project id
PROJECT_ID=$(npx --yes @testingbot/cli@latest maestro app.apk ./flows --async --json | jq -r .appId)
# Later: block until it finishes (exit 2 if any flow failed), with the live flow table
npx --yes @testingbot/cli@latest status --id "$PROJECT_ID" --wait
# Fetch reports and artifacts once it finished
npx --yes @testingbot/cli@latest artifacts --id "$PROJECT_ID" --report junit --report-output-dir ./reports
npx --yes @testingbot/cli@latest artifacts --id "$PROJECT_ID" --download-artifacts failed --artifacts-output-dir ./artifacts
# Browse recent projects
npx --yes @testingbot/cli@latest list --count 20
```
`status --wait` only watches the project; interrupting it does not cancel the run. `status` without `--wait` prints a snapshot and exits `0` while the project is still running, so it also works for polling.
### Expo / EAS Build
Pass an `eas build` download URL with `--app-url`, or run inside EAS Workflows with the `@testingbot/eas-workflow` wrapper. Both are covered on the [Expo / EAS page](https://testingbot.com/support/app-automate/maestro/ci-cd/expo-eas).
### Upload the app once, run many suites
Every run prints its project ID after the app upload. Pass it to a later run with `--app-binary-id` to skip the upload entirely; each run still gets its own project and results. `testingbot upload` does the upload on its own and prints the ID.
```bash
APP_ID=$(npx --yes @testingbot/cli@latest upload app.apk --json | jq -r .appId)
npx --yes @testingbot/cli@latest maestro --app-binary-id "$APP_ID" ./flows/smoke --device "Pixel 9"
npx --yes @testingbot/cli@latest maestro --app-binary-id "$APP_ID" ./flows/regression --device-matrix "Pixel 9:14,Samsung Galaxy S24:14:real"
```
## View Result Details
A link to the current test results will be printed to your CI logs during execution.
The test details page provides detailed test result information, including:
- Real-time test execution status
- Video recordings of each test run
- Screenshots captured during tests
- Device logs and Maestro output
- Test artifacts for debugging
## GitHub Actions
The [TestingBot Maestro GitHub Action](https://testingbot.com/support/app-automate/maestro/ci-cd/github-actions) wraps the CLI, fills in branch, commit, repository and pull request from the workflow context, writes a results table to the job summary and exposes the results as outputs:
```yaml
- uses: testingbot/maestro-action@v1
with:
api-key: ${{ secrets.TB_KEY }}
api-secret: ${{ secrets.TB_SECRET }}
app-file: app/build/outputs/apk/debug/app-debug.apk
workspace: .maestro
```
Inputs, outputs, a device-matrix example and the raw-CLI alternative are on the [GitHub Actions page](https://testingbot.com/support/app-automate/maestro/ci-cd/github-actions).
## GitHub PR status checks
With the TestingBot GitHub App installed, every Maestro run started from a pull request posts a `TestingBot / tests` check listing each flow on each device, and can be made a required check so a pull request cannot be merged while a flow is failing. Nothing to configure beyond installing the app. See [GitHub PR Checks for Maestro](https://testingbot.com/support/app-automate/maestro/ci-cd/github-pr-checks).
## GitLab CI
Add Maestro testing to your `.gitlab-ci.yml`. Store your credentials in **Settings → CI/CD → Variables** and mark them as protected and masked.
```yaml
stages:
- build
- test
build:
stage: build
image: gradle:8-jdk17
script:
- ./gradlew assembleDebug
artifacts:
paths:
- app/build/outputs/apk/debug/
maestro-tests:
stage: test
image: node:20
dependencies:
- build
script:
- npx --yes @testingbot/cli@latest maestro
app/build/outputs/apk/debug/app-debug.apk
./maestro-flows
--device "Pixel 8"
--deviceVersion "14"
--report junit
--report-output-dir ./test-results
--commit-sha $CI_COMMIT_SHA
--pull-request-id $CI_MERGE_REQUEST_IID
--repo-owner $CI_PROJECT_NAMESPACE
--repo-name $CI_PROJECT_NAME
variables:
TB_KEY: $TB_KEY
TB_SECRET: $TB_SECRET
artifacts:
when: always
paths:
- test-results/
reports:
junit: test-results/*.xml
```
Add `TB_KEY` and `TB_SECRET` as CI/CD variables in your GitLab project settings. Enable "Mask variable" to prevent them from appearing in job logs.
## Jenkins
Add Maestro testing to your Jenkins pipeline (`Jenkinsfile`). Store your credentials using **Manage Jenkins → Manage Credentials** as secret text.
```groovy
pipeline {
agent any
environment {
TB_KEY = credentials('testingbot-api-key')
TB_SECRET = credentials('testingbot-api-secret')
}
stages {
stage('Build') {
steps {
sh './gradlew assembleDebug'
}
}
stage('Maestro Tests') {
steps {
sh """
npx --yes @testingbot/cli@latest maestro \\
app/build/outputs/apk/debug/app-debug.apk \\
./maestro-flows \\
--device "Pixel 8" \\
--deviceVersion "14" \\
--report junit \\
--report-output-dir test-results \\
--commit-sha ${env.GIT_COMMIT} \\
--pull-request-id ${env.CHANGE_ID ?: ''} \\
--repo-owner "your-org" \\
--repo-name "your-repo"
"""
}
post {
always {
junit 'test-results/*.xml'
archiveArtifacts artifacts: 'test-results/**', allowEmptyArchive: true
}
}
}
}
}
```
Create credentials of type "Secret text" in Jenkins for `TB_KEY` and `TB_SECRET`. Reference them using the `credentials()` helper in your pipeline.
## CircleCI
Add Maestro testing to your `.circleci/config.yml`. Store your credentials in **Project Settings → Environment Variables**.
```yaml
version: 2.1
jobs:
maestro-tests:
docker:
- image: cimg/node:20
steps:
- checkout
- run:
name: Run Maestro Tests
command: |
npx --yes @testingbot/cli@latest maestro \
./app.apk \
./maestro-flows \
--device "Pixel 8" \
--deviceVersion "14" \
--report junit \
--report-output-dir test-results \
--commit-sha $CIRCLE_SHA1 \
--pull-request-id $CIRCLE_PR_NUMBER \
--repo-owner $CIRCLE_PROJECT_USERNAME \
--repo-name $CIRCLE_PROJECT_REPONAME
- store_test_results:
path: test-results
- store_artifacts:
path: test-results
destination: maestro-results
workflows:
test:
jobs:
- maestro-tests
```
Add `TB_KEY` and `TB_SECRET` as environment variables in your CircleCI project settings. CircleCI automatically masks these values in build output.
## Bitbucket Pipelines
Add Maestro testing to your `bitbucket-pipelines.yml`. Store your credentials in **Repository settings → Pipelines → Repository variables** and enable "Secured".
```yaml
image: node:20
pipelines:
default:
- step:
name: Build and Test
caches:
- node
script:
- ./gradlew assembleDebug
- npx --yes @testingbot/cli@latest maestro
app/build/outputs/apk/debug/app-debug.apk
./maestro-flows
--device "Pixel 8"
--deviceVersion "14"
--report junit
--report-output-dir test-results
--commit-sha $BITBUCKET_COMMIT
--pull-request-id $BITBUCKET_PR_ID
--repo-owner $BITBUCKET_WORKSPACE
--repo-name $BITBUCKET_REPO_SLUG
artifacts:
- test-results/**
```
Add `TB_KEY` and `TB_SECRET` as secured repository variables in Bitbucket. Secured variables are encrypted and masked in build logs.
## Frequently asked questions
Which CI/CD systems can run Maestro tests on TestingBot?
Any CI runner that can execute a shell command. The page below has copy-paste-ready examples for [GitHub Actions](https://testingbot.com#github), [GitLab CI](https://testingbot.com#gitlab), [Jenkins](https://testingbot.com#jenkins), [CircleCI](https://testingbot.com#circleci) and [Bitbucket Pipelines](https://testingbot.com#bitbucket). The same TestingBot CLI works on any other CI runner with a shell.
How do I store TestingBot credentials in CI securely?
Set TB\_KEY and TB\_SECRET as encrypted CI secrets. GitHub Actions uses repository or environment secrets, GitLab CI uses protected variables, Jenkins uses credentials, CircleCI uses environment variables in contexts, and Bitbucket uses secured repository variables. Never commit the key or secret in plain text.
What exit codes does the TestingBot CLI return?
Exit code 0 when every flow passed, 2 when one or more flows failed, and 1 for a CLI or infrastructure error such as a failed upload or missing credentials. See the [Exit Codes section](https://testingbot.com#exitcodes) for the full mapping so you can wire conditional follow-up steps in CI.
How do I publish Maestro test reports in CI?
The CLI writes JUnit, HTML or Allure reports that most CI runners can parse to show per-flow results, and prints a machine-readable JSON document with --json. See the [Test Reports section](https://testingbot.com#reports) for the file path and configuration in each CI system.
How do I link CI metadata (branch, commit, pull request) to TestingBot test runs?
Pass --repo-owner, --repo-name, --branch, --commit-sha, --pull-request-id and --pr-url flags to the CLI, or use the TestingBot Maestro GitHub Action which fills them in from the workflow context. The values appear in the TestingBot dashboard, and with the TestingBot GitHub App installed they also post a status check on the pull request. See the [CI/CD Metadata section](https://testingbot.com#metadata).
Can I check on an async Maestro run later from CI?
Yes. Start the run with --async, then use `testingbot status --id --wait` in a later job to block until it finishes, and `testingbot artifacts` to fetch reports and videos. See the [Async runs section](https://testingbot.com#async).
---
URL: https://testingbot.com/support/app-automate/maestro/ci-cd/github-actions
# Maestro on GitHub Actions
The [TestingBot Maestro GitHub Action](https://github.com/marketplace/actions/testingbot-maestro) runs your Maestro flows on TestingBot real devices, emulators and simulators from a workflow. It wraps the [TestingBot CLI](https://testingbot.com/support/app-automate/maestro#cli), fills in branch, commit, repository and pull request from the workflow context, writes a results table to the job summary and exposes every result as an output. Store your credentials in **Settings → Secrets and variables → Actions** as repository secrets.
## Workflow
```yaml
name: Maestro Tests
on:
push:
branches: [main]
pull_request:
branches: [main]
jobs:
maestro-tests:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Build Android App
run: ./gradlew assembleDebug
- name: Run Maestro Tests on TestingBot
id: maestro
uses: testingbot/maestro-action@v1
with:
api-key: ${{ secrets.TB_KEY }}
api-secret: ${{ secrets.TB_SECRET }}
app-file: app/build/outputs/apk/debug/app-debug.apk
workspace: .maestro
device: Pixel 9
device-version: '14'
report: junit
download-artifacts: failed
- name: Upload Test Results
uses: actions/upload-artifact@v4
if: always()
with:
name: maestro-results
path: |
testingbot-reports
testingbot-artifacts
```
The action fails the step with exit code `2` when a flow fails and `1` on CLI or infrastructure errors. Its outputs let later steps react to the result:
| Output | Description |
| --- | --- |
| `app-id` | Project ID; pass it as `app-binary-id` in later jobs to skip the upload |
| `console-url` | Link to the results in the TestingBot dashboard |
| `outcome` | `passed`, `failed`, `started` (async), `dry-run` or `error` |
| `flow-results` | JSON array of per-flow results with name, status, device, duration and errors |
| `results-file` | Path to the full JSON document written by the CLI |
Every CLI feature is available as an input: `device-matrix`, `real-device`, `include-tags`, `exclude-flows`, `env`, `retry`, `shard-split`, `async` and more. The inputs mirror the Maestro Cloud action, so migrating is usually a one-line change; see the [action README](https://github.com/testingbot/maestro-action#readme) for the full list.
Prefer the CLI directly? The same run without the action:
```yaml
- uses: actions/setup-node@v4
with:
node-version: '22'
- name: Run Maestro Tests
run: |
npx --yes @testingbot/cli@latest maestro \
app/build/outputs/apk/debug/app-debug.apk ./maestro-flows \
--device "Pixel 9" --deviceVersion "14" \
--report junit --report-output-dir ./test-results \
--commit-sha ${{ github.event.pull_request.head.sha || github.sha }} \
--branch ${{ github.head_ref || github.ref_name }} \
--pull-request-id ${{ github.event.pull_request.number }} \
--repo-owner ${{ github.repository_owner }} \
--repo-name ${{ github.event.repository.name }}
env:
TB_KEY: ${{ secrets.TB_KEY }}
TB_SECRET: ${{ secrets.TB_SECRET }}
```
Add `TB_KEY` and `TB_SECRET` as repository secrets in your GitHub repository settings. Never commit credentials to your repository.
## Triggers and pull requests from forks
Run on pushes to your default branch and on pull requests. For `pull_request` events the action uses the pull request's head commit, which is what branch protection evaluates. Pull requests from forks cannot read repository secrets under `pull_request`; if you need them, use `pull_request_target` and check out the PR head explicitly so the tests run against the proposed code rather than the base branch:
```yaml
on:
push:
branches: [main]
pull_request_target:
branches: [main]
jobs:
maestro:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha }}
```
A Linux runner is enough for iOS too: the tests run on TestingBot's simulators and devices, not on the runner, so no macOS minutes are needed unless you build the app in the same job.
## Android, iOS and Expo apps
| Platform | `app-file` |
| --- | --- |
| Android | An `.apk`, for example `app/build/outputs/apk/debug/app-debug.apk`. Emulators by default; add `real-device: true` for a physical device. |
| iOS simulator | A simulator `.app` bundle or a `.zip` of it (`xcodebuild -sdk iphonesimulator`). |
| iOS real device | An `.ipa`. Real devices are selected automatically for `.ipa` files. |
| Expo / EAS | Download the build in a previous step and pass its path, or pass the `eas build` URL with `extra-args: --app-url `. For EAS Workflows use the [EAS integration](https://testingbot.com/support/app-automate/maestro/ci-cd/expo-eas). |
## Inputs
| Input | Description |
| --- | --- |
| `api-key`, `api-secret` | TestingBot credentials. Required. |
| `app-file` | App under test (`.apk`, `.ipa`, `.app`, `.zip`, `.tar.gz`). Either this or `app-binary-id`. |
| `app-binary-id` | Project ID of an app uploaded earlier (the `app-id` output); skips the upload. |
| `workspace` | Flows directory, file or glob; several entries one per line. Default `.maestro`. |
| `name` | Run name in the dashboard. Defaults to the PR title, then the commit message. |
| `async` | Submit and return immediately; the step never fails on results. |
| `timeout` | Minutes to wait for results before cancelling and failing. Default 60. |
| `env` | Flow environment variables, one `KEY=VALUE` per line. |
| `include-tags`, `exclude-tags`, `exclude-flows` | Flow selection, comma-separated. |
| `device`, `device-version`, `platform`, `real-device` | Single device selection. Platform is detected from the app when omitted. |
| `device-matrix` | Run every flow on each listed device, `[:][:real]`, one per line. Not with `device`. |
| `device-locale`, `timezone`, `orientation`, `google-play` | Device settings. |
| `retry`, `shard-split`, `maestro-version` | Retries (0-2), parallel shards per device, Maestro version. |
| `report`, `report-output-dir` | `junit`, `html`, `html-detailed` or `allure`, saved to `testingbot-reports` by default. |
| `download-artifacts`, `artifacts-output-dir` | `all` or `failed`: logs, screenshots and video as a zip. |
| `other-app`, `metadata`, `groups`, `throttle-network`, `geo-country-code`, `tunnel`, `tunnel-identifier` | Companion apps, free-form metadata, dashboard groups, network profile, IP geolocation and the TestingBot Tunnel. |
| `check-name` | Names the pull request check `TestingBot / `, for one check per job on the same commit (`iOS`, `Android`). Keep it stable per job. |
| `cli-version`, `dry-run`, `extra-args` | Pin the CLI version, validate without running, or append raw CLI arguments. |
The [action README](https://github.com/testingbot/maestro-action#readme) lists every input with its default.
## Outputs
Give the step an `id` and read its outputs in later steps or jobs:
| Output | Description |
| --- | --- |
| `app-id` | TestingBot project ID. Pass it as `app-binary-id` to skip the upload in a later job. |
| `console-url` | Link to the results in the TestingBot dashboard. |
| `outcome` | `passed`, `failed`, `started` (async), `dry-run` or `error`. |
| `flow-results` | JSON array with one entry per flow attempt: `name`, `status`, `passed`, `attempt`, `latest`, `device`, `durationSeconds`, `errors`, `url`. |
| `results-file` | Path to the full JSON document the CLI wrote. |
### Post the result to Slack
```yaml
- id: maestro
uses: testingbot/maestro-action@v1
with:
api-key: ${{ secrets.TB_KEY }}
api-secret: ${{ secrets.TB_SECRET }}
app-file: app.apk
- uses: slackapi/slack-github-action@v2
if: always()
with:
method: chat.postMessage
token: ${{ secrets.SLACK_BOT_TOKEN }}
payload: |
channel: "C0123456789"
text: "Maestro on TestingBot: ${{ steps.maestro.outputs.outcome }} — ${{ steps.maestro.outputs.console-url }}"
```
### Keep the results as a workflow artifact
```yaml
- uses: actions/upload-artifact@v4
if: always()
with:
name: maestro-results
path: |
${{ steps.maestro.outputs.results-file }}
testingbot-reports
testingbot-artifacts
```
The job summary always gets a results table with a link per flow, so most runs need no extra reporting step.
## Examples
### Upload once, run two suites on different devices
```yaml
- id: smoke
uses: testingbot/maestro-action@v1
with:
api-key: ${{ secrets.TB_KEY }}
api-secret: ${{ secrets.TB_SECRET }}
app-file: app.apk
workspace: .maestro/smoke
device: Pixel 9
- uses: testingbot/maestro-action@v1
with:
api-key: ${{ secrets.TB_KEY }}
api-secret: ${{ secrets.TB_SECRET }}
app-binary-id: ${{ steps.smoke.outputs.app-id }}
workspace: .maestro/regression
device-matrix: |
Pixel 9:14
Samsung Galaxy S24:14:real
```
### Pass secrets to your flows
```yaml
- uses: testingbot/maestro-action@v1
with:
api-key: ${{ secrets.TB_KEY }}
api-secret: ${{ secrets.TB_SECRET }}
app-file: app.apk
env: |
USERNAME=${{ secrets.TEST_USERNAME }}
PASSWORD=${{ secrets.TEST_PASSWORD }}
include-tags: smoke,critical
exclude-tags: wip
```
Flows read them as `${USERNAME}`. GitHub masks secret values in the logs; the action never prints the `env` values itself.
### Fire and forget on pull requests
```yaml
- uses: testingbot/maestro-action@v1
with:
api-key: ${{ secrets.TB_KEY }}
api-secret: ${{ secrets.TB_SECRET }}
app-file: app.apk
async: true
```
The step returns within seconds and the runner is free while the flows execute on TestingBot. Check on the run later with `testingbot status --id --wait` or in the dashboard. With the TestingBot GitHub App installed the pull request still gets its [status check](https://testingbot.com/support/app-automate/maestro/ci-cd/github-pr-checks) when the run finishes, so merges can be gated without holding a runner.
## Pull request status checks
The action passes the repository and the pull request's head commit along, so with the [TestingBot GitHub App](https://testingbot.com/support/app-automate/maestro/ci-cd/github-pr-checks) installed a `TestingBot / tests` check appears on the pull request, listing each flow on each device, and can be made a required check. Running iOS and Android as separate jobs? Set `check-name` on each so they post `TestingBot / iOS` and `TestingBot / Android`.
## Migrating from the Maestro Cloud action
Swap the action and the credentials; the other inputs keep their names. There is no `project-id`.
```yaml
- uses: testingbot/maestro-action@v1
with:
api-key: ${{ secrets.TB_KEY }}
api-secret: ${{ secrets.TB_SECRET }}
app-file: app.zip
workspace: .maestro
include-tags: smoke
```
Outputs map as `MAESTRO_CLOUD_CONSOLE_URL` → `console-url`, `MAESTRO_CLOUD_APP_BINARY_ID` → `app-id`, `MAESTRO_CLOUD_UPLOAD_STATUS` → `outcome` and `MAESTRO_CLOUD_FLOW_RESULTS` → `flow-results`. The full flag mapping is on the [migration page](https://testingbot.com/support/app-automate/maestro/migrate-from-maestro-cloud).
## Frequently asked questions
### How do I run Maestro tests on TestingBot from GitHub Actions?
Add the `testingbot/maestro-action@v1` step with your API key and secret as repository secrets, the app file and the flows directory. The action uploads the app and flows, runs them on TestingBot and fails the step when a flow fails. See the [workflow example](https://testingbot.com#workflow).
### Can the action reuse an app uploaded in an earlier step?
Yes. Every run exposes an `app-id` output; pass it as `app-binary-id` in a later step or job to skip the upload. See the [examples](https://testingbot.com#examples).
### Does the action post a status check on the pull request?
Yes, when the TestingBot GitHub App is installed for the repository. The action passes the head commit and repository along and the check appears automatically. See [GitHub PR Checks for Maestro](https://testingbot.com/support/app-automate/maestro/ci-cd/github-pr-checks).
### Which exit codes does the action step return?
The same as the CLI: 2 when one or more flows failed, 1 on a CLI or infrastructure error, 0 otherwise, including `async` and `dry-run`.
### Do I need a macOS runner for iOS?
No. The flows run on TestingBot's simulators and real devices, so `ubuntu-latest` is enough for both platforms. Use a macOS runner only if you build the iOS app in the same job.
### How do I run tests on pull requests from forks?
Forked pull requests cannot read repository secrets under `pull_request`. Use `pull_request_target` and check out `github.event.pull_request.head.sha` explicitly. See [triggers](https://testingbot.com#triggers).
### Is it a drop-in replacement for the Maestro Cloud action?
Mostly. Swap the `uses` line and the credentials; `app-file`, `workspace`, `env`, tags, `async` and the other inputs keep their names. See [migrating](https://testingbot.com#migrate).
---
URL: https://testingbot.com/support/app-automate/maestro/ci-cd/github-pr-checks
# GitHub PR Checks for Maestro
With the TestingBot GitHub App installed, every Maestro run started from a pull request posts a native `TestingBot / tests` status check on it: each flow on each device, failures first, with a link to the run in the TestingBot dashboard. Make it a [required check](https://testingbot.com#merge-gate) and a pull request cannot be merged while a Maestro flow is failing. There is nothing to register or resolve from your workflow; the [Maestro GitHub Action](https://testingbot.com/support/app-automate/maestro/ci-cd/github-actions) and the [TestingBot CLI](https://testingbot.com/support/app-automate/maestro#cli) pass the commit along and TestingBot finalizes the check when every flow has finished.
## Install the GitHub App
1. In the members area, open [Integrations](https://testingbot.com/members/integrations) and choose **GitHub PR Checks**.
2. Click **Install the TestingBot GitHub App**. You are redirected to GitHub.
3. Pick the account or organization and the repositories to enable, then install. You are redirected back and the integration shows as **Connected**.
Install through TestingBot rather than from GitHub's app settings, so the installation is linked to your TestingBot account. Only account owners and admins can connect or disconnect the integration. Sub-accounts of a team share the owner's installation: a run started with a sub-account's API key posts checks as long as the app is installed under the team owner.
## Repositories and disconnecting
Checks appear only on repositories the app can reach. The integration page shows whether the app covers all repositories or a selection, with a link to change the selection on GitHub; changes apply immediately because TestingBot listens to GitHub's installation events. Disconnecting from the integration page stops checks; reconnecting later reuses the existing installation.
## Run from a pull request
With the Maestro GitHub Action a single step is enough; it fills in the repository, head commit, branch and pull request from the workflow context:
```yaml
- uses: testingbot/maestro-action@v1
with:
api-key: ${{ secrets.TB_KEY }}
api-secret: ${{ secrets.TB_SECRET }}
app-file: app/build/outputs/apk/debug/app-debug.apk
workspace: .maestro
```
From any other CI, pass the metadata flags to the CLI. The commit SHA must be the full 40-character head commit of the pull request, not the merge commit:
```bash
npx --yes @testingbot/cli@latest maestro app.apk ./flows \
--repo-owner "$REPO_OWNER" --repo-name "$REPO_NAME" \
--commit-sha "$HEAD_SHA" --branch "$BRANCH" \
--pull-request-id "$PR_NUMBER"
```
Async runs are a good fit: with `--async` (or `async: true` on the action) the CI step returns within seconds, the flows run on TestingBot, and the check is finalized when they finish, so merges are gated without holding a runner. The CLI prints `GitHub check registered for owner/repo@sha` when the link was made. If that line is missing, one of the values was absent, the commit SHA was not the full head SHA, or the app is not installed for the repository owner. In EAS Workflows the [eas-workflow wrapper](https://testingbot.com/support/app-automate/maestro/ci-cd/expo-eas) sets the same flags from `TB_GH_*` variables.
## Pull request comment
Alongside the check, TestingBot maintains one self-updating comment on the pull request with the same device, flow and result table. Pushing new commits updates the comment rather than adding new ones.
## Require green flows before merging
1. In your GitHub repository, go to **Settings** → **Branches** (or **Rules** → **Rulesets** ).
2. Add a rule for your default branch and enable **Require status checks to pass before merging**.
3. Select `TestingBot / tests` as a required check. GitHub lists it once it has run on the branch at least once in the last 7 days, so open one pull request first.
The check is green only when every run finished and every flow passed. A cancelled run, a failed flow or a run that never completed all conclude as failing, so the merge stays blocked until the tests are genuinely green. Flows that failed and then passed on a `--retry` count as passed; only the newest attempt per flow is judged, the same rule the dashboard and the CLI exit code use. With `--device-matrix` every device is a row in the same check.
## Several runs on one pull request
Each Maestro run posts one check, named `TestingBot / tests` by default. When more than one run tests the same commit, an iOS job and an Android job for example, give each a check name so they post separate checks that branch protection can require individually: `--check-name iOS` on the CLI, `check-name: iOS` on the action, or `TB_CHECK_NAME=iOS` with the EAS wrapper. The check is then called `TestingBot / iOS` and gets its own pull request comment.
```yaml
jobs:
ios:
steps:
- uses: testingbot/maestro-action@v1
with:
app-file: app.zip
check-name: iOS
android:
steps:
- uses: testingbot/maestro-action@v1
with:
app-file: app.apk
check-name: Android
```
Keep the name stable per job: GitHub matches required checks by name, so a name derived from the commit or run number would create a new check every time and could never be required. Two runs on the same commit with the same name share one check, and the last to finish decides its result. To cover several devices in one job, prefer `--device-matrix`; every device becomes a row in the same check.
## Troubleshooting
- **No check appears.** Look for `GitHub check registered` in the CLI output. Without it, check that the repository owner and name and a full 40-character commit SHA were passed (the Maestro GitHub Action does this for you) and that the app is installed for the repository owner.
- **The check is on the wrong commit.** For `pull_request` events pass `github.event.pull_request.head.sha`; `github.sha` is the merge commit, which is not part of the pull request.
- **The check is stuck as in-progress.** It finalizes when every run and flow has settled. If a run was cancelled or lost, a watchdog concludes the check as failing after about 50 minutes so the pull request is never left waiting.
- **The check is not selectable as a required check.** GitHub only offers it after it has run at least once on the protected branch in the last 7 days.
- **The run was started with another team's key.** The check posts only when the API key belongs to the team that installed the app. Check which key your CI secrets hold.
- **Re-running after a failure.** The check carries a **Re-run analysis** button that re-evaluates the current results. To re-run the flows themselves, use `--retry` on the run or re-run the CI job; each new run updates the check.
Browser and Appium builds use the same app with a different registration flow; see [GitHub PR Checks](https://testingbot.com/support/integrations/ci-cd/github-pr-checks).
## Frequently asked questions
### Do I need to register the build from my workflow?
No. Unlike browser builds, a Maestro run registers itself: when it carries the repository and the pull request head commit, and the app is installed, the check is created when the run starts and finalized when every flow has finished.
### When is the check green?
Only when every run finished and every flow passed. Cancelled, failed or incomplete runs conclude as failing so the merge stays blocked. A flow that passed on a retry counts as passed. See [the merge gate](https://testingbot.com#merge-gate).
### Does it work with --async?
Yes. The check is created when the run starts and finalized by TestingBot when the run completes, whether or not the CLI is still waiting.
### Can I use it from CI other than GitHub Actions?
Yes. Pass `--repo-owner`, `--repo-name`, the full `--commit-sha` and optionally `--branch` and `--pull-request-id` to the CLI from any CI. See [run from a pull request](https://testingbot.com#run).
### Can I have one check per platform on the same pull request?
Yes. Give each run a check name (`--check-name iOS`, the action's `check-name` input or `TB_CHECK_NAME` in EAS) and it posts as `TestingBot / iOS` with its own comment, so branch protection can require each platform separately. See [several runs on one pull request](https://testingbot.com#one-check).
### Why does my Maestro check not appear?
The CLI prints `GitHub check registered` when the link was made. Without that line, the metadata was incomplete, the commit SHA was not the full head SHA, or the app is not installed for the repository owner. See [troubleshooting](https://testingbot.com#troubleshooting).
---
URL: https://testingbot.com/support/app-automate/maestro/ci-cd/expo-eas
# Maestro tests for Expo apps on TestingBot
With Expo there is no local binary: `eas build` produces a download URL, a `.tar.gz` for iOS simulator builds and an `.apk` for Android. The TestingBot CLI takes that URL directly, and inside EAS Workflows a wrapper turns the results into step outputs. Both paths run your flows on TestingBot real devices, emulators and simulators with no Maestro Cloud account.
## Prerequisites and secrets
- An Expo project with an EAS build profile for the platform you test. iOS flows need a **simulator** build (`"ios": { "simulator": true }` in `eas.json`) unless you run on real devices with an `.ipa`.
- Maestro flows in the repository, by convention under `.maestro/`.
- Your TestingBot key and secret from the [dashboard](https://testingbot.com/members/user/edit). In EAS Workflows a `linux-medium` runner is enough for both platforms, because the flows run on TestingBot's simulators and devices; no macOS runner is needed.
Store the credentials as EAS project secrets once. EAS injects them into every step environment; the `${{ secrets.X }}` syntax does not exist in EAS Workflows.
```bash
eas env:create --scope project --name TB_KEY --value --visibility secret --type string \
--environment production --environment preview --environment development
eas env:create --scope project --name TB_SECRET --value --visibility secret --type string \
--environment production --environment preview --environment development
```
Secrets your flows need, such as a test user's password, are created the same way and passed with `--secret-env "PASSWORD=$TEST_PASSWORD"`, which keeps the value out of the run's options and reports (see [Secrets](https://testingbot.com/support/app-automate/maestro/options#secret-env)). Plain configuration goes in `--env`.
## From any CI: eas build and --app-url
With Expo there is no local binary: `eas build` produces a download URL, a `.tar.gz` for iOS simulator builds and an `.apk` for Android. From any CI, pass that URL with `--app-url`; the CLI downloads it and, for iOS, extracts the `.app` before uploading:
```bash
URL=$(eas build --platform ios --profile preview --json --non-interactive | jq -r '.[0].artifacts.buildUrl')
npx --yes @testingbot/cli@latest maestro --app-url "$URL" ./flows --device "iPhone 16"
```
## Inside EAS Workflows
Inside [EAS Workflows](https://docs.expo.dev/eas/workflows/get-started/) the build is already on the runner after `eas/download_build`, so pass the artifact path. Credentials come from EAS project secrets (`eas env:create --name TB_KEY --visibility secret …`), which EAS injects into the step environment; the `${{ secrets.X }}` syntax is not available there. The [@testingbot/eas-workflow](https://github.com/testingbot/testingbot-eas) wrapper turns the results into step outputs for `github-comment` and `slack` jobs:
[Android](https://testingbot.com#) [iOS](https://testingbot.com#)
```yaml
# .eas/workflows/maestro-android.yml
name: Maestro E2E (Android)
on:
pull_request: {}
jobs:
build_android:
type: build
params:
platform: android
profile: preview
e2e_android:
needs: [build_android]
runs_on: linux-medium
# Job-level env: EAS may silently ignore env set on a step.
env:
TB_EAS_BUILD_ID: ${{ needs.build_android.outputs.build_id }}
TB_EAS_PLATFORM: android
TB_EAS_PROFILE: preview
TB_GH_SHA: ${{ github.sha }}
TB_GH_BRANCH: ${{ github.ref_name }}
TB_GH_REPO_NAME: ${{ github.repository }}
outputs:
console_url: ${{ steps.testingbot.outputs.console_url }}
run_status: ${{ steps.testingbot.outputs.run_status }}
summary: ${{ steps.testingbot.outputs.summary }}
steps:
- uses: eas/checkout
- uses: eas/download_build
id: download
with:
build_id: ${{ needs.build_android.outputs.build_id }}
- id: testingbot
run: |
npx --yes @testingbot/eas-workflow@v1 \
--app-file ${{ steps.download.outputs.artifact_path }} \
--flows ./.maestro \
--device "Pixel 9" --deviceVersion "14" \
--report junit --report-output-dir ./reports
report:
after: [e2e_android]
type: github-comment
params:
payload: |
### Maestro on TestingBot — ${{ after.e2e_android.outputs.run_status }}
${{ after.e2e_android.outputs.summary }}
[View results](${{ after.e2e_android.outputs.console_url }})
```
```yaml
# .eas/workflows/maestro-ios.yml
name: Maestro E2E (iOS)
on:
pull_request: {}
jobs:
build_ios:
type: build
params:
platform: ios
profile: preview # a simulator profile: "ios": { "simulator": true }
e2e_ios:
needs: [build_ios]
runs_on: linux-medium
env:
TB_EAS_BUILD_ID: ${{ needs.build_ios.outputs.build_id }}
TB_EAS_PLATFORM: ios
TB_EAS_PROFILE: preview
TB_GH_SHA: ${{ github.sha }}
TB_GH_BRANCH: ${{ github.ref_name }}
TB_GH_REPO_NAME: ${{ github.repository }}
outputs:
console_url: ${{ steps.testingbot.outputs.console_url }}
run_status: ${{ steps.testingbot.outputs.run_status }}
summary: ${{ steps.testingbot.outputs.summary }}
steps:
- uses: eas/checkout
- uses: eas/download_build
id: download
with:
build_id: ${{ needs.build_ios.outputs.build_id }}
- id: testingbot
run: |
npx --yes @testingbot/eas-workflow@v1 \
--app-file ${{ steps.download.outputs.artifact_path }} \
--flows ./.maestro \
--device "iPhone 16" --deviceVersion "18"
report:
after: [e2e_ios]
type: github-comment
params:
payload: |
### Maestro on TestingBot — ${{ after.e2e_ios.outputs.run_status }}
${{ after.e2e_ios.outputs.summary }}
[View results](${{ after.e2e_ios.outputs.console_url }})
```
`after` rather than `needs` on the report job means the comment is still posted when the e2e job fails, which is exactly when you want it. Downstream jobs read the outputs as `needs.e2e_android.outputs.console_url`; the same values feed a `slack` job.
The CLI records the EAS build id, profile, platform and commit on the run automatically when it runs inside an EAS job, and the `TB_GH_*` variables add the repository and pull request so a [GitHub status check](https://testingbot.com/support/integrations/ci-cd/github-pr-checks#maestro) is posted too.
EAS download links are signed and expire after about an hour; a rejected link is reported as such so you know to request a fresh one. A local `.tar.gz` can also be passed as the app file directly, and `testingbot upload ` uploads once for reuse with `--app-binary-id`.
## Pitfalls specific to EAS Workflows
- **Set `env` on the job, not the step.** Step-level `env` can be ignored silently by EAS.
- **Pull request expressions are `null` outside pull requests.** `github.event.pull_request.number` evaluates to `null` on a manual or push trigger and EAS rejects the job. Set `TB_GH_PR_NUMBER` and `TB_GH_PR_URL` only in a workflow that runs on `pull_request`, or leave them out; `github.sha` and `github.ref_name` safely become empty strings.
- **`github.repository` is `owner/repo`.** Pass it as `TB_GH_REPO_NAME`; the wrapper splits it, so `TB_GH_REPO_OWNER` is optional.
- **Never name a variable `EAS_BUILD_*`.** That namespace belongs to the EAS Build worker; overwriting `EAS_BUILD_ID` ends the job silently after the prepare phase. The wrapper's variables use the `TB_` prefix for this reason.
- **Build links expire.** An `eas build` URL is signed and stops working after about an hour; inside EAS Workflows use `eas/download_build` instead of a stored URL.
## Wrapper options and outputs
`@testingbot/eas-workflow` takes `--app-file`, `--app-url` or `--app-binary-id` plus `--flows`; every other flag is forwarded to `testingbot maestro`, so `--device`, `--device-matrix`, `--retry`, `--report` and the rest all work. Step outputs: `console_url`, `app_id`, `run_urls`, `run_status`, `outcome`, `total_flows_count`, `successful_flows_count`, `failed_flows_count`, the flow name lists and a one-line `summary`. The step exits 2 when a flow fails and 1 on a CLI or infrastructure error.
Environment variables: `TB_KEY` and `TB_SECRET` (required), `TB_GH_SHA`, `TB_GH_BRANCH`, `TB_GH_PR_NUMBER`, `TB_GH_PR_URL`, `TB_GH_REPO_OWNER` and `TB_GH_REPO_NAME` for the run's CI metadata and [PR status check](https://testingbot.com/support/app-automate/maestro/ci-cd/github-pr-checks), `TB_CHECK_NAME` to name that check per job (`iOS`, `Android`), `TB_RUN_NAME` and `TB_GROUPS` for the dashboard, `TB_CLI_VERSION` to pin the CLI. Never name a variable `EAS_BUILD_*`; that namespace belongs to the EAS Build worker.
Full reference and complete Android and iOS workflow files: [testingbot/testingbot-eas](https://github.com/testingbot/testingbot-eas).
## Migrating from the maestro-cloud job type
Replace the pre-packaged job with a custom job. No Maestro Cloud project id or API key is needed.
```yaml
# Before
jobs:
e2e:
needs: [build_android]
type: maestro-cloud
params:
build_id: ${{ needs.build_android.outputs.build_id }}
maestro_project_id: proj_abc123
flows: ./.maestro
# After
jobs:
e2e:
needs: [build_android]
steps:
- uses: eas/checkout
- uses: eas/download_build
id: download
with:
build_id: ${{ needs.build_android.outputs.build_id }}
- id: testingbot
run: |
npx --yes @testingbot/eas-workflow@v1 \
--app-file ${{ steps.download.outputs.artifact_path }} \
--flows ./.maestro
```
`include_tags`, `exclude_tags`, `maestro_version`, `maestro_config`, `device_locale`, `name` and `async` become the CLI flags of the same meaning; `device_model` and `device_os` become `--device` and `--deviceVersion`. Output names carry over, only `maestro_cloud_url` is renamed to `console_url`, so downstream `github-comment` and `slack` jobs need no changes.
## Frequently asked questions
### How do I run Maestro flows against an EAS Build?
Pass the build URL with `--app-url`. The CLI downloads it and, for an iOS simulator build, extracts the `.app` from the `.tar.gz` before uploading. See [eas build and --app-url](https://testingbot.com#app-url).
### How do I use it inside EAS Workflows?
Download the build with `eas/download_build` and run `npx --yes @testingbot/eas-workflow@v1 --app-file ... --flows ...`. Credentials come from EAS project secrets. See [inside EAS Workflows](https://testingbot.com#workflows).
### Why can I not use ${{ secrets.TB\_KEY }} in an EAS workflow?
EAS Workflows do not support that syntax. Create `TB_KEY` and `TB_SECRET` with `eas env:create --visibility secret`; EAS injects them into the step environment.
### Do I need a macOS runner for iOS flows?
No. The flows run on TestingBot's simulators and devices; `linux-medium` is enough for both platforms. The iOS build job itself still runs on EAS's macOS workers.
### Why does my job fail with a null expression?
`github.event.pull_request.*` is `null` on manual and push triggers and EAS rejects it. Keep those variables in workflows triggered by `pull_request` only. See [pitfalls](https://testingbot.com#pitfalls).
### Does an EAS Build link expire?
Yes, after about an hour. The CLI reports a rejected link as expired so you know to request a fresh one, for example by re-running `eas build --json`.
### Can I replace the maestro-cloud job type?
Yes. Swap it for a custom job with `eas/download_build` and the wrapper; parameters map to CLI flags of the same meaning and the outputs keep their names. See [migrating](https://testingbot.com#migrate).
---
URL: https://testingbot.com/support/app-automate/maestro/tunnel
# Run Maestro with the TestingBot Tunnel
TestingBot runs your Maestro flows on real devices, simulators and emulators in our cloud datacenter. Those devices cannot, by default, reach an app or API that lives on your machine, your CI runner or a private staging network. The [TestingBot Tunnel](https://testingbot.com/support/tunnel) bridges that gap: it opens a secure, SSH-encrypted connection so the device under test can reach your internal hosts as if it were sitting on your own network.
The easiest way to use a tunnel with Maestro is the `-t` flag on the [TestingBot CLI](https://testingbot.com/support/app-automate/maestro#cli). It starts a tunnel, runs your flows through it, and tears the tunnel down when the run completes.
## When you need a tunnel
Use a tunnel whenever your Maestro flow drives an app that talks to a backend the public internet cannot see. Common scenarios:
- **Staging and pre-production backends.** Your app points at a staging API (`https://staging.internal.example.com`) that is firewalled off from the public internet. The tunnel lets the cloud device resolve and reach it.
- **Localhost development servers.** You are iterating locally and your API runs on `http://localhost:3000` on your laptop or CI runner. The tunnel forwards that port to the device.
- **Protected or token-gated endpoints.** Your backend requires a session or bearer token that is only issued inside your network. A small [helper script](https://testingbot.com#token) can `fetch` that token from a protected endpoint over the tunnel at runtime, so the secret never lives in your flow files.
- **Private databases and internal services.** The app calls an internal microservice, payment sandbox or database proxy that has no public DNS entry.
- **Mock and contract-test servers.** You run a mock server (WireMock, Mockoon, MSW) on your CI runner to return deterministic responses; the tunnel exposes it to the device for stable, hermetic tests.
- **Geo or VPN-restricted APIs.** An upstream API only accepts traffic from your office or VPN egress IP. Routing through the tunnel makes requests originate from your network.
If your backend is already publicly reachable (a deployed staging URL with valid TLS and no IP allowlist), you do not need a tunnel. The tunnel is only required when the device cannot otherwise reach the host.
## Quick Start
Add the `-t` flag (short for `--tunnel`) to your normal Maestro run command. The CLI starts a tunnel before the run and stops it afterwards:
```bash
testingbot maestro app.apk ./flows \
--device "Pixel 8" \
--deviceVersion "14" \
-t
```
That single command will:
- Authenticate and start a TestingBot Tunnel for this run
- Upload your app and Maestro flows
- Run the flows, routing all device traffic through the tunnel
- Shut the tunnel down once the run finishes
| Flag | Description |
| --- | --- |
| `-t, --tunnel` | Start a TestingBot Tunnel for this test run |
| `--tunnel-identifier ` | Identifier for the tunnel, allowing multiple tunnels to run in parallel |
The `-t` flag cannot be combined with `--async`. The tunnel has to stay open for the whole run, so the CLI must wait for the flows to finish.
Prefer to manage the tunnel yourself (for example, to keep one tunnel up across many runs)? Start the [TestingBot Tunnel](https://testingbot.com/support/tunnel) client separately and leave it running. When a tunnel is active on your account, Maestro runs automatically route through it, with no extra flags needed.
## How it works
When the tunnel is up, the cloud device sends its network requests over the encrypted tunnel to the tunnel client running in your environment. The client forwards each request to the target host (your localhost, staging server or internal API) and relays the response back to the device.
Because the request originates from inside your network, internal DNS, VPN routes and IP allowlists all behave as they normally would for local traffic. The tunnel supports HTTP/1, HTTP/2, WebSocket and Server-Sent Events, so streaming and real-time backends work as well.
For WebSocket or Server-Sent Events backends, start the standalone tunnel client with the `--nobump --nocache` flags. See the [tunnel command reference](https://testingbot.com/support/tunnel/commandline).
## Running multiple tunnels
If you run several Maestro builds in parallel, give each one its own named tunnel so traffic routes to the correct environment. Use `--tunnel-identifier` to set a unique name:
```bash
# Build A, pointing at the "staging" tunnel
testingbot maestro app.apk ./flows/smoke \
--device "Pixel 8" \
-t --tunnel-identifier staging
# Build B, pointing at the "qa" tunnel
testingbot maestro app.apk ./flows/regression \
--device "iPhone 16" --real-device \
-t --tunnel-identifier qa
```
Without an identifier, all traffic flows through the single default tunnel on your account. See [running multiple tunnels](https://testingbot.com/support/tunnel/multiple) for more detail.
## Staging token helper
A very common need: your staging or pre-production backend is protected and the app needs a short-lived auth token before it can do anything useful. You do not want to hardcode that token (it rotates, and committing secrets to flow files is unsafe). Instead, fetch it at runtime from a token endpoint that is only reachable over the tunnel.
Maestro can run inline JavaScript with the `runScript` command. The script executes in Maestro's JS environment and can make an HTTP request to your protected backend, which is reachable because the device routes through the tunnel. Store the result in `output` so later steps can use it.
### 1. The helper script
Create a small script, for example `scripts/fetch_token.js`, that calls your protected token endpoint:
```javascript
// scripts/fetch_token.js
// Runs inside Maestro. The request reaches your staging backend
// through the TestingBot Tunnel, so localhost / internal hosts resolve.
var response = http.post('http://localhost:3000/internal/auth/token', {
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
client: 'maestro-e2e',
scope: 'staging'
})
})
var token = json(response.body).access_token
// Expose the token to the rest of the flow
output.authToken = token
```
Maestro exposes an `http` helper and a `json()` parser inside `runScript`. If your backend is on your laptop, the host is `localhost`; for a named staging host, use its internal hostname (for example `http://staging.internal:8080`) and it will resolve over the tunnel.
### 2. Call it from your flow
Run the helper near the start of the flow, then use the token in subsequent steps, for example to deep-link into an authenticated screen or to set it in app state:
```yaml
appId: com.example.app
---
- runScript: scripts/fetch_token.js
# Use the token captured by the helper.
# For example, open a deep link that carries the session token:
- openLink: "exampleapp://session?token=${output.authToken}"
- assertVisible: "Welcome back"
```
### 3. Run it through the tunnel
Run as usual with the `-t` flag so `localhost` (or your staging host) is reachable from the device:
```bash
testingbot maestro app.apk ./flows \
--device "Pixel 8" \
--deviceVersion "14" \
-t
```
Keep the token endpoint scoped to non-production data and issue short-lived tokens. The tunnel is encrypted end to end, but treat any credential the flow can read as test-grade only.
The same pattern works for any runtime value the device needs from a protected source: feature-flag overrides, a seeded test-user id, a one-time passcode from a mailbox API, or a CSRF token.
## Using the tunnel in CI/CD
The `-t` flag behaves identically in CI. Build your app, then run Maestro with the tunnel so the cloud device can reach the mock server or staging backend running on the CI runner:
```bash
npx --yes @testingbot/cli@latest maestro \
app/build/outputs/apk/debug/app-debug.apk \
./maestro-flows \
--device "Pixel 8" \
--deviceVersion "14" \
-t \
--report junit \
--report-output-dir ./test-results
```
On GitHub Actions, the [TestingBot Maestro action](https://github.com/testingbot/maestro-action) starts the tunnel for you with `tunnel: true`:
```yaml
- uses: testingbot/maestro-action@v1
with:
api-key: ${{ secrets.TB_KEY }}
api-secret: ${{ secrets.TB_SECRET }}
app-file: app/build/outputs/apk/debug/app-debug.apk
workspace: ./maestro-flows
device: Pixel 8
device-version: '14'
tunnel: true
report: junit
```
For full GitHub Actions, GitLab CI, Jenkins, CircleCI and Bitbucket Pipelines examples, including secret handling and reports, see the [Maestro CI/CD integration page](https://testingbot.com/support/app-automate/maestro/ci-cd).
Because `-t` cannot be combined with `--async`, the CI step blocks until the run finishes, which is what you want so the pipeline can pass or fail on the result.
## CLI Reference
| CLI Option | Description |
| --- | --- |
| `-t, --tunnel` | Start a [TestingBot Tunnel](https://testingbot.com/support/tunnel) for this test run (cannot be combined with `--async`) |
| `--tunnel-identifier ` | Identifier for the tunnel, allowing multiple tunnels to run in parallel |
See the full [Maestro options](https://testingbot.com/support/app-automate/maestro/options#tunnel) and the [TestingBot Tunnel documentation](https://testingbot.com/support/tunnel) for advanced settings such as upstream proxies, custom DNS, header injection and PAC support.
## Frequently asked questions
Why do I need a tunnel to run Maestro on TestingBot?
TestingBot devices run in our cloud datacenter and cannot reach apps or APIs that live on your laptop, your CI runner or a private staging network. The [TestingBot Tunnel](https://testingbot.com/support/tunnel) opens a secure, SSH-encrypted connection so the device under test can resolve and reach those internal hosts as if it were on your own network. See the [use cases section](https://testingbot.com#usecases) for common scenarios.
How do I start a tunnel for a Maestro run?
Pass the `-t` (or `--tunnel`) flag to the [TestingBot CLI](https://testingbot.com/support/app-automate/maestro#cli). The CLI starts a tunnel, runs your flows through it, and shuts the tunnel down when the run finishes. See the [quick start](https://testingbot.com#quickstart).
Can I run multiple Maestro tunnels in parallel?
Yes. Give each tunnel a unique name with `--tunnel-identifier ` so parallel runs route through the correct tunnel. See [running multiple tunnels](https://testingbot.com#multiple).
How does my app get a staging auth token during a Maestro flow?
Your app, running on the cloud device, makes its normal network calls through the tunnel back to your protected backend. A small helper that performs a `fetch`/AJAX call to a token endpoint behind the tunnel can retrieve a short-lived token at runtime, so you never hardcode secrets in the flow. See the [token helper example](https://testingbot.com#token).
Can I use the tunnel in CI/CD?
Yes. The `-t` flag works the same way in any CI runner. Combine it with the examples on the [Maestro CI/CD page](https://testingbot.com/support/app-automate/maestro/ci-cd). The `-t` flag cannot be combined with `--async`, because the tunnel must stay up for the duration of the run.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/app-automate/maestro/migrate-from-maestro-cloud
# Migrate from Maestro Cloud to TestingBot
Maestro flows are plain YAML and are not tied to any one runner. If you already run them on [Maestro Cloud](https://maestro.dev/cloud), moving them to TestingBot is a change of upload command and device flags, not a rewrite. Your flow files, your tags and your Maestro syntax stay as they are.
This page maps every `maestro cloud` option to its TestingBot equivalent, so you can port a working pipeline in one sitting.
New to Maestro on TestingBot? Start at [Maestro testing](https://testingbot.com/support/app-automate/maestro) for the full setup, then come back here for the mapping.
## Before you start
Install the TestingBot CLI and authenticate once:
```bash
npm install -g @testingbot/cli
testingbot login
```
Credentials are stored in `~/.testingbot`. In CI, pass `--api-key` and `--api-secret`, or set `TB_KEY` and `TB_SECRET` as secrets.
## Command mapping
The whole migration in one line. On Maestro Cloud:
```bash
maestro cloud --app-file app.apk --flows ./flows
```
On TestingBot:
```bash
testingbot maestro app.apk ./flows --device "Pixel 8" --deviceVersion "14"
```
The app binary and the flows are positional arguments rather than named flags, but the `maestro cloud` spelling is accepted too: `--app-file`, `--flows`, `--apiKey`, `--device-model`, `--device-os`, `--format`, `--output` and `--test-suite-name` work as hidden aliases, so an existing command line runs unchanged once you add `--api-secret`:
```bash
testingbot maestro --app-file app.apk --flows ./flows --device-model Pixel-7 --device-os android-34 --format JUNIT --output report.xml
```
Everything the flows directory can contain stays the same, and you can pass a directory, several directories, individual flow files, or a pre-zipped bundle:
```bash
# A directory of flows
testingbot maestro app.apk ./flows
# Several directories
testingbot maestro app.apk ./flows/smoke ./flows/regression
# Individual flow files
testingbot maestro app.apk login.yaml checkout.yaml
# A pre-zipped bundle
testingbot maestro app.apk flows.zip
```
## Flag mapping
Every `maestro cloud` option and what to use instead. Flag names that are identical on both sides are marked, so you can leave those parts of your command untouched.
| Maestro Cloud | TestingBot CLI | Notes |
| --- | --- | --- |
| `--app-file` | first positional argument | Same binary formats: .apk, .aab, .ipa, .app. `--app-file` also works as an alias |
| `--flows` | second positional argument | Directory, files or zip. `--flows a,b` also works as an alias |
| `--device-model` | `--device` | For example `--device "Pixel 8"`. `--device-model iPhone-17-Pro` is accepted and mapped |
| `--device-os` | `--deviceVersion` and `--platform` | Split into two flags, or keep `--device-os iOS-18-2` / `android-34`: it is mapped (Android API levels become OS versions) |
| `--device-locale` | `--device-locale` | Identical |
| `--async` | `--async` | Identical |
| `--include-tags` | `--include-tags` | Identical |
| `--exclude-tags` | `--exclude-tags` | Identical |
| `-e`, `--env` | `--env` | Same `KEY=value` form |
| `--name` | `--name` | Identical |
| `--format`, `--output` | `--report`, `--report-output-dir` | See [reports](https://testingbot.com#reports). Both are accepted as aliases; `--report allure` is available too |
| `--api-key` | `--api-key` and `--api-secret` | TestingBot uses a key and secret pair |
| `--branch` | `--branch` | Identical; `--pr-url` and `-m KEY=VALUE` metadata are available too |
| `--commit-sha` | `--commit-sha` | Identical |
| `--pull-request-id` | `--pull-request-id` | Identical |
| `--repo-name` | `--repo-name` | Identical |
| `--repo-owner` | `--repo-owner` | Identical |
| `--app-binary-id` | `--app-binary-id` | Identical. `testingbot upload app.apk` prints the ID; see [Reuse an Uploaded App](https://testingbot.com/support/app-automate/maestro/options#app-binary-id) |
| `--test-suite-name` | `--name` | Accepted as an alias |
| `--android-api-level`, `--ios-version` | `--deviceVersion` | Deprecated upstream in favour of `--device-os` |
TestingBot adds flags that have no `maestro cloud` counterpart, including `--device-matrix`, `--json`, `--exclude-flows`, `--retry`, `--shard-split`, `--real-device`, `--tunnel`, `--orientation`, `--timezone`, `--geo-country-code`, `--throttle-network`, `--download-artifacts` and `--maestro-version`. They are documented in [Maestro options](https://testingbot.com/support/app-automate/maestro/options).
## Device and OS selection
Maestro Cloud selects an environment with a single combined string, for example `--device-os "android-34"`, or an OS plus model pair on iOS. TestingBot splits that into a device name and a version:
```bash
# Android
testingbot maestro app.apk ./flows --device "Pixel 8" --deviceVersion "14"
# iOS simulator
testingbot maestro app.ipa ./flows --device "iPhone 16" --deviceVersion "18.6" --platform "iOS"
# Physical iPhone
testingbot maestro app.ipa ./flows --device "iPhone 16" --real-device
```
Add `--real-device` to run on physical hardware instead of a simulator or emulator. The full list of device names and versions is on [Maestro options](https://testingbot.com/support/app-automate/maestro/options).
## Tags and environment variables
These copy across unchanged, both the flag names and their behaviour:
```bash
testingbot maestro app.apk ./flows \
--device "Pixel 8" \
--include-tags smoke,checkout \
--exclude-tags flaky \
--env USERNAME=demo --env BASE_URL=https://staging.example.com
```
Tags still come from the `tags:` block in your flow files, so no flow edits are needed.
## Reports and artifacts
Maestro Cloud writes a report with `--format` and `--output`. TestingBot uses a report type and an output directory:
```bash
testingbot maestro app.apk ./flows --device "Pixel 8" \
--report junit \
--report-output-dir ./reports
```
Video, logs and screenshots can be pulled down in the same run:
```bash
testingbot maestro app.apk ./flows --device "Pixel 8" \
--download-artifacts \
--artifacts-output-dir ./artifacts
```
## Running flows in parallel
Split a flows directory across concurrent sessions with `--shard-split`. Your plan's concurrency decides how many run at once:
```bash
# Three parallel sessions
testingbot maestro app.apk ./flows \
--device "Pixel 8" --deviceVersion "14" --shard-split 3
# Everything in one session
testingbot maestro app.apk ./flows \
--device "Pixel 8" --deviceVersion "14" --shard-split 1
```
## CI/CD
On GitHub Actions, swap the action and the credentials; the other inputs keep their names:
```yaml
- uses: testingbot/maestro-action@v1
with:
api-key: ${{ secrets.TESTINGBOT_KEY }}
api-secret: ${{ secrets.TESTINGBOT_SECRET }}
app-file: app.zip
workspace: .maestro
include-tags: smoke
```
There is no `project-id`. The outputs map as `MAESTRO_CLOUD_CONSOLE_URL` → `console-url`, `MAESTRO_CLOUD_APP_BINARY_ID` → `app-id`, `MAESTRO_CLOUD_UPLOAD_STATUS` → `outcome` and `MAESTRO_CLOUD_FLOW_RESULTS` → `flow-results`. Maestro Cloud's native pull request integration is covered by [GitHub PR status checks](https://testingbot.com/support/app-automate/maestro/ci-cd/github-pr-checks).
From any other CI, the metadata flags keep the same names, so the parts of your pipeline that tag a run with a branch, commit or pull request need no edits:
```bash
npx --yes @testingbot/cli@latest maestro app.apk ./flows \
--device "Pixel 8" \
--deviceVersion "14" \
--api-key $TB_KEY \
--api-secret $TB_SECRET \
--repo-owner "$GITHUB_REPOSITORY_OWNER" \
--commit-sha "$GITHUB_SHA"
```
Ready-made GitHub Actions, GitLab CI, Jenkins, CircleCI and Bitbucket Pipelines examples are on [Maestro CI/CD integration](https://testingbot.com/support/app-automate/maestro/ci-cd). If your app talks to a staging environment that is not reachable from the internet, see [local and staging testing](https://testingbot.com/support/app-automate/maestro/tunnel).
## What differs
Worth knowing before you port a pipeline:
- **Two positional arguments.** The app and the flows are passed by position rather than through `--app-file` and `--flows`.
- **Device and version are separate flags.** There is no single combined `--device-os` string.
- **Authentication is a key and secret pair** , not a single API key.
- **Real hardware is opt-in** with `--real-device`.
- **Reports are configured by type and directory** rather than format and file path.
## Frequently asked questions
### Do I have to rewrite my Maestro flows to run them on TestingBot?
No. Flows are plain Maestro YAML and run unchanged. What changes is the command that uploads and runs them, and the flags that pick a device. Your flow files, tags and Maestro syntax stay exactly as they are.
### What replaces the maestro cloud command on TestingBot?
The TestingBot CLI: `testingbot maestro `. See the [command mapping](https://testingbot.com#commands) for a side-by-side, and the [flag mapping](https://testingbot.com#flags) for every option.
### How do I pick a device and OS version on TestingBot?
With `--device` and `--deviceVersion`, instead of a single combined `--device-os` string. Add `--real-device` to run on physical hardware rather than a simulator or emulator. See [device and OS selection](https://testingbot.com#devices).
### Can I keep my --include-tags and --exclude-tags filters?
Yes, both flags exist on the TestingBot CLI under the same names and behave the same way, so tag filters copy across unchanged. See [tags and environment variables](https://testingbot.com#tags).
### Will my CI pipeline still publish JUnit reports?
Yes. Where Maestro Cloud uses `--format` and `--output`, TestingBot uses `--report junit` and `--report-output-dir`. The CI metadata flags for branch, commit and pull request keep the same names. See [reports and artifacts](https://testingbot.com#reports).
Run your existing Maestro flows on TestingBot with a [free trial](https://testingbot.com/users/sign_up), or read why teams choose us on the [Maestro Cloud alternative](https://testingbot.com/maestro-cloud-alternative) page.
---
URL: https://testingbot.com/support/app-automate/mobilewright
# Mobilewright Testing
[Mobilewright](https://github.com/mobile-next/mobilewright) is an open source mobile UI test framework with a Playwright-style API. You write tests in TypeScript or JavaScript against `device`, `screen` and `expect` fixtures, and Mobilewright drives your iOS or Android app.
TestingBot maintains [`@testingbot/mobilewright-driver`](https://github.com/testingbot/mobilewright-testingbot), which runs those same tests on our device cloud instead of a device attached to your laptop: real iPhones, iPads and Android phones and tablets in our European datacenter, plus iOS simulators and Android emulators.
- No local Appium server, Xcode or Android SDK required.
- Your app is uploaded and pre-installed on the device before the first test runs.
- Every device slot becomes a TestingBot session with video, command logs and device logs.
- Pass and fail verdicts, build names and git metadata are reported for you.
The driver requires **Node.js 18 or newer** and **Mobilewright 0.0.53 or newer** , the first release that accepts driver instances. Its peer dependency is `@mobilewright/protocol >=0.0.53 <0.1.0`, which Mobilewright installs for you.
## Quickstart
The fastest way to start is the scaffolder. It asks a few questions and writes a `mobilewright.config.ts` plus a sample test into your project. Add `--yes` to accept the defaults.
```bash
npx @testingbot/mobilewright-driver init
```
To set things up by hand instead, install the packages as dev dependencies:
```bash
npm install --save-dev mobilewright @mobilewright/test @testingbot/mobilewright-driver
```
### Credentials
The driver reads your TestingBot key and secret from the environment. You can find them on your [member edit page](https://testingbot.com/members/user/edit).
```bash
export TESTINGBOT_KEY=your_key
export TESTINGBOT_SECRET=your_secret
```
`TB_KEY` and `TB_SECRET` are accepted as aliases. If you would rather not use environment variables, pass `key` and `secret` to the driver directly. Do not commit them to your repository.
## Configuration
Point Mobilewright at TestingBot by passing a `TestingBotDriver` instance in your `mobilewright.config.ts`. Because TestingBot sessions start with the app already installed, the app under test has to be declared up front in the `apps` option.
```typescript
import { defineConfig } from 'mobilewright';
import { TestingBotDriver } from '@testingbot/mobilewright-driver';
export default defineConfig({
testDir: '.',
bundleId: 'com.example.MyApp',
driver: new TestingBotDriver({
apps: {
android: './build/app.apk',
'ios-simulator': './build/app-sim.zip',
'ios-real': './build/app.ipa', // must be test-signed for real devices
},
}),
projects: [
{ name: 'android', use: { platform: 'android', deviceType: 'emulator' } },
{ name: 'ios', use: { platform: 'ios', deviceType: 'real', osVersion: '>=17' } },
],
});
```
A few things worth knowing about `apps`:
- Local `.apk`, `.ipa` and simulator `.zip` paths are uploaded to [TestingBot Storage](https://testingbot.com/support/api/storage#upload) automatically. Already-uploaded builds can be referenced with their `tb://` URL.
- The keys are `android`, `android-emulator`, `android-real`, `ios`, `ios-simulator` and `ios-real`. The most specific one wins: `ios-real` overrides `ios`.
- A value can be an array. The first entry is the app under test, the rest are installed alongside it, which is how you ship a mock server or a test harness with your build.
A Mobilewright test looks the same whether it runs locally or on TestingBot:
```typescript
import { test, expect } from '@mobilewright/test';
test('shows the welcome screen', async ({ device, screen, bundleId }) => {
await device.launchApp(bundleId);
await expect(screen.getByText('Welcome')).toBeVisible();
});
```
## Selecting devices
Device selection comes from the Mobilewright project config, in the `use` block. Regular expressions and version ranges work on both real and virtual devices.
| Key | Example | Notes |
| --- | --- | --- |
| `platform` | `'ios'` or `'android'` | Required. |
| `deviceType` | `'real'`, `'simulator'`, `'emulator'` | Defaults to a virtual device. |
| `deviceName` | `/iPhone 1[45]/`, `/pixel/i` | Regex, matched against the TestingBot device catalogs. |
| `osVersion` | `'17'`, `'17.4'`, `'>=16 <18'` | Exact version, prefix or range expression. |
| `deviceId` | `'2241'` | Pin one specific physical device. The id comes from [GET /v1/devices](https://testingbot.com/support/api/devices). The Mobilewright docs mark `deviceId` as local-driver-only; this driver resolves it against the TestingBot catalog, so it works here too. |
Real devices resolve against the live physical catalog, where idle devices are preferred. If the device you asked for is busy, TestingBot queues your session until it frees up rather than failing. Virtual devices resolve against the simulator and emulator catalog, where the newest matching OS version wins. See the [supported device list](https://testingbot.com/support/app-automate/devices) for what is currently available.
## Running tests
```bash
npx mobilewright test
```
Each device slot maps to a TestingBot session on your [dashboard](https://testingbot.com/members), with video and a pass or fail verdict reported automatically. Prefix the command with `DEBUG=testingbot:*` to print every allocation, hub command, upload and result the driver performs.
```bash
DEBUG=testingbot:* npx mobilewright test
```
## Driver options
Everything the driver accepts. Only `apps` is required; the values below are examples, and the comments call out the real defaults. Anything left unset is simply not sent, in which case the TestingBot server-side default applies.
```typescript
new TestingBotDriver({
apps: { android: './build/app.apk' }, // app under test, most specific key wins
// ('ios-real' beats 'ios'); tb:// URLs allowed.
// Array = app under test first, helper apps after:
// android: ['./app.apk', './mock-server.apk']
key: '...', // default: TESTINGBOT_KEY (or TB_KEY) env
secret: '...', // default: TESTINGBOT_SECRET (or TB_SECRET) env
sessionPerTest: false, // true = fresh session (and video) per test
name: 'checkout flow', // session name (default 'mobilewright')
hubUrl: '...', // default: https://hub.testingbot.com/wd/hub
apiUrl: '...', // default: https://api.testingbot.com/v1
build: 'ci-1234', // default: auto-detected from CI env / TESTINGBOT_BUILD
testResults: 'on', // 'off' disables pass/fail reporting
allocationTimeout: 395_000, // ms to wait for a device (default 395_000)
commandTimeout: 60_000, // ms per WebDriver command (default 60_000)
idleTimeout: 230, // seconds before TestingBot reaps an idle session (default 230)
maxDuration: 1800, // max session length in seconds (unset by default)
appiumVersion: '2.11.2', // pin the Appium version (unset = TestingBot's default)
timeZone: 'Europe/Brussels', // device timezone (tz database name)
geoCountryCode: 'DE', // route device traffic via a proxy in this country
throttleNetwork: '3G', // or { downloadSpeed, uploadSpeed, latency, loss }
autoGrantPermissions: true, // Android: auto-grant app permission dialogs
autoAcceptAlerts: true, // iOS: auto-accept system permission alerts
screenshots: true, // screenshot at every step (default false)
video: true, // session video (default true)
recordLogs: 'strip-parameters', // command logs: true | false | 'strip-parameters'
public: false, // make results publicly accessible
extra: 'commit=abc123', // custom metadata on the test detail page
tabletOnly: false, // restrict allocation to tablets
phoneOnly: false, // ...or to phones
capabilities: {}, // extra Appium capabilities (escape hatch)
tbOptions: {}, // extra tb:options entries (escape hatch)
});
```
`capabilities` and `tbOptions` are the escape hatches for anything the driver does not expose directly. See the [Appium capabilities reference](https://testingbot.com/support/app-automate/appium/capabilities) for what you can put in them.
## TestingBot commands inside tests
Some TestingBot features are runtime commands rather than session capabilities. Import the `testingbot` helper and call it from a test body; it targets the device session Mobilewright connected for that test.
```typescript
import { test, expect } from '@mobilewright/test';
import { testingbot } from '@testingbot/mobilewright-driver';
test('checkout survives a slow network', async ({ device, screen, bundleId }) => {
await device.launchApp(bundleId);
await testingbot.annotate('starting checkout'); // shows in the session timeline
await testingbot.throttle('3G');
await expect(screen.getByText('Order placed')).toBeVisible();
console.log(testingbot.dashboardUrl());
});
```
| Command | TestingBot command | What it does |
| --- | --- | --- |
| `throttle(conditions)` | `tb:throttle` | Network conditions mid-test: `'Edge'`, `'3G'`, `'4G'`, `'airplane'`, `'disable'`, or `{ downloadSpeed, uploadSpeed, latency, loss }` in kb/s and ms. |
| `annotate(text)` | `tb:test-context` | Logs a step into the session command list so the timeline shows what the test was doing. |
| `setName(name)` | `tb:test-name` | Names the session on the dashboard. |
| `setBuild(build)` | `tb:test-build` | Groups the session under a build. |
| `setTags([...])` | `tb:test-tags` | Tags the session. |
| `setResult(passed)` | `tb:test-result` | Overrides the verdict, as a boolean or `'passed'` / `'failed'`. The driver reports it automatically, so use this only for outcomes Mobilewright cannot see. |
| `updateInfo({...})` | `tb:test-info` | Bulk update of `name`, `build`, `public`, `statusMessage` and `extra`. |
| `breakpoint()` | `tb:break` | Pauses the session for manual inspection with a live view. Debugging only, never leave it in CI. |
| `shell(command, args)` | `mobile: shell` | ADB shell on Android. A whitelisted subset is available on physical devices. |
| `execute(script, ...args)` | any | Escape hatch for any Appium or TestingBot execute-script command. |
| `sessionId()` / `dashboardUrl()` | - | This test's TestingBot session id and dashboard link. |
Throttling is reset automatically when the test's device session is released, so slow-network conditions never leak into the next test that reuses the device. Calling a command outside a test body raises an explanatory error rather than a null reference.
### Recipe: name every session after its test
```typescript
test.beforeEach(async ({ device }, testInfo) => {
await testingbot.setName(testInfo.title);
await testingbot.setTags([testInfo.project.name]);
});
```
Requesting the `device` fixture is what makes the session exist before the hook runs. With `sessionPerTest: true` each session gets exactly one name; in pooled mode the last test to run on a session wins.
## Sessions, naming and reporting
- By default a device slot's session is reused across tests, so you pay device startup once. A session that hosted exactly one test is renamed after that test and carries its own verdict. A session that hosted several keeps its configured name (`mobilewright` unless you set one) and gets a run summary in its **status message** instead, such as `12/12 tests passed`, or `1 of 3 tests failed. ...` with the individual failure messages appended.
- Set `sessionPerTest: true` for one session, one video and one named verdict per test. That is the clearest dashboard, at the cost of device startup on every test.
- Every session is tagged `mobilewright`, so you can filter the dashboard down to these runs. Git metadata is reported on top of that when Mobilewright captures it: the commit hash, branch, author and subject land in the session's **extra** field, alongside your own `extra` rather than replacing it, and the branch is added as a second tag. Enable it with `captureGitInfo: { commit: true }` in `mobilewright.config.ts`.
- Failure messages are reported to the session's status message. `build` groups sessions on the dashboard and is auto-detected from GitHub Actions, GitLab, CircleCI, Buildkite, Bitrise, Travis, Azure DevOps, Jenkins and TeamCity, or read from a `TESTINGBOT_BUILD` variable.
## Testing localhost and staging servers
If your app talks to a backend that is not reachable from the internet, use the [TestingBot Tunnel](https://testingbot.com/support/tunnel). Either run the tunnel yourself and hand the driver its identifier:
```typescript
driver: new TestingBotDriver({ tunnelIdentifier: 'my-tunnel' })
```
Or let the driver manage the tunnel around the run. This requires Java and `npm install --save-dev testingbot-tunnel-launcher`:
```typescript
driver: new TestingBotDriver({ tunnel: true }) // anonymous tunnel
driver: new TestingBotDriver({ tunnel: { identifier: 'ci-42' } }) // named tunnel
```
The driver starts the tunnel before any device is allocated and closes it after the run.
## Running in CI/CD
Store your credentials as secrets and run Mobilewright like any other test command. The build name is auto-detected, so no extra configuration is needed.
```yaml
jobs:
mobile-tests:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with: { node-version: 20, cache: npm }
- run: npm ci
- run: npx mobilewright test
env:
TESTINGBOT_KEY: ${{ secrets.TESTINGBOT_KEY }}
TESTINGBOT_SECRET: ${{ secrets.TESTINGBOT_SECRET }}
# build name auto-detects as "owner/repo #run" - no config needed
```
The same pattern works on the other CI systems listed under [CI/CD integrations](https://testingbot.com/support/integrations/ci-cd).
## How Mobilewright maps to TestingBot
| Mobilewright concept | TestingBot implementation |
| --- | --- |
| Device allocation | An Appium session on `hub.testingbot.com`. The session UUID is the device id, and busy devices queue server side. |
| App install | Uploaded to TestingBot Storage at allocation time and passed as `appium:app`, helper apps as `appium:otherApps`. Real iOS builds are re-signed automatically. There are no mid-session installs. |
| Taps, swipes and gestures | W3C pointer actions. |
| View hierarchy | Appium page source, mapped to Mobilewright's `ViewNode` tree. |
| Webviews | Appium contexts, through `webViewBridge`. |
| Pass and fail reporting | `PUT /v1/tests/:session` from the `TestObserver` hooks. |
| Video | Recorded by TestingBot. `startRecording({ output })` downloads the MP4 locally at run end. |
## Troubleshooting
| Symptom | Cause and fix |
| --- | --- |
| `TestingBot credentials are missing` | Set `TESTINGBOT_KEY` and `TESTINGBOT_SECRET`, or pass `{ key, secret }` to the driver. |
| `TestingBot sessions must start with an app` | Declare the app in the driver's `apps` option, per platform and device type. |
| `No TestingBot real device matches ...` | Your criteria matched nothing in the catalog. Check the [available devices](https://testingbot.com/support/app-automate/devices) and loosen `osVersion` or `deviceName`. |
| Allocation waits for minutes | The matching device is busy. TestingBot queues the session until it frees up, bounded by `allocationTimeout`. |
| `Test timeout ... while setting up "device"` | More `workers` than your plan's parallel limit. TestingBot queues up to twice the limit server side and the driver waits out the rest. The driver logs your plan's number: set `workers` to at most that, or raise the Mobilewright `timeout`. |
| `... is a simulator-only build and cannot be installed on a real device` | Build a test-signed `.ipa` for `deviceType: 'real'` projects. |
| `"..." is not installed on this device` | The config's `bundleId` is not the package id of the build in `apps`. Check it with `aapt2 dump packagename app.apk`, and look for a stale `bundleId` or environment override. |
| `cannot install "..." mid-session` | TestingBot has no mid-session installs. Every app in `installApps` must also be listed in the driver's `apps` option. |
| Sessions show a run summary instead of a test name | The session hosted several tests, so the summary goes to the status message and the name is left alone. Use `sessionPerTest: true` to get one named session per test. |
## Current limitations
- One TestingBot session may host several Mobilewright tests unless `sessionPerTest: true`. Multi-test sessions report the run level verdict.
- Modifier key chords such as `pressKeys(['ctrl+a'])` work on Android only, because XCUITest cannot hold modifier keys.
- `pressButton` on iOS supports `HOME`, `VOLUME_UP` and `VOLUME_DOWN`. `listApps()` reports the foreground app only.
- Screenshots are always PNG. `applyDeviceSettings` turns Android animations off on a best-effort basis and is a no-op on iOS.
- Real iOS devices need a test-signed `.ipa`. Simulator builds are rejected on real devices with a clear error.
- While attached to a webview, the session's single Appium context is the web layer.
## Frequently asked questions
What is Mobilewright?
Mobilewright is an open source mobile UI test framework from the mobile-next project. You write tests in TypeScript or JavaScript with a Playwright-style API (`device`, `screen`, `expect`) and run them against iOS and Android apps. TestingBot publishes `@testingbot/mobilewright-driver`, a driver that points those same tests at our device cloud.
How do I run Mobilewright tests on TestingBot?
Run `npx @testingbot/mobilewright-driver init` in your test project to scaffold a config and a sample test, export `TESTINGBOT_KEY` and `TESTINGBOT_SECRET`, then run `npx mobilewright test`. See the [quickstart](https://testingbot.com#quickstart) for the manual setup.
Can I run Mobilewright on real iOS and Android devices?
Yes. Set `deviceType: 'real'` in a project's `use` block to allocate physical iPhones, iPads and Android phones or tablets from our EU datacenter. Leave it out and you get an iOS simulator or Android emulator instead. Real iOS devices need a test-signed `.ipa`.
Do I have to upload my app before running?
No. Declare your builds in the driver's `apps` option and local `.apk`, `.ipa` and simulator `.zip` files are uploaded to [TestingBot Storage](https://testingbot.com/support/api/storage#upload) for you. If you already uploaded a build you can reference its `tb://` URL instead.
Why do I get "Test timeout while setting up device"?
You are running more Mobilewright `workers` than your plan's parallel limit. TestingBot queues up to twice the limit server side and the driver waits out the rest, so the test-scoped device fixture eventually times out. The driver logs your plan's parallel limit: set `workers` to at most that number, or raise the Mobilewright `timeout`.
Can I test a staging or localhost backend?
Yes. Pass `tunnelIdentifier` to reuse a [TestingBot Tunnel](https://testingbot.com/support/tunnel) you started yourself, or let the driver open and close one around the run with `tunnel: true`. See [Testing localhost and staging](https://testingbot.com#tunnel).
Why does one session contain several tests?
By default a device slot's session is reused across tests so you only pay device startup once, and such a session reports a run level verdict in its status message, such as "12/12 tests passed". Set `sessionPerTest: true` for one session, one video and one named verdict per test.
How do I debug a failing Mobilewright run?
Prefix the run with `DEBUG=testingbot:*` to log every allocation, hub command, upload and result the driver performs. Every session also has video, command logs and device logs on its test detail page in the [dashboard](https://testingbot.com/members).
## Next steps
- [ **Driver on GitHub**
Source, issues and releases](https://github.com/testingbot/mobilewright-testingbot)
- [ **Supported devices**
Real iPhones, iPads and Android devices on the grid](https://testingbot.com/support/app-automate/devices)
- [ **Real device testing**
How the real-hardware cloud works](https://testingbot.com/real-device-testing)
- [ **TestingBot Tunnel**
Test against internal or staging APIs](https://testingbot.com/support/tunnel)
- [ **Appium guide**
The protocol the driver speaks underneath](https://testingbot.com/support/app-automate/appium)
- [ **Pricing**
Real-device plans and concurrency](https://testingbot.com/pricing)
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/app-automate/espresso
# Espresso Testing
TestingBot provides access to Android physical devices and Android emulators, capable of running your Android Espresso tests.
[Espresso](https://developer.android.com/training/testing/espresso) is a mobile automation framework from Google, designed to run UI tests for Android applications.
Using Espresso in combination with TestingBot's Android cloud service is very straightforward:
- Upload your native mobile Android app, together with the Espresso test apk file. These are created with either Java or Kotlin.
- Specify on which devices you want to run the Espresso tests.
- Retrieve the results of the Espresso tests.
## Getting Started
To get started, please follow the steps below:
1. You will need a `key` and `secret` to connect to the TestingBot grid.
These credentials are available in the [TestingBot member area](https://testingbot.com/members).
2. Espresso tests require a separate `apk` file for testing. So you will need to upload both the app (`.apk` or `.aab` file) and the test file (`.apk`).
You can also use the [TestingBot Espresso Demo App](https://github.com/testingbot/android-espresso-demo-app) to give the Espresso Cloud Testing functionality a try.
## TestingBot CLI
The easiest way to run Espresso tests on TestingBot is using our CLI tool. It handles uploading your app, test APK, and running tests in a single command.
### Installation
```bash
npm install -g @testingbot/cli
```
### Authentication
Authenticate using this command:
```bash
testingbot login
```
This opens your browser for authentication. After logging in, your credentials are saved to `~/.testingbot` and you can start using the CLI.
#### Other Authentication Methods
- **Command-line options** : `--api-key` and `--api-secret`
- **Environment variables** : `TB_KEY` and `TB_SECRET`
- **Config file** : Create `~/.testingbot` with content `key:secret`
### Quick Start
Run Espresso tests with a single command:
```bash
testingbot espresso app.apk app-test.apk --device "Pixel 8" --platform-version "14"
```
```bash
testingbot espresso app.apk app-test.apk --device "Galaxy S24" --real-device
```
The CLI will:
1. Upload your app APK to TestingBot
2. Upload your test APK
3. Start the test run
4. Show real-time progress and results
If you build your Android app with Gradle, our [Gradle plugin](https://testingbot.com/support/integrations/ci-cd/gradle) can upload your app and run Espresso tests straight from your build, no separate CLI step required.
## Upload Android App
To run Android Espresso tests on TestingBot, you will first have to upload your `.apk` or `.aab` file to TestingBot Storage.
[CLI](https://testingbot.com#) [cURL](https://testingbot.com#)
When using the CLI, the app upload is handled automatically as part of the `testingbot espresso` command. Simply specify your app APK file as the first argument:
```bash
testingbot espresso /path/to/app/Application-debug.apk /path/to/test/Application-debug-test.apk --device "Pixel 8"
```
Alternatively, you can use the `--app` option:
```bash
testingbot espresso --app /path/to/app/Application-debug.apk --test-app /path/to/test/Application-debug-test.apk --device "Pixel 8"
```
```bash
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/espresso/app" \
-F "file=@/path/to/app/file/Application-debug.apk"
```
The API response will include an `id` response, which you can use in the other API calls.
```json
{
"id": 4012
}
```
## Upload Espresso Test Suite
You can now upload the `.apk` file containing the Espresso tests.
The upload process is similar to the [app upload](https://testingbot.com#upload) process.
[CLI](https://testingbot.com#) [cURL](https://testingbot.com#)
When using the CLI, the test APK upload is handled automatically. Specify your test APK as the second argument:
```bash
testingbot espresso app.apk app-test.apk --device "Pixel 8"
```
Or use the `--test-app` option:
```bash
testingbot espresso --app app.apk --test-app app-test.apk --device "Pixel 8"
```
```bash
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/espresso/:id/tests" \
-F "file=@/path/to/app/file/Application-debug-test.apk"
```
Replace the `:id` with the identifier you received during the app upload call.
## Specify devices
You can specify one or more Android device capabilities with your API call.
These capabilities can target either Android emulators, or physical Android devices.
 Android 16.0 › Pixel 9
Loading environments...
Please wait while we load the available browsers and platforms.
We offer special parameters which you can use to allocate a device:
Regex Input | Result || `"Pixel.*"` | This will allocate any available Pixel device (phone) |
| `"*"` | This will allocate a random available Android device |
| `"Pixel [8-9]"` | This will allocate either an Pixel 8 or 9 |
Some Examples:
```bash
# find any Samsung Galaxy, except the S23 family
-d '{"capabilities":[{"deviceName":"^Galaxy(?!.*23).*$", "platformName":"Android", "realDevice": true}]}'
# find any device which name starts with Pixel
-d '{"capabilities":[{"deviceName":"Pixel.*", "platformName":"Android", "realDevice": true}]}'
```
### Tablet Only
You can specify the `tabletOnly` capability when you only want to allocate a tablet device.
### Phone Only
You can specify the `phoneOnly` capability when you only want to allocate a phone device.
```bash
-d '{"capabilities":[{"deviceName":"*", "tabletOnly": true, "platformName":"Android", "realDevice": true}]}'
-d '{"capabilities":[{"deviceName":"*", "phoneOnly": true, "platformName":"Android", "realDevice": true}]}'
```
## Run Espresso Tests
Now that you have uploaded both your Android application and the Espresso test suite, you can run your first Espresso test on the TestingBot cloud.
[CLI](https://testingbot.com#) [cURL](https://testingbot.com#)
Use the CLI to run tests with device selection options:
```bash
# Basic usage with device selection
testingbot espresso app.apk app-test.apk --device "Pixel 8" --platform-version "14"
# Real device
testingbot espresso app.apk app-test.apk --device "Galaxy S24" --real-device
# With test name and build identifier
testingbot espresso app.apk app-test.apk \
--device "Pixel 8" \
--platform-version "14" \
--name "Smoke Tests" \
--build "v2.1.0"
```
### Device Options
| Option | Description |
| --- | --- |
| `--device ` | Device name (e.g., "Pixel 8", "Samsung.\*") |
| `--platform-version ` | Android version (e.g., "12", "13", "14") |
| `--real-device` | Use a real device instead of emulator |
| `--tablet-only` | Only allocate tablet devices |
| `--phone-only` | Only allocate phone devices |
#### Test Filtering
```bash
# Run specific test classes
testingbot espresso app.apk app-test.apk \
--device "Pixel 8" \
--class "com.example.LoginTest,com.example.HomeTest"
# Run tests with specific annotations
testingbot espresso app.apk app-test.apk \
--device "Pixel 8" \
--annotation "com.example.SmokeTest" \
--size "small,medium"
# Exclude specific test classes
testingbot espresso app.apk app-test.apk \
--device "Pixel 8" \
--not-class "com.example.SlowTest"
```
#### Test Filtering Options
| Option | Description |
| --- | --- |
| `--class ` | Run tests in specific classes (comma-separated fully qualified names) |
| `--not-class ` | Exclude tests in specific classes |
| `--package ` | Run tests in specific packages (comma-separated) |
| `--not-package ` | Exclude tests in specific packages |
| `--annotation ` | Run tests with specific annotations (comma-separated) |
| `--not-annotation ` | Exclude tests with specific annotations |
| `--size ` | Run tests by size: small, medium, large (comma-separated) |
#### Network & Location
```bash
# With network throttling and geolocation
testingbot espresso app.apk app-test.apk \
--device "Pixel 8" \
--throttle-network "3G" \
--geo-location "DE" \
--language "de"
```
```bash
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/espresso/:id/run" \
-d '{"capabilities":[{"version":"12", "deviceName":"Pixel 6", "platformName":"Android"}]}' \
-H "Content-Type: application/json"
```
Replace the `:id` with the identifier you received during the app upload call.
The API response will include a `success` response, indicating if the tests will start or if an error occurred.
```json
{
"success": true
}
```
## View Espresso Test Results
You will now see a test appear in the [TestingBot dashboard](https://testingbot.com/members), together with a video, test logs and other metadata.
You can also use the CLI or an API call to get the results from the run(s):
[CLI](https://testingbot.com#) [cURL](https://testingbot.com#)
By default, the CLI waits for test completion and displays real-time progress:
```bash
testingbot espresso app.apk app-test.apk --device "Pixel 8"
```
The CLI will show:
- Upload progress for app and test APK
- Device allocation status
- Live output from Espresso tests
- Final pass/fail status
### Async Mode
Use `--async` to start tests without waiting for results:
```bash
testingbot espresso app.apk app-test.apk --device "Pixel 8" --async
```
#### Download JUnit Report
Download the JUnit report after test completion:
```bash
testingbot espresso app.apk app-test.apk --device "Pixel 8" \
--report junit \
--report-output-dir ./reports
```
```bash
curl -u api_key:api_secret \
"https://api.testingbot.com/v1/app-automate/espresso/:id"
```
Replace the `:id` with the identifier you received during the app upload call.
The API response will include a response indicating the current state of the Espresso test results.
```json
{
"runs": [
{
"status": "DONE",
"capabilities": {
"version": "12",
"deviceName": "Pixel 6",
"platformName": "Android"
},
"success": false,
"test": {
"sessionId": "30f642d082c3-804f3c0ec2df-a18b2d431ffb-169920992163-48455547",
"environment": {
"name": "chrome",
"os": "Pixel 6 - 12.0",
"version": "12.0"
}
}
}
],
"version": "1.0"
}
```
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/app-automate/espresso/instrumentation
# Espresso Instrumentation Logs
Instrumentation logs are logs generated by Android Espresso. These logs list all the steps that were performed during the test, with any errors or meta data available from the Android Espresso test runner (usually `AndroidJUnitRunner`).
These logs are available in the [TestingBot dashboard](https://testingbot.com/members) as well as through the [REST API](https://testingbot.com/support/api).
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/app-automate/espresso/filter
# Filter Espresso Test Cases
By default, TestingBot will execute all tests present in the Espresso `.apk`.
The Espresso test runner (which is usually **AndroidJUnitRunner** ) provides multiple options to filter test cases for execution.
You might want to use these filter options for specific use cases, such as:
- Run only specific tests for a bugfix.
- Only run a subset of tests, for quick smoke testing. Instead of waiting on all your Espresso tests.
- Run specific tests for specific environments (production, staging, ...)
Below you'll find the different filter options that TestingBot and Espresso provide.
- [class](https://testingbot.com#class): will run all tests in a specific class
- [notClass](https://testingbot.com#notClass): will run all tests except the tests in a particular class
- [size](https://testingbot.com#size): will run a specific test size annotated with `SmallTest`, `MediumTest` or `LargeTest`
- [package](https://testingbot.com#package): will run all tests in a specific Java package
- [notPackage](https://testingbot.com#notPackage): will run all tests except the tests in a specific package
- [annotation](https://testingbot.com#annotation): will run all tests with a specific annotation
- [notAnnotation](https://testingbot.com#notAnnotation): will run all tests except the tests with a specific annotation
- [numShards](https://testingbot.com#shards): filter test run to a shard of all tests, where numShards is an integer greater than 0 and shardIndex is an integer between 0 (inclusive) and numShards (exclusive)
## class Filter
To only run Espresso tests for a specific `class`, you can use the class option.
You can pass a list of fully qualified Java class names. See the example below.
[CLI](https://testingbot.com#) [cURL](https://testingbot.com#)
```bash
testingbot espresso app.apk app-test.apk \
--device "Pixel 6" \
--platform-version "12" \
--class "com.android.app.one,com.android.app.two"
```
This will start an Espresso test session on the specified device and only run tests for the classes specified.
```bash
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/espresso/:id/run" \
-d '{ "espressoOptions": { "class": ["com.android.app.one", "com.android.app.two"] }, "capabilities":[{"version":"12", "deviceName":"Pixel 6", "platformName":"Android"}]}' \
-H "Content-Type: application/json"
```
This call with start an Espresso test Session on the specific device and only run the Espresso tests for the classNames specified.
## notClass Filter
Use the `notClass` option to run all Espresso tests except for the tests in the classes specified with this option.
You can pass a list of fully qualified Java class names. See the example below.
[CLI](https://testingbot.com#) [cURL](https://testingbot.com#)
```bash
testingbot espresso app.apk app-test.apk \
--device "Pixel 6" \
--platform-version "12" \
--not-class "com.android.app.one,com.android.app.two"
```
```bash
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/espresso/:id/run" \
-d '{ "espressoOptions": { "notClass": ["com.android.app.one", "com.android.app.two"] }, "capabilities":[{"version":"12", "deviceName":"Pixel 6", "platformName":"Android"}]}' \
-H "Content-Type: application/json"
```
## size Filter
The `size` option allows you to specify which tests you want to run, depending on the annotated size: `SmallTest`, `MediumTest` or `LargeTest`
You can pass an array of sizes, see below for an example.
[CLI](https://testingbot.com#) [cURL](https://testingbot.com#)
```bash
testingbot espresso app.apk app-test.apk \
--device "Pixel 6" \
--platform-version "12" \
--size "small,medium"
```
```bash
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/espresso/:id/run" \
-d '{ "espressoOptions": { "size": ["small", "medium"] }, "capabilities":[{"version":"12", "deviceName":"Pixel 6", "platformName":"Android"}]}' \
-H "Content-Type: application/json"
```
## package Filter
The `package` filter allows you to specify for which package the Espresso tests should run.
You can pass an array of packages, see below for an example.
[CLI](https://testingbot.com#) [cURL](https://testingbot.com#)
```bash
testingbot espresso app.apk app-test.apk \
--device "Pixel 6" \
--platform-version "12" \
--package "com.android.one,com.android.two"
```
```bash
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/espresso/:id/run" \
-d '{ "espressoOptions": { "package": ["com.android.one", "com.android.two"] }, "capabilities":[{"version":"12", "deviceName":"Pixel 6", "platformName":"Android"}]}' \
-H "Content-Type: application/json"
```
## notPackage Filter
The `notPackage` filter allows you to indicate that Espresso tests should run for everything except for the packages defined.
You can pass an array of packages, see below for an example.
[CLI](https://testingbot.com#) [cURL](https://testingbot.com#)
```bash
testingbot espresso app.apk app-test.apk \
--device "Pixel 6" \
--platform-version "12" \
--not-package "com.android.one,com.android.two"
```
```bash
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/espresso/:id/run" \
-d '{ "espressoOptions": { "notPackage": ["com.android.one", "com.android.two"] }, "capabilities":[{"version":"12", "deviceName":"Pixel 6", "platformName":"Android"}]}' \
-H "Content-Type: application/json"
```
## annotation Filter
The `annotation` filter will run tests annotated with the field (or fields) you specified.
You can pass an array of annotations, see below for an example.
[CLI](https://testingbot.com#) [cURL](https://testingbot.com#)
```bash
testingbot espresso app.apk app-test.apk \
--device "Pixel 6" \
--platform-version "12" \
--annotation "com.android.foo.MyAnnotation,com.android.foo.MyAnnotationTwo"
```
```bash
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/espresso/:id/run" \
-d '{ "espressoOptions": { "annotation": ["com.android.foo.MyAnnotation", "com.android.foo.MyAnnotationTwo"] }, "capabilities":[{"version":"12", "deviceName":"Pixel 6", "platformName":"Android"}]}' \
-H "Content-Type: application/json"
```
## notAnnotation Filter
The `notAnnotation` filter will run all tests except those annotated with the field (or fields) you specified.
You can pass an array of notAnnotations, see below for an example.
[CLI](https://testingbot.com#) [cURL](https://testingbot.com#)
```bash
testingbot espresso app.apk app-test.apk \
--device "Pixel 6" \
--platform-version "12" \
--not-annotation "com.android.foo.MyAnnotation,com.android.foo.MyAnnotationTwo"
```
```bash
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/espresso/:id/run" \
-d '{ "espressoOptions": { "notAnnotation": ["com.android.foo.MyAnnotation", "com.android.foo.MyAnnotationTwo"] }, "capabilities":[{"version":"12", "deviceName":"Pixel 6", "platformName":"Android"}]}' \
-H "Content-Type: application/json"
```
## shards Filter
You can split your Espresso tests across multiple parallel shards to reduce total run time. Pass `numShards` (total number of shards, an integer greater than 0) and `shardIndex` (which shard this run executes, an integer between 0 inclusive and `numShards` exclusive).
For example, to split into 4 shards and run shard 0 (the first), pass `numShards=4` and `shardIndex=0`. Submit one test run per shard index to run all four shards in parallel.
[CLI](https://testingbot.com#) [cURL](https://testingbot.com#)
```bash
testingbot espresso app.apk app-test.apk \
--device "Pixel 6" \
--platform-version "12" \
--num-shards 4 \
--shard-index 0
```
```bash
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/espresso/:id/run" \
-d '{ "espressoOptions": { "numShards": 4, "shardIndex": 0 }, "capabilities":[{"version":"12", "deviceName":"Pixel 6", "platformName":"Android"}]}' \
-H "Content-Type: application/json"
```
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/app-automate/espresso/options
# Set Espresso Test Runner
By default, TestingBot will run your Espresso tests with `${packageName}/androidx.test.runner.AndroidJUnitRunner` (TestingBot substitutes `${packageName}` with your app's package id automatically).
If you have defined your own `testInstrumentationRunner`, you can pass this custom runner during the start Espresso call:
[CLI](https://testingbot.com#) [cURL](https://testingbot.com#)
```bash
testingbot espresso app.apk app-test.apk \
--device "Pixel 6" \
--platform-version "12" \
--test-runner "${packageName}/customTestRunner"
```
```bash
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/espresso/:id/run" \
-d '{ "espressoOptions": { "testRunner": "${packageName}/customTestRunner" }, "capabilities":[{"version":"12", "deviceName":"Pixel 6", "platformName":"Android"}]}' \
-H "Content-Type: application/json"
```
Replace the `:id` with the identifier you received during the app upload call.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/app-automate/espresso/screenshots
# Capturing Espresso Screenshots
Espresso does not have a built-in way to capture screenshots during test automation. You can however achieve this by using two options:
- **Native Screenshot grabbing**
Use native Java or Kotlin code to take a screenshot of the device, and save it on the device.
Once the test is completed, you can fetch the image through `ADB`, or if you run it via TestingBot, the screenshot will appear on the [test result page](https://testingbot.com#view).
- **Using the Spoon library**
The [Square Screenshots feature](https://square.github.io/spoon/#screenshots) is part of the Spoon Library.
It will take screenshots during your Espresso tests, but only for Android devices with a version of Android 10 or lower.
[Using Native Screenshot](https://testingbot.com#) [Using Spoon Library](https://testingbot.com#)
1. **Add the native screenshots file to your project**
You can see the implementation of Native Screenshots in the [TestingBot Espresso Demo App](https://github.com/testingbot/android-espresso-demo-app/blob/main/app/src/androidTest/java/com/testingbot/calculator/NativeScreenshot.java).
```java
package com.testingbot.calculator;
import android.graphics.Bitmap;
import android.os.Build;
import android.os.Environment;
import androidx.test.runner.screenshot.BasicScreenCaptureProcessor;
import androidx.test.runner.screenshot.ScreenCapture;
import androidx.test.runner.screenshot.Screenshot;
import java.io.File;
import java.io.IOException;
import java.util.regex.Pattern;
public final class NativeScreenshot {
private static String methodName;
private static String className;
private static final Pattern SCREENSHOT_NAME_VALIDATION = Pattern.compile("[a-zA-Z0-9_-]+");
private NativeScreenshot() {}
/**
* Captures screenshot using Androidx Screenshot library and stores in the filesystem.
* If there is any runtime exception while capturing screenshot, the method throws
* Exception and the test might fail if exception is not handled properly.
* @param screenshotName a screenshot identifier
* @return path to the screenshot file
*/
public static String capture(String screenshotName) {
StackTraceElement testClass = findTestClassTraceElement(Thread.currentThread().getStackTrace());
className = testClass.getClassName().replaceAll("[^A-Za-z0-9._-]", "_");
methodName = testClass.getMethodName();
EspressoScreenCaptureProcessor screenCaptureProcessor = new EspressoScreenCaptureProcessor();
if (!SCREENSHOT_NAME_VALIDATION.matcher(screenshotName).matches()) {
throw new IllegalArgumentException("ScreenshotName must match " + SCREENSHOT_NAME_VALIDATION.pattern() + ".");
} else {
ScreenCapture capture = Screenshot.capture();
capture.setFormat(Bitmap.CompressFormat.PNG);
capture.setName(screenshotName);
try {
return screenCaptureProcessor.process(capture);
} catch (IOException e) {
throw new RuntimeException("Unable to capture screenshot.", e);
}
}
}
/**
* Extracts the currently executing test's trace element based on the test runner
* or any framework being used.
* @param trace stacktrace of the currently running test
* @return StackTrace Element corresponding to the current test being executed.
*/
private static StackTraceElement findTestClassTraceElement(StackTraceElement[] trace) {
for(int i = trace.length - 1; i >= 0; --i) {
StackTraceElement element = trace[i];
if ("android.test.InstrumentationTestCase".equals(element.getClassName()) && "runMethod".equals(element.getMethodName())) {
return extractStackElement(trace, i);
}
if ("org.junit.runners.model.FrameworkMethod$1".equals(element.getClassName()) && "runReflectiveCall".equals(element.getMethodName())) {
return extractStackElement(trace, i);
}
if ("cucumber.runtime.model.CucumberFeature".equals(element.getClassName()) && "run".equals(element.getMethodName())) {
return extractStackElement(trace, i);
}
}
throw new IllegalArgumentException("Could not find test class!");
}
/**
* Based on the test runner or framework being used, extracts the exact traceElement.
* @param trace stacktrace of the currently running test
* @param i a reference index
* @return trace element based on the index passed
*/
private static StackTraceElement extractStackElement(StackTraceElement[] trace, int i) {
int testClassTraceIndex = Build.VERSION.SDK_INT >= 23 ? i - 2 : i - 3;
return trace[testClassTraceIndex];
}
private static class EspressoScreenCaptureProcessor extends BasicScreenCaptureProcessor {
private static final String SCREENSHOT = "screenshots";
EspressoScreenCaptureProcessor() {
File screenshotDir = new File(String.valueOf(Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_DOWNLOADS)), SCREENSHOT);
File classDir = new File(screenshotDir, className);
mDefaultScreenshotPath = new File(classDir, methodName);
}
/**
* Converts the filename to a standard path to be stored on device.
* Example: "post_addition" converts to "1648038895211_post_addition"
* which is later suffixed by the file extension i.e. png.
* @param filename a screenshot identifier
* @return custom filename format
*/
@Override
protected String getFilename(String filename) {
return System.currentTimeMillis() + "_" + filename;
}
}
}
```
2. **Modify your Espresso test scripts to take screenshots**
You can call the `NativeScreenshot.capture("tag")` method to capture screenshots in your Espresso tests.
The `tag` can be used to provide a name to the screenshot (usually you would use the testname for this).
```java
@Test
public void ensureCalculatorWorks() {
onView(withId(R.id.buttonOne)).perform(click());
onView(withId(R.id.buttonAdd)).perform(click());
onView(withId(R.id.buttonTwo)).perform(click());
onView(withId(R.id.editText)).check(matches(withText("3")));
NativeScreenshot.capture("ensureCalculatorWorks");
}
```
3. **Android 11 and higher**
For Android 11 and higher, make sure to add the following `RuleChain` to your Espresso test:
```java
@Rule
public RuleChain screenshotRule = RuleChain
.outerRule(GrantPermissionRule.grant(Manifest.permission.WRITE_EXTERNAL_STORAGE, Manifest.permission.READ_EXTERNAL_STORAGE))
.around(activityScenarioRule);
```
1. **Add the required dependencies to your project**
See the [Spoon Documentation](https://github.com/square/spoon#download) on how to add Spoon to your project.
You can either use a `JAR` or use `Maven`.
2. **Change storage permissions**
If you do not yet have the `WRITE_EXTERNAL_STORAGE` permission in your `AndroidManifest.xml`, please add it.
It will allow Spoon to save the screenshots on your device.
```xml
...
```
If the targetSdk for your app is using Android 10 (level 29), please make sure to add `android:requestLegacyExternalStorage="true"` in the Application tag of your `AndroidManifest.xml` file.
3. **Modify your Espresso test scripts to take screenshots**
You can call the `Spoon.screenshot(activityName, tag)` method to capture screenshots in your Espresso tests.
The `tag` can be used to provide a name to the screenshot (usually you would use the testname for this).
```java
@Rule
public ActivityScenarioRule activityRule =
new ActivityScenarioRule<>(MainActivity.class);
@Test
public void ensureCalculatorWorks() {
onView(withId(R.id.buttonOne)).perform(click());
onView(withId(R.id.buttonAdd)).perform(click());
onView(withId(R.id.buttonTwo)).perform(click());
onView(withId(R.id.editText)).check(matches(withText("3")));
activityRule.getScenario().onActivity(activity ->
Spoon.screenshot(activity, "ensureCalculatorWorks"));
}
```
## Viewing Espresso Screenshots
The Espresso tests that you run on TestingBot will appear automatically in the TestingBot dashboard.
If screenshots were generated during the test, they will be added on the Espresso test result page.
Additionally, these screenshots will also be available for download through our [REST API](https://testingbot.com/support/api).
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/app-automate/espresso/set-ip-geolocation
# Use IP geolocation
TestingBot's GeoIP feature allows you to test your app from various parts of the world, by using IP addresses in [over 20 different countries](https://testingbot.com/support/web-automate/selenium/test-options#geo).
When you enable GeoIP, all traffic from your mobile Android app will originate from a specific IP address hosted in the country that you choose.
## GeoIP example
To use GeoIP, you can specify the geoIP option when starting an Espresso test.
Please see the example below where we specify that the app should connect via Germany.
[CLI](https://testingbot.com#) [cURL](https://testingbot.com#)
```bash
testingbot espresso app.apk app-test.apk \
--device "Pixel 6" \
--platform-version "12" \
--geo-location "DE"
```
The `DE` abbreviation is the ISO code of Germany. You can find other [GeoIP country codes](https://testingbot.com/support/web-automate/selenium/test-options#geo) to use.
```bash
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/espresso/:id/run" \
-d '{ "espressoOptions": { "geoLocation": "DE" }, "capabilities":[{"version":"12", "deviceName":"Pixel 6", "platformName":"Android"}]}' \
-H "Content-Type: application/json"
```
The `DE` abbreviation is the ISO code of Germany. You can find other [GeoIP country codes](https://testingbot.com/support/web-automate/selenium/test-options#geo) to use.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/app-automate/espresso/set-localization-options
# Set localization options
With TestingBot you can run Android Espresso tests on a localized version of your Android app.
You can configure the language, locale and timezone of the remote device before running an Android Espresso test.
## Setting Language for Android Espresso Tests
You can test a localized version of your app with Android Espresso, by changing the language of the application under test.
You will need to specify a `language` option with the ISO code of the language you want to use.
[CLI](https://testingbot.com#) [cURL](https://testingbot.com#)
```bash
testingbot espresso app.apk app-test.apk \
--device "Pixel 6" \
--platform-version "12" \
--language "fr"
```
Use an [ISO 639-1 language code](https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes) to set the language of your app.
Your app needs to be compiled with support for this language to notice a change.
```bash
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/espresso/:id/run" \
-d '{ "espressoOptions": { "language": "fr" }, "capabilities":[{"platform":"ANDROID", "version":12, "deviceName":"Pixel 6", "platformName":"Android"}]}' \
-H "Content-Type: application/json"
```
Use an [ISO 639-1 language code](https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes) to set the language of your app.
Your app needs to be compiled with support for this language to notice a change.
## Set a locale for Android Espresso Testing
It is possible to set a locale, different than the default device's locale, with the `locale` parameter.
Please use the [ISO 3166-1 alpha-2](https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2) country code. For example: `DE` for Germany.
[CLI](https://testingbot.com#) [cURL](https://testingbot.com#)
```bash
testingbot espresso app.apk app-test.apk \
--device "Pixel 6" \
--platform-version "12" \
--locale "DE"
```
```bash
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/espresso/:id/run" \
-d '{ "espressoOptions": { "locale": "DE" }, "capabilities":[{"version":"12", "deviceName":"Pixel 6", "platformName":"Android"}]}' \
-H "Content-Type: application/json"
```
## Change timezone for Android Espresso
It might be necessary during your tests to change the timezone to a different timezone than the default one on the device (UTC).
Please use a timezone format from the [list of timezones](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones).
[CLI](https://testingbot.com#) [cURL](https://testingbot.com#)
```bash
testingbot espresso app.apk app-test.apk \
--device "Pixel 6" \
--platform-version "12" \
--timezone "America/New_York"
```
```bash
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/espresso/:id/run" \
-d '{ "espressoOptions": { "timeZone": "America/New_York" }, "capabilities":[{"version":"12", "deviceName":"Pixel 6", "platformName":"Android"}]}' \
-H "Content-Type: application/json"
```
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/app-automate/espresso/simulate-network-conditions
# Simulate Network Conditions
TestingBot allows you to test your mobile apps under various network conditions, including download speed, upload speed, packet loss and latency.
## Simulate network conditions using a predefined network profile
TestingBot provides a set of predefined network profiles which you can choose from.
Profile Name | Bandwidth down/up (kbps) | Packet Loss (%) | Latency (ms) || Edge | 250/150 | 0 | 300 |
| 3G | 400/100 | 0 | 100 |
| 4G | 18000/9000 | 0 | 100 |
| airplane | 0 | 100 | 0 |
[CLI](https://testingbot.com#) [cURL](https://testingbot.com#)
```bash
testingbot espresso app.apk app-test.apk \
--device "Pixel 6" \
--platform-version "12" \
--throttle-network "3G"
```
```bash
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/espresso/:id/run" \
-d '{ "espressoOptions": { "throttle_network": "3G" }, "capabilities":[{"version":"12", "deviceName":"Pixel 6", "platformName":"Android"}]}' \
-H "Content-Type: application/json"
```
## Simulate network conditions using a custom network profile
You can specify a custom set of parameters, including:
- Download rate (kbps)
- Upload rate (kbps)
- Packet loss (%)
- Latency (ms)
[CLI](https://testingbot.com#) [cURL](https://testingbot.com#)
The CLI supports predefined network profiles. For custom network profiles, use the cURL API directly.
```bash
# Predefined profiles: 4G, 3G, Edge, airplane
testingbot espresso app.apk app-test.apk \
--device "Pixel 6" \
--platform-version "12" \
--throttle-network "Edge"
```
```bash
curl -u api_key:api_secret \
-X POST "https://api.testingbot.com/v1/app-automate/espresso/:id/run" \
-d '{ "espressoOptions": { "throttle_network": { "uploadSpeed": 10240, "downloadSpeed": 10240, "latency": 0, "loss": 0 } }, "capabilities":[{"version":"12", "deviceName":"Pixel 6", "platformName":"Android"}]}' \
-H "Content-Type: application/json"
```
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/app-automate/espresso/test-reports
# Espresso Test Reports
TestingBot currently offers both test reports from inside the TestingBot member dashboard, or through REST-API where you can use a [JUnit formatted XML report](https://testingbot.com#junit).
## Dashboard Report
When you run Espresso Tests with TestingBot, you will find the test results, together with a video of the test, in the [TestingBot member dashboard](https://testingbot.com/members).
## API report
You can query the TestingBot API to fetch the result of the Espresso test run(s).
[CLI](https://testingbot.com#) [cURL](https://testingbot.com#)
By default, the CLI waits for test completion and displays results in real-time:
```bash
testingbot espresso app.apk app-test.apk --device "Pixel 6"
```
The CLI will show:
- Upload progress for app and test APK
- Device allocation status
- Live output from Espresso tests
- Final pass/fail status
Use `--async` to start tests without waiting for results:
```bash
testingbot espresso app.apk app-test.apk --device "Pixel 6" --async
```
```bash
curl -u api_key:api_secret \
"https://api.testingbot.com/v1/app-automate/espresso/:id/:run_id"
```
Replace the `:id` and `:run_id` with the identifiers you received during the app upload call and the run call.
You will get back a response similar to this one:
```json
{"id":86,"created_at":"2023-11-05T18:45:51.000Z","status":"DONE","capabilities":{"version":"12","deviceName":"Pixel 6","platformName":"Android"},"success":false,"test":{"sessionId":"30f642d082c3-804f3c0ec2df-a18b2d431ffb-169920992163-48455547","environment":{"name":"chrome","os":"Pixel 6 - 12.0","version":"12.0"}},"version":"1.0"}
```
## JUnit XML report
You can query our API to fetch a JUnit XML report which you can use a post-build step in your CI/CD environment.
**Report Structure**
In the XML report, each session maps to a `testsuite` tag. Inside that tag, multiple `testcase` items are available, each mapping to a single Espresso test.
If a test failed, a `failure` or `error` tag will be available.
[CLI](https://testingbot.com#) [cURL](https://testingbot.com#)
Download the JUnit XML report after test completion:
```bash
testingbot espresso app.apk app-test.apk \
--device "Pixel 6" \
--report junit \
--report-output-dir ./reports
```
The schema corresponds to the Android Studio's Gradle schema used when running JUnit tests.
```bash
curl -u api_key:api_secret \
"https://api.testingbot.com/v1/app-automate/espresso/:id/report"
```
The schema corresponds to the Android Studio's Gradle schema used when running JUnit tests.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/app-automate/help
# Mobile App Help
Please see the topic below for help with Automated App testing:
## Help Topics
- [Prepare your App](https://testingbot.com/support/app-automate/help/prepare)
- [Upload your App](https://testingbot.com/support/app-automate/help/upload)
- [Tunnel + Real Devices](https://testingbot.com/support/app-automate/help/tunnel)
- [FAQ](https://testingbot.com/support/app-automate/help/faq)
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/app-automate/help/prepare
# Prepare your Mobile App for Automated Testing
TestingBot provides Android Emulators and iOS Simulators as well as [Real Devices](https://testingbot.com/support/app-automate/devices) to run tests against your mobile app.
Below are the steps necessary to prepare your mobile app for Automated Testing.
## Building your app
You do not need to install any code or plugin to run a test.
Below are some things that are necessary to successfully run a mobile test:
### For Android:
- Please supply the URL to your `.apk` or `.aab` file.
**Important:** the .apk file needs to be compiled for the emulator/device you are running your test on (x86 or arm).
- Your app needs to have internet permissions
- The URL to the `.apk` file must be publicly available, or you can use our [TestingBot Storage](https://testingbot.com/support/app-automate/help/upload) to temporarily host your mobile app.
### For iOS (Real Device):
- Please supply the URL to an [.ipa file](https://testingbot.com#ipa). The URL must be publicly available, or you can use our [TestingBot Storage](https://testingbot.com/support/app-automate/help/upload) to temporarily host your mobile app.
- By default, we [automatically resign](https://testingbot.com/support/app-automate/help/app-resigning) your [.ipa file](https://testingbot.com#ipa) so that your development builds can run on our devices.
If you don't want this, specify `resigningEnabled: false` in your `tb:options` capabilities.
- Please make sure your target version matches the version of the simulator you want to test on.
### For iOS (Simulator):
- Please supply the URL to a .zip file that contains your .app
- The .app needs to be an iOS Simulator build:
- Create a Simulator build:
```bash
xcodebuild -sdk iphonesimulator -configuration Debug
```
- Zip the .app bundle:
```bash
zip -r MyApp.zip MyApp.app
```
- Please make sure your target version matches the version of the simulator you want to test on.
## Building IPA file
To build an IPA file, please follow these steps:
- Open your project in XCode.
- Select **Generic iOS Device** as a project device target.
- In the **Product** menu, select **Archive**.
- Once archived, click **Export**.
- You will be asked what kind of export mode you want, you can pick any of these methods: **Ad Hoc** , **Enterprise** or **Development mode**.
- When asked for **Distribution options** , please set these options:
- App Thinning: disabled
- Uncheck **Rebuild from Bitcode**
- Uncheck **Strip Swift Symbols**
- Uncheck **Include manifest for over-the-air installation**
- Select your **Distribution Certificate** and **Provisioning Profile (either Automatic or Manual)**.
- Now your `IPA` file should be generated, export it to a folder on your hard disk.
- You can now upload the `IPA` file to a public URL or use our [TestingBot Storage](https://testingbot.com/support/app-automate/help/upload) to temporarily host your app.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/app-automate/help/upload
# Upload your App
In order to run tests against your mobile app, our devices need to be able to download your app.
There are two ways to configure your tests to do this:
- Upload your app to TestingBot Storage: our private upload servers.
- Specify an HTTP or HTTPS url to the app you uploaded on a public internet website.
## TestingBot Storage
With TestingBot Storage, you can upload your app to our servers.
The advantage of this is that your tests will start faster, as the device can download the app from our internal network, instead of the public internet.
To get started, simply submit your app to our API:
```bash
$ curl -X POST "https://api.testingbot.com/v1/storage" \
-u "key:secret" -F "file=@/path/to/app/file/Application-debug.apk"
```
This call will return a unique identifier for your app (`{"app_url": "tb://..."}`), which you can use in the desired capabilities (see example below).
More information regarding this API call and other similar calls (update uploaded file, delete uploaded file, list uploaded files) are available in [our API documentation](https://testingbot.com/support/api/storage#upload).
Note: uploads to TestingBot Storage are automatically deleted after 60 days.
```ruby
require 'appium_lib'
caps = {
"platformName" => "iOS",
"appium:deviceName" => "iPhone 15",
"appium:platformVersion" => "18.0",
"appium:app" => "tb://...", # specify the link to your app here
"tb:options" => {
"name" => "single_test",
"build" => "Ruby Appium Sample"
}
}
appium_driver = Appium::Driver.new({
caps: caps,
appium_lib: {
server_url: "https://API_KEY:API_SECRET@hub.testingbot.com/wd/hub"
}}, true)
driver = appium_driver.start_driver
driver.find_element(:name, "inputA").send_keys 5
driver.find_element(:name, "inputB").send_keys 10
driver.quit
```
## Public Internet
If you have your app available on a public website, you can specify this in the desired capabilities:
```ruby
require 'appium_lib'
caps = {
"platformName" => "iOS",
"appium:deviceName" => "iPhone 15",
"appium:platformVersion" => "18.0",
"appium:app" => "https://...", # specify the link to your app here
"tb:options" => {
"name" => "single_test",
"build" => "Ruby Appium Sample"
}
}
appium_driver = Appium::Driver.new({
caps: caps,
appium_lib: {
server_url: "https://API_KEY:API_SECRET@hub.testingbot.com/wd/hub"
}}, true)
driver = appium_driver.start_driver
driver.find_element(:name, "inputA").send_keys 5
driver.find_element(:name, "inputB").send_keys 10
driver.quit
```
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/app-automate/help/tunnel
# Using TestingBot Tunnel with Real Devices
With TestingBot tunnel, we create a secure connection between a real device running in our datacenter, and your computer and internal network.
This way, you can run tests against apps using internal (staging) APIs, from our real mobile devices. Or test websites running on your local network through Safari/Chrome running on a real device.
TestingBot Tunnel in combination with Real Devices is available for both **public devices** and **private devices**.
More information regarding our TestingBot Tunnel is available on our [TestingBot Tunnel documentation page](https://testingbot.com/support/tunnel).
## Known Limitations
- **Localhost testing does not work on iOS**.
Due to an iOS Restriction, it is currently not possible to run tests against `http(s)://localhost` with TestingBot Tunnel.
A possible workaround would be to change the hosts file on the machine running TestingBot Tunnel, and add a custom hostname (for example `mylocalwebsite` and the IP address `127.0.0.1`).
Then configure your tests to use `http(s)://mylocalwebsite/...` instead of localhost.
## Security and Privacy (Advisory)
At TestingBot, we clean each device after every test. We take the utmost care to clean as much from the device as we can, to make sure your test runs without any leftover artifacts from previous tests.
However, due to the complexity of automatically cleaning different mobile device models, we cannot 100% guarantee that all data generated from a previous session is removed from our **public devices**.
Using our TestingBot tunnel in combination with a public device might have potential security risks.
We recommend to [opt for private, dedicated devices](https://testingbot.com/enterprise/private-device-cloud) for maximum security.
With **private devices** , your company will be the only one running tests on the device.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/app-automate/help/faq
# Mobile App Testing FAQ
TestingBot provides mobile application testing on both **virtual devices** as well as **physical devices**.
Virtual devices are Android emulators and iOS simulators.
TestingBot provides a large collection of these, available for mobile app testing, both automated and manual testing.
This FAQ document will list questions and answers regarding the mobile native app testing service.
## Can I run Android emulator tests using Espresso instead of Appium?
TestingBot allows you to upload native mobile Android apps and use [Espresso testing](https://testingbot.com/support/app-automate/espresso) on both Android emulators and physical devices.
Of course, you can do [Appium automated app testing](https://testingbot.com/support/app-automate/appium) as well on TestingBot.
## Which apps are available on the devices?
TestingBot uses stock images for Android emulators, which come equipped with the PlayStore and Chrome.
Similarly, we use stock iOS simulators which come with Safari by default.
TestingBot does not use jailbroken or rooted physical devices.
All Android devices come with factory settings, with PlayStore enabled (if available) and Chrome.
The iOS devices come with Safari, the Safari version depends on the iOS version installed on the device.
## How does manual testing work?
We provide access to a realtime view of an Android emulator, iOS simulator, Android physical device and iOS device.
You can interact with the device, from your own mouse and keyboard, without installing any plugins or software.
The screens are streamed either through WebRTC or VNC, depending on your network connectivity settings.
## When are new devices added?
Our public device pool contains a collection of the most popular devices used, backed by public statistics of mobile device usage.
When new devices are released to the public, we aim to procure, setup and make these available to our customers in less than 2 weeks.
The criteria we use for selecting which devices to add depends on the popularity of the device and the demand for testing on such a device.
If you are looking to test on a specific device, please do not hesitate to [reach out to us](https://testingbot.com/contact/new).
## Is it possible to test push notifications?
It is possible to test push notifications in the following scenarios:
- **Android Physical Devices**
It will work automatically. Simply install the app and allow push notifications.
- **iOS Physical Devices**
This is only possible for private devices, which require an [Enterprise plan](https://testingbot.com/enterprise). [Reach out to us](https://testingbot.com/contact/new) for more information.
- **Android Emulators**
Unfortunately this is not supported yet.
If you'd like to be a part of the beta program, please let us know.
- **iOS Simulators**
This is possible since Appium 1.21.0.
## Can I upload or download files from a device or sim/emu?
Yes, you can upload files to devices using Appium's `pushFile` command and download files using the `pullFile` command.
Please see our [Upload Files documentation](https://testingbot.com/support/app-automate/appium/upload-files) for more information.
## How is my app protected during mobile app testing?
Our Real Device Cloud is located in a secure datacenter in Europe. We make sure that every device is fully protected, both physically and digitally.
All traffic happens through SSL and each device is cleaned after usage:
- All user apps are removed
- Generated data, such as browser data, is wiped
- SD card, accounts, and other data that was altered is removed
TestingBot's [automated cleaning process](https://testingbot.com/support/app-automate/help/cleaning) makes sure that no data is available from an earlier test.
If you think this did happen, please reach out to us so we can reset the device and check why this happened.
## Is audio supported, or microphone access?
Audio is available for Android devices running Android 12 and higher.
Older physical devices do not have the capability to stream audio, these will only stream a realtime video of the screen.
## My app needs to make requests to internal services. Can I use a VPN for this?
We suggest using the [TestingBot Tunnel](https://testingbot.com/support/tunnel). It will set up the device or sim/emu to proxy all requests through the tunnel to your internal services.
An alternative solution is to [whitelist our IP addresses](https://testingbot.com/support/other/configuration#ip-range), which would allow our devices to reach your protected services.
## Can I use Bluetooth on the physical devices?
Yes you can, please [reach out](https://testingbot.com/contact/new) to discuss your specific use case.
## Can I trigger a disconnect or reconnect from a charging cable?
This is currently not supported.
## Can I change the orientation of the device?
Yes, both on simulators/emulators as well as on physical devices. You can change the orientation to either `PORTRAIT` or `LANDSCAPE`.
You can do this via our manual testing feature, or [change the screen orientation during an automated test](https://testingbot.com/support/app-automate/appium/screen-orientation).
## Are code coverage reports by JaCoCo supported?
We currently do not support this.
## Can I change the geolocation during my test?
Yes, you can [change the geolocation](https://testingbot.com/support/web-automate/selenium/test-options#geo) either during manual testing or via automated tests.
## Why do I sometimes see a lock/PIN screen?
On trial devices, in our public device pool, we need to make some areas password protected to prevent users from changing certain settings.
The password protection is not in place on non-trial devices.
## Can I test payments on the physical devices?
Payments can be tested on private devices only. Please [reach out to us](https://testingbot.com/contact/new) for more information.
## Can I inspect the mobile app in your manual testing feature?
Yes, you can inspect the mobile app from the manual testing feature. Simply open the **DevTools** option and click the **Inspector** tab.
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/app-automate/help/cleaning
# Real Device Cleaning Process
TestingBot automatically cleans each physical mobile device after every test session. We do this to ensure each new test starts with a clean state, without any leftover data from previous sessions.
The cleaning process we use is automated for both iOS and Android. For devices in the public device pool, each test session automatically ends with a thorough cleaning process where we delete any apps installed by the user, together with various other files and data that were generated.
If you are using a private device, there's the possibility to alter or completely disable the cleaning process. You would for example use this when you want to make sure that a specific state is kept between various test sessions.
## Cleaning Process
TestingBot uses a proprietary process to clean its devices. Below are the various processes that happen when a device is being cleaned.
- All apps that have been installed by the customer are deleted. Some applications, which are used by TestingBot to operate the device, are kept.
- The browser history, together with any cached website data and cookies are removed.
- AppleID sessions are removed.
- Network settings such as WiFi and DNS configurations are reset to their default state.
- The system locale, language and time zone are reset to their default. (English, UTC+2).
- Any mock GPS location is reset to Brussels, Belgium.
- Media files such as Photos, Videos and other Files are removed from the device.
TestingBot's cleaning process aims to clean as much as possible from the device. If you prefer the guarantee of a pristine device, please consider using a [private mobile device](https://testingbot.com/enterprise/private-device-cloud).
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/app-automate/help/app-resigning
# iOS App Resigning
When you [upload an iOS app](https://testingbot.com/support/app-automate/help/upload) to TestingBot, we will automatically re-sign the `.ipa` file with our own provisioning profile, to be able to install and run your app on our mobile devices.
Because of this procedure, the [entitlements](https://developer.apple.com/documentation/bundleresources/entitlements) of your iOS app will be removed.
You can choose to disable this re-signing, if your app is signed using the [Apple Developer Enterprise Program](https://developer.apple.com/programs/enterprise/). This way you can test **push notifications** or [universal links](https://developer.apple.com/ios/universal-links/) on our physical devices.
## Disable iOS Resigning
[Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [PHP](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#)
```ruby
capabilities = {
"tb:options" => {
"resigningEnabled" => false
}
}
```
```java
Map tbOptions = new HashMap<>();
tbOptions.put("resigningEnabled", false);
options.setCapability("tb:options", tbOptions);
```
```php
$capabilities = [
"tb:options" => [
"resigningEnabled" => false
]
];
```
```python
capabilities = {
"tb:options": {
"resigningEnabled": False
}
}
```
```javascript
const capabilities = {
"tb:options": {
"resigningEnabled": false
}
};
```
```csharp
var tbOptions = new Dictionary
{
["resigningEnabled"] = false
};
options.AddAdditionalAppiumOption("tb:options", tbOptions);
```
## Looking for more help?
Have questions or need more information? Reach out via email or Slack.
[Email us](https://testingbot.com/contact/new) [Join our Slack](https://join.slack.com/t/testingb0t/shared_invite/zt-3bcw9xch-jk19~6XPs_xBrsAgAedkCw)
---
URL: https://testingbot.com/support/app-automate/smart-tv
# Smart TV Testing
Run tests on physical Smart TV devices with TestingBot, using the open-source test framework Appium.
Currently TestingBot supports running automated tests on physical AppleTV 4k devices.
To get started with testing your Smart TV app in the cloud, you will need to follow these steps:
- Upload your native Smart TV app, either to a public URL or using TestingBot Storage.
- Create an Appium test script that will connect to TestingBot.
- The results will be available in the TestingBot dashboard, together with a video of the test and logs.
## Upload Smart TV App
To run Smart TV tests on TestingBot, you will first have to upload your `.ipa` file to TestingBot Storage.
[cURL](https://testingbot.com#)
```bash
$ curl -X POST "https://api.testingbot.com/v1/storage" \
-u key:secret -F "file=@/path/to/app/file/tvos.ipa"
```
This call will return a unique identifier for your SmartTV app (`{"app_url": "tb://..."}`), which you can use in the capabilities in your test (see example below).
More information regarding this API call and other similar calls (update uploaded file, delete uploaded file, list uploaded files) are available in [the TestingBot API documentation](https://testingbot.com/support/api/storage#upload).
Note: uploads to TestingBot Storage are automatically deleted after 60 days.
## Run your first SmartTV test
Once you've [uploaded your Smart TV app](https://testingbot.com#upload), you can start with running your first test on the TestingBot SmartTV cloud.
TestingBot currently only supports running tests on physical AppleTV 4k devices. Please see the example tests below, where we use the [TestingBot tvOS example app](https://github.com/testingbot/tvos-example) to input two numbers in the calculator and verify its sum.
[Ruby](https://testingbot.com#) [Python](https://testingbot.com#) [Java](https://testingbot.com#) [NodeJS](https://testingbot.com#) [C#](https://testingbot.com#)
```ruby
require 'rubygems'
require 'appium_lib'
caps = {}
caps['name'] = 'Ruby tvOS Example'
caps['deviceName'] = 'Apple TV 4K'
caps['platformName'] = 'tvOS'
caps['version'] = '17.4'
caps['app'] = 'https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa'
caps['realDevice'] = true
appium_driver = Appium::Driver.new({
caps: caps,
appium_lib: {
server_url: "https://key:secret@hub.testingbot.com/wd/hub"
}}, true)
driver = appium_driver.start_driver
wait = Selenium::WebDriver::Wait.new(:timeout => 30)
wait.until { driver.find_element(:accessibility_id, "inputField1").displayed? }
inputField1 = driver.find_element(:accessibility_id, "inputField1")
inputField1.click
inputField1.send_keys 5
driver.execute_script('mobile: pressButton', { name: 'down' })
driver.execute_script('mobile: pressButton', { name: 'select' })
driver.switch_to.active_element.send_keys 10
driver.execute_script('mobile: pressButton', { name: 'down' })
driver.execute_script('mobile: pressButton', { name: 'select' })
result = driver.find_element(:accessibility_id, "resultField")
if (!result.nil?) && (result.text.to_i == 15)
puts "Test Passed"
else
puts "Test Failed"
end
driver.quit
```
```javascript
const wdio = require('webdriverio');
const caps = {
name: 'Node.js tvOS Example',
deviceName: 'Apple TV 4K',
platformName: 'tvOS',
platformVersion: '17.4',
app: 'https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa',
realDevice: true
};
const options = {
protocol: 'https',
hostname: 'hub.testingbot.com',
port: 443,
path: '/wd/hub',
user: 'key', // Replace 'key' with your TestingBot key
key: 'secret', // Replace 'secret' with your TestingBot secret
capabilities: caps
};
(async () => {
// Start the Appium driver
const driver = await wdio.remote(options);
try {
// Wait for the first input field to be displayed
const inputField1 = await driver.$('~inputField1');
await inputField1.waitForDisplayed({ timeout: 30000 });
// Interact with the first input field
await inputField1.click();
await inputField1.setValue(5);
// Navigate down and select
await driver.execute('mobile: pressButton', { name: 'down' });
await driver.execute('mobile: pressButton', { name: 'select' });
// Send keys to the active element
await driver.switchToParentFrame(); // Ensure focus is on the active frame
await driver.activeElement().setValue(10);
// Navigate down and select the result
await driver.execute('mobile: pressButton', { name: 'down' });
await driver.execute('mobile: pressButton', { name: 'select' });
// Verify the result
const result = await driver.$('~resultField');
const resultText = await result.getText();
if (resultText && parseInt(resultText, 10) === 15) {
console.log('Test Passed');
} else {
console.log('Test Failed');
}
} catch (err) {
console.error('Error occurred:', err);
} finally {
// Quit the driver
await driver.deleteSession();
}
})();
```
```python
from appium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
# Define capabilities
caps = {
"name": "Python tvOS Example",
"deviceName": "Apple TV 4K",
"platformName": "tvOS",
"platformVersion": "17.4",
"app": "https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa",
"realDevice": True
}
# Appium server URL
server_url = "https://key:secret@hub.testingbot.com/wd/hub"
# Initialize the driver
driver = webdriver.Remote(server_url, caps)
try:
# Wait for the first input field to be displayed
wait = WebDriverWait(driver, 30)
input_field1 = wait.until(EC.presence_of_element_located((By.ACCESSIBILITY_ID, "inputField1")))
# Interact with the first input field
input_field1.click()
input_field1.send_keys("5")
# Navigate down and select the second input field
driver.execute_script('mobile: pressButton', {'name': 'down'})
driver.execute_script('mobile: pressButton', {'name': 'select'})
# Send keys to the active element (second input field)
driver.switch_to.active_element.send_keys("10")
# Navigate down and select the result field
driver.execute_script('mobile: pressButton', {'name': 'down'})
driver.execute_script('mobile: pressButton', {'name': 'select'})
# Verify the result
result_field = driver.find_element(By.ACCESSIBILITY_ID, "resultField")
result_text = result_field.text
if result_text and int(result_text) == 15:
print("Test Passed")
else:
print("Test Failed")
except Exception as e:
print(f"An error occurred: {e}")
finally:
# Quit the driver
driver.quit()
```
```java
import io.appium.java_client.AppiumDriver;
import io.appium.java_client.MobileBy;
import io.appium.java_client.remote.MobileCapabilityType;
import org.openqa.selenium.By;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.remote.DesiredCapabilities;
import java.net.URL;
import java.time.Duration;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;
public class TvOSTest {
public static void main(String[] args) {
AppiumDriver driver = null;
try {
// Set capabilities
DesiredCapabilities caps = new DesiredCapabilities();
caps.setCapability("name", "Java tvOS Example");
caps.setCapability("deviceName", "Apple TV 4K");
caps.setCapability("platformName", "tvOS");
caps.setCapability("platformVersion", "17.4");
caps.setCapability("app", "https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa");
caps.setCapability("realDevice", true);
URL serverUrl = new URL("https://key:secret@hub.testingbot.com/wd/hub");
driver = new AppiumDriver<>(serverUrl, caps);
// Wait for the first input field to be displayed
WebDriverWait wait = new WebDriverWait(driver, Duration.ofSeconds(30));
WebElement inputField1 = wait.until(ExpectedConditions.presenceOfElementLocated(MobileBy.AccessibilityId("inputField1")));
inputField1.click();
inputField1.sendKeys("5");
// Navigate down and select the second input field
driver.executeScript("mobile: pressButton", Map.of("name", "down"));
driver.executeScript("mobile: pressButton", Map.of("name", "select"));
// Send keys to the active element (second input field)
WebElement activeElement = driver.switchTo().activeElement();
activeElement.sendKeys("10");
// Navigate down and select the result field
driver.executeScript("mobile: pressButton", Map.of("name", "down"));
driver.executeScript("mobile: pressButton", Map.of("name", "select"));
WebElement resultField = driver.findElement(MobileBy.AccessibilityId("resultField"));
String resultText = resultField.getText();
if (resultText != null && Integer.parseInt(resultText) == 15) {
System.out.println("Test Passed");
} else {
System.out.println("Test Failed");
}
} catch (Exception e) {
e.printStackTrace();
} finally {
// Quit the driver
if (driver != null) {
driver.quit();
}
}
}
}
```
```csharp
using OpenQA.Selenium;
using OpenQA.Selenium.Appium;
using OpenQA.Selenium.Appium.Enums;
using OpenQA.Selenium.Appium.iOS;
using OpenQA.Selenium.Support.UI;
using System;
using System.Collections.Generic;
class TvOSTest
{
static void Main(string[] args)
{
AppiumDriver driver = null;
try
{
// Set capabilities
var caps = new AppiumOptions();
caps.AddAdditionalCapability(MobileCapabilityType.PlatformName, "tvOS");
caps.AddAdditionalCapability(MobileCapabilityType.DeviceName, "Apple TV 4K");
caps.AddAdditionalCapability(MobileCapabilityType.PlatformVersion, "17.4");
caps.AddAdditionalCapability(MobileCapabilityType.App, "https://github.com/testingbot/tvos-example/releases/download/1.0.0/tvos-example.ipa");
caps.AddAdditionalCapability("realDevice", true);
caps.AddAdditionalCapability("name", "C# tvOS Example");
// Initialize driver
var serverUri = new Uri("https://key:secret@hub.testingbot.com/wd/hub");
driver = new IOSDriver(serverUri, caps);
// Wait for the first input field to be displayed
var wait = new WebDriverWait(driver, TimeSpan.FromSeconds(30));
var inputField1 = wait.Until(d => d.FindElement(MobileBy.AccessibilityId("inputField1")));
// Interact with the first input field
inputField1.Click();
inputField1.SendKeys("5");
// Navigate down and select the second input field
driver.ExecuteScript("mobile: pressButton", new Dictionary
