Features
Start a free trial

TestNG Automated App Testing

See our TestNG example repository for a simple example on how to run TestNG mobile app tests.

Installation

First, make sure you install the necessary dependencies to run a test. 

<!-- Install using maven -->
<dependency>
    <groupId>org.testng</groupId>
    <artifactId>testng</artifactId>
    <version>7.1.0</version>
</dependency>

TestNG Example

Please see the example below. We're going to test our sample app on a Real Device.
In this test we'll add values to two input fields and see if the calculator works. 

public class SingleTest extends TestingBotTestNGTest {
  @Test
  public void test() throws Exception {
    AndroidElement inputA = (AndroidElement) new WebDriverWait(driver, 30).until(
	    ExpectedConditions.elementToBeClickable(MobileBy.AccessibilityId("inputA")));
	inputA.sendKeys("10");
	AndroidElement inputB = (AndroidElement) new WebDriverWait(driver, 30).until(
	    ExpectedConditions.elementToBeClickable(MobileBy.AccessibilityId("inputB")));
	inputB.sendKeys("5");
	Thread.sleep(5000);

	AndroidElement sum = (AndroidElement) new WebDriverWait(driver, 30).until(
	    ExpectedConditions.elementToBeClickable(MobileBy.AccessibilityId("sum")));
	Assert.assertEquals(sum.getText(), "15");
  }
}
public class SingleTest extends TestingBotTestNGTest {
  @Test
  public void test() throws Exception {
    IOSElement inputA = (IOSElement) new WebDriverWait(driver, 30).until(
	    ExpectedConditions.elementToBeClickable(MobileBy.AccessibilityId("inputA")));
	inputA.sendKeys("10");
	IOSElement inputB = (IOSElement) new WebDriverWait(driver, 30).until(
	    ExpectedConditions.elementToBeClickable(MobileBy.AccessibilityId("inputB")));
	inputB.sendKeys("5");
	Thread.sleep(5000);

	IOSElement sum = (IOSElement) new WebDriverWait(driver, 30).until(
	    ExpectedConditions.elementToBeClickable(MobileBy.AccessibilityId("sum")));
	Assert.assertEquals(sum.getText(), "15");
  }
}

TestingBotTestNGTest

Please create a class called TestingBotTestNGTest: 

public class TestingBotTestNGTest {
  public WebDriver driver;
 
  @BeforeMethod(alwaysRun=true)
  @org.testng.annotations.Parameters(value={"config", "environment"})
  public void setUp(String config_file, String environment) throws Exception {
      JSONParser parser = new JSONParser();
      JSONObject config = (JSONObject) parser.parse(new FileReader("src/test/resources/conf/" + config_file));
      JSONObject envs = (JSONObject) config.get("environments");
 
      DesiredCapabilities capabilities = new DesiredCapabilities();
 
      MapMap<String, String> envCapabilities = (MapMap<String, String>) envs.get(environment);
      Iterator it = envCapabilities.entrySet().iterator();
      while (it.hasNext()) {
        Map.Entry pair = (Map.Entry)it.next();
        capabilities.setCapability(pair.getKey().toString(), pair.getValue());
      }
        
      Map<String, String> commonCapabilities = (MapMap<String, String>) config.get("capabilities");
      it = commonCapabilities.entrySet().iterator();
      while (it.hasNext()) {
        Map.Entry pair = (Map.Entry)it.next();
        if(capabilities.getCapability(pair.getKey().toString()) == null){
            capabilities.setCapability(pair.getKey().toString(), pair.getValue());
        }
      }
 
      String key = System.getenv("TESTINGBOT_KEY");
      if (key == null) {
        key = (String) config.get("key");
      }
 
      String secret = System.getenv("TESTINGBOT_SECRET");
      if(secret == null) {
        secret = (String) config.get("secret");
      }

      String app = System.getenv("TESTINGBOT_APP_ID");
      if (app != null && !app.isEmpty()) {
        capabilities.setCapability("appium:app", app);
      }
 
      driver = new AndroidDriver(new URL("https://"+key+":"+secret+"@hub.testingbot.com/wd/hub"), capabilities);
  }
 
  @AfterMethod(alwaysRun=true)
  public void tearDown() throws Exception {
    driver.quit();
  }
}

Uploading Your App

Please see our Upload Mobile App documentation to find out how to upload your app to TestingBot for testing.

Specify Browsers & Devices

Please see our example Github project on how to configure your desired capabilities with our example TestingBotJUnitTest.java. 

{
  "server": "hub.testingbot.com",
  "key": "TESTINGBOT_KEY",
  "secret": "TESTINGBOT_SECRET",

  "capabilities": {
    "tb:options": {
        "build": "junit-testingbot",
        "name": "single_test"
    }
  },

  "environments": [{
    "appium:app": "https://testingbot.com/appium/sample.apk",
    "appium:platformName": "Android",
    "appium:version": "10.0",
    "appium:deviceName": "Galaxy S10"
  }]
}

To let TestingBot know on which device you want to run your test on, you need to specify the deviceName, version, platformName and other optional options in the capabilities field. 

{
    "appium:app": "https://testingbot.com/appium/sample.apk",
    "appium:platformName": "Android",
    "appium:version": "10.0",
    "appium:deviceName": "Galaxy S10"
}

To see how to do this, please select a combination of device type and device name in the drop-down menus below.

1. Select a Device Type
2. Select a Device Name

Testing Internal Websites

We've built TestingBot Tunnel, to provide you with a secure way to run tests against your staged/internal webapps.
Please see our TestingBot Tunnel documentation for more information about this easy to use tunneling solution.

The example below shows how to easily run a TestNG test with our Tunnel:

1. Download our tunnel and start the tunnel: 

java -jar testingbot-tunnel.jar key secret

2. Adjust your test: instead of pointing to 'hub.testingbot.com/wd/hub' like the example above - change it to point to your tunnel's IP address.
Assuming you run the tunnel on the same machine you run your tests, change to 'localhost:4445/wd/hub'. localhost is the machine running the tunnel, 4445 is the default port of the tunnel.

This way your test will go securely through the tunnel to TestingBot and back:

{
  "server": "localhost:4445",
  "key": "TESTINGBOT_KEY",
  "secret": "TESTINGBOT_SECRET",

  "capabilities": {
    "tb:options": {
        "build": "junit-testingbot",
        "name": "single_test"
    }
  },

  "environments": [{
    "appium:app": "https://testingbot.com/appium/sample.apk",
    "appium:platformName": "Android",
    "appium:version": "10.0",
    "appium:deviceName": "Galaxy S10"
  }]
}

Mark tests as passed/failed

To see if a test passed or not in our member area, or to send additional meta-data to TestingBot, you need to use our API.

TestingBot has a Java client for using the TestingBot API.

Once included with your tests, you can send back test status and other meta-data to TestingBot: 

import com.testingbot.testingbotrest.TestingbotREST;

@After
public void tearDown() throws Exception {
	TestingbotREST r = new TestingbotREST("key", "secret");
	Map<String, String> data = new HashMap<String, String>();
	data.put("success", "1");
	data.put("name", "My Test");
	r.updateTest(driver.getSessionId(), data);
	driver.quit();
}

Preparing your App

You do not need to install any code or plugin to run a test.
Below are some things that are necessary to successfully run a mobile test:

For Android:
  • Please supply the URL to your .apk or .aab file.
    Important: the .apk file needs to be a x86 build (x86 ABI) for Android emulators.
For iOS Real Device:
  • Please supply the URL to an .ipa file.
For iOS Simulator:
  • Please supply the URL to a .zip file that contains your .app
  • The .app needs to be an iOS Simulator build:
    • Create a Simulator build:
      xcodebuild -sdk iphonesimulator -configuration Debug
    • Zip the .app bundle:
      zip -r MyApp.zip MyApp.app

Additional Options

Appium provides a lot of options to configure your test.

Some important options that might help:

For Android:
  • appActivity and appPackage: by default, Appium will try to extract the main Activity from your apk. If this fails, you can supply your own with these options.
  • chromeOptions: additional chromedriver options you can supply.
  • otherApps: a JSON array of other apps that need to be installed, in URL format.
For Android & iOS:
  • locale: the language of the simulator/device (ex: fr_CA)
  • newCommandTimeout: How long (in seconds) Appium will wait for a new command from the client before assuming the client quit and ending the session