Robot Framework Automated App Testing

See our Robot Framework app example repository for a complete example on how to run Robot Framework mobile app tests on TestingBot.

With AppiumLibrary you can run Appium tests with Robot Framework against native and hybrid mobile apps on Android and iOS.

Robot Framework is a keyword-driven test automation framework that produces easy to read test cases and HTML reports. Combined with TestingBot's device cloud, you can run your Robot Framework app tests on a large range of real Android and iOS devices, Android emulators and iOS simulators.

Installation

Make sure you have Python and pip installed, then install the required libraries:

pip install robotframework robotframework-appiumlibrary Appium-Python-Client requests

To run a test, use the robot command. The examples below rely on two small Python helper libraries (TestingBot.py and AppOptions.py) — see the Helper Libraries section for their content, or clone the example repository.

PYTHONPATH=$PYTHONPATH:. robot test_android_emulator.robot

Android Examples

Below are Robot Framework examples that download and install our sample Android app (a calculator), fill in two input fields and verify the sum. Switch between running on an Android emulator or on a real Android device.

Save the example below in a test_android_emulator.robot file:

*** Settings ***

Library  AppiumLibrary
Library  TestingBot
Library  AppOptions

Test Setup  Open test app
Test Teardown  Close test app

*** Test Cases ***

Android Sample App Sum Test
    Wait Until Page Contains Element    accessibility_id=inputA    timeout=30s
    Input Text    accessibility_id=inputA    10
    Wait Until Page Contains Element    accessibility_id=inputB    timeout=30s
    Input Text    accessibility_id=inputB    5
    Wait Until Page Contains Element    accessibility_id=sum    timeout=30s
    Element Text Should Be    accessibility_id=sum    15

*** Keywords ***

Open test app
    ${options}=    Get App Options    Android

    # W3C + Appium capabilities for a TestingBot Android emulator
    Call Method    ${options}    set_capability    platformName    Android
    Call Method    ${options}    set_capability    appium:deviceName    Pixel.*
    Call Method    ${options}    set_capability    appium:platformVersion    16.0
    Call Method    ${options}    set_capability    appium:automationName    UiAutomator2
    Call Method    ${options}    set_capability    appium:app    https://testingbot.com/appium/sample.apk

    # TestingBot capabilities
    ${tb_options}=    Create Dictionary    key=api_key    secret=api_secret    name=Android Emulator App Test    build=MyAppBuild
    Call Method    ${options}    set_capability    tb:options    ${tb_options}

    Open Testingbot App    ${options}

Close test app
    Report Testingbot Status    ${SUITE_NAME} | ${TEST_NAME}    ${TEST_STATUS}
    Close Application

PYTHONPATH=$PYTHONPATH:. robot test_android_emulator.robot

Save the example below in a test_android_physical.robot file:

*** Settings ***

Library  AppiumLibrary
Library  TestingBot
Library  AppOptions

Test Setup  Open test app
Test Teardown  Close test app

*** Test Cases ***

Android Sample App Sum Test
    Wait Until Page Contains Element    accessibility_id=inputA    timeout=30s
    Input Text    accessibility_id=inputA    10
    Wait Until Page Contains Element    accessibility_id=inputB    timeout=30s
    Input Text    accessibility_id=inputB    5
    Wait Until Page Contains Element    accessibility_id=sum    timeout=30s
    Element Text Should Be    accessibility_id=sum    15

*** Keywords ***

Open test app
    ${options}=    Get App Options    Android

    # W3C + Appium capabilities for a TestingBot Android real device
    Call Method    ${options}    set_capability    platformName    Android
    Call Method    ${options}    set_capability    appium:deviceName    Galaxy S24
    Call Method    ${options}    set_capability    appium:platformVersion    14.0
    Call Method    ${options}    set_capability    appium:automationName    UiAutomator2
    Call Method    ${options}    set_capability    appium:app    https://testingbot.com/appium/sample.apk

    # TestingBot capabilities — realDevice=True routes the test to a physical device
    ${tb_options}=    Create Dictionary    key=api_key    secret=api_secret    name=Android Physical App Test    build=MyAppBuild    realDevice=True
    Call Method    ${options}    set_capability    tb:options    ${tb_options}

    Open Testingbot App    ${options}

Close test app
    Report Testingbot Status    ${SUITE_NAME} | ${TEST_NAME}    ${TEST_STATUS}
    Close Application

PYTHONPATH=$PYTHONPATH:. robot test_android_physical.robot

iOS Examples

Below are Robot Framework examples for our iOS sample app. Switch between running on an iOS simulator or on a real iPhone/iPad.

Save the example below in a test_ios_simulator.robot file:

*** Settings ***

Library  AppiumLibrary
Library  TestingBot
Library  AppOptions

Test Setup  Open test app
Test Teardown  Close test app

*** Test Cases ***

iOS Sample App Sum Test
    Wait Until Page Contains Element    accessibility_id=inputA    timeout=30s
    Input Text    accessibility_id=inputA    10
    Wait Until Page Contains Element    accessibility_id=inputB    timeout=30s
    Input Text    accessibility_id=inputB    5

*** Keywords ***

Open test app
    ${options}=    Get App Options    iOS

    # W3C + Appium capabilities for a TestingBot iOS simulator
    Call Method    ${options}    set_capability    platformName    iOS
    Call Method    ${options}    set_capability    appium:deviceName    iPhone.*
    Call Method    ${options}    set_capability    appium:platformVersion    26.0
    Call Method    ${options}    set_capability    appium:automationName    XCUITest
    Call Method    ${options}    set_capability    appium:app    https://testingbot.com/appium/sample.zip

    # TestingBot capabilities
    ${tb_options}=    Create Dictionary    key=api_key    secret=api_secret    name=iOS Simulator App Test    build=MyAppBuild
    Call Method    ${options}    set_capability    tb:options    ${tb_options}

    Open Testingbot App    ${options}

Close test app
    Report Testingbot Status    ${SUITE_NAME} | ${TEST_NAME}    ${TEST_STATUS}
    Close Application

PYTHONPATH=$PYTHONPATH:. robot test_ios_simulator.robot

Save the example below in a test_ios_physical.robot file:

*** Settings ***

Library  AppiumLibrary
Library  TestingBot
Library  AppOptions

Test Setup  Open test app
Test Teardown  Close test app

*** Test Cases ***

iOS Sample App Sum Test
    Wait Until Page Contains Element    accessibility_id=inputA    timeout=30s
    Input Text    accessibility_id=inputA    10
    Wait Until Page Contains Element    accessibility_id=inputB    timeout=30s
    Input Text    accessibility_id=inputB    5

*** Keywords ***

Open test app
    ${options}=    Get App Options    iOS

    # W3C + Appium capabilities for a TestingBot iOS real device
    Call Method    ${options}    set_capability    platformName    iOS
    Call Method    ${options}    set_capability    appium:deviceName    iPhone.*
    Call Method    ${options}    set_capability    appium:platformVersion    26.*
    Call Method    ${options}    set_capability    appium:automationName    XCUITest
    Call Method    ${options}    set_capability    appium:app    https://testingbot.com/appium/sample.ipa

    # TestingBot capabilities — realDevice=True routes the test to a physical device
    ${tb_options}=    Create Dictionary    key=api_key    secret=api_secret    name=iOS Physical App Test    build=MyAppBuild    realDevice=True
    Call Method    ${options}    set_capability    tb:options    ${tb_options}

    Open Testingbot App    ${options}

Close test app
    Report Testingbot Status    ${SUITE_NAME} | ${TEST_NAME}    ${TEST_STATUS}
    Close Application

PYTHONPATH=$PYTHONPATH:. robot test_ios_physical.robot

Make sure to always close the session in your test teardown (Close Application), otherwise the session will keep running until TestingBot's idle timeout.

Helper Libraries

The examples above import two small Python helper libraries — AppOptions and TestingBot — that encapsulate driver creation and test status reporting. You can copy the snippets below, or grab them from the example repository.

Save the following in a AppOptions.py file:

from appium.options.android import UiAutomator2Options
from appium.options.ios import XCUITestOptions


def get_app_options(platform):
    if platform.lower() == 'android':
        return UiAutomator2Options()
    elif platform.lower() == 'ios':
        return XCUITestOptions()
    else:
        raise ValueError(f"Unsupported platform: {platform}")

Save the following in a TestingBot.py file:

import os
import requests
from appium import webdriver
from robot.libraries.BuiltIn import BuiltIn

TB_KEY = os.environ.get('TB_KEY', 'api_key')
TB_SECRET = os.environ.get('TB_SECRET', 'api_secret')
HUB_URL = f'https://{TB_KEY}:{TB_SECRET}@hub.testingbot.com/wd/hub'


def open_testingbot_app(options):
    driver = webdriver.Remote(HUB_URL, options=options)
    appium = BuiltIn().get_library_instance('AppiumLibrary')
    appium._cache.register(driver)


def report_testingbot_status(name, status):
    appium = BuiltIn().get_library_instance('AppiumLibrary')
    session_id = appium._current_application().session_id

    payload = {'test[name]': name, 'test[success]': int(status == 'PASS')}
    url = f'https://api.testingbot.com/v1/tests/{session_id}'
    response = requests.put(url, data=payload, auth=(TB_KEY, TB_SECRET))
    assert response.status_code == 200, response.text

Uploading Your App

Please see our Upload Mobile App documentation to find out how to upload your .apk, .ipa or .zip to TestingBot and reference it from the appium:app capability.

Specify Devices

To let TestingBot know which device to run your test on, set the platformName, appium:deviceName and appium:platformVersion capabilities. Pick a combination below to see the matching snippet:

Testing Internal Websites

We've built TestingBot Tunnel, to provide you with a secure way to run tests against your staged/internal apps or backends.
Please see our TestingBot Tunnel documentation for more information.

1. Download our tunnel and start it:

java -jar testingbot-tunnel.jar key secret

2. In your TestingBot.py helper, point HUB_URL at the tunnel instead of hub.testingbot.com. Assuming the tunnel runs on the same machine as your tests, use http://localhost:4445/wd/hub (4445 is the default tunnel port).

HUB_URL = 'http://localhost:4445/wd/hub'

Your test will then route securely through the tunnel to TestingBot and back.

Run tests in Parallel

You can run multiple Robot Framework tests in parallel using the Pabot library.

Install Pabot:

pip install -U robotframework-pabot

Run your tests with pabot instead of robot:

PYTHONPATH=$PYTHONPATH:. pabot --processes 2 test_android_emulator.robot test_ios_simulator.robot

Some recommendations when running tests in parallel:

Make sure each test case is independent and does not rely on shared state.
Set --processes to the number of parallel sessions your TestingBot plan allows.
Consider using --processtimeout to set a timeout for each process.

Queuing

Every plan comes with a limit of parallel tests. If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and start them as soon as slots become available.

Preparing your App

You do not need to install any code or plugin to run a test.
Below are some things that are necessary to successfully run a mobile test:

For Android:

Please supply the URL to your .apk or .aab file.
Important: the .apk file needs to be a x86 build (x86 ABI) for Android emulators.

For iOS Real Device:

Please supply the URL to an .ipa file.

For iOS Simulator:

Please supply the URL to a .zip file that contains your .app
The .app needs to be an iOS Simulator build:
- Create a Simulator build:
```
xcodebuild -sdk iphonesimulator -configuration Debug
```
- Zip the .app bundle:
```
zip -r MyApp.zip MyApp.app
```

Additional Options

Appium provides a lot of options to configure your test.

Some important options that might help:

For Android:

appActivity and appPackage: by default, Appium will try to extract the main Activity from your apk. If this fails, you can supply your own with these options.
chromeOptions: additional chromedriver options you can supply.
otherApps: a JSON array of other apps that need to be installed, in URL format.

For Android & iOS:

locale: the language of the simulator/device (ex: fr_CA)

This sets the locale on the entire device/simulator, except on physical iOS devices. For real iOS devices, we will pass -AppleLocale as argument to the app.
newCommandTimeout: How long (in seconds) Appium will wait for a new command from the client before assuming the client quit and ending the session