Features

Electron Testing

Electron is a framework that allows you to create cross-platform native applications by using Javascript code. Electron will embed a Chromium engine in a native app format, allowing web developers to create desktop applications by writing HTML and Javascript code.

TestingBot supports running automated WebDriver tests against Electron-based apps on these platforms:

  • Windows 11
  • Windows 10
  • macOS Sonoma
  • macOS Ventura
  • macOS Monterey

All Electron versions from version 16 to the most recent version are supported.

TestingBot has created a demo Electron app, which acts as a simple calculator, to showcase automated Electron app testing.

Run your first Electron test

App management

To test an Electron app, you'll first need to upload a zip file of your Electron application to a URL where TestingBot can download it from. You can choose to upload this yourself, for example to AWS S3 or use TestingBot Storage to upload your .zip file.

Note that files uploaded to TestingBot Storage are automatically pruned after 62 days.

Copy code
$ curl -X POST "https://api.testingbot.com/v1/storage" \
-u key:secret -F "file=@/path/to/app/file/electron-app.zip"
Copy code
require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.upload_local_file('electron-app.zip')
Copy code
TestingbotREST restApi = new TestingbotREST(key, secret);
TestingbotStorageUploadResponse uploadResponse = restApi.uploadToStorage(File file);
Copy code
$api = new TestingBot\TestingBotAPI($key, $secret);
$api->uploadLocalFileToStorage($pathToLocalFile);
Copy code
import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.storage.upload_local_file(localFilePath)
Copy code
const TestingBot = require('testingbot-api');

const api = new TestingBot({
  api_key: "your-tb-key",
  api_secret: "your-tb-secret"
});

api.uploadFile(localFilePath, function(error, response) {
    // response.app_url
});

TestingBot Storage will return a app_url which you can use in the tb:app capability of your test.

This is how the capabilities in your test should currently look like:

Copy code
caps = {
  browserName: "electron",
  browserVersion: "33",
  "tb:app": "tb://the-app-url-you-received" # or https://s3.amazonaws.com/...
}
Copy code
ChromeOptions chromeOpts = new ChromeOptions();
chromeOpts.setExperimentalOption("w3c", true);

MutableCapabilities tbOptions = new MutableCapabilities();
tbOptions.setCapability("key", "api_key");
tbOptions.setCapability("secret", "api_secret");

DesiredCapabilities caps = new DesiredCapabilities();
caps.setCapability(ChromeOptions.CAPABILITY, chromeOpts);
caps.setCapability("tb:options", tbOptions);
caps.setCapability("tb:app", "tb://the-app-url-you-received");
caps.setCapability("browserVersion", "33");
caps.setCapability("browserName", "electron");
Copy code
tbOptions = {
    'name': 'Electron Sample'
}

chromeOpts =  {
    'browserName': "electron",
    'goog:chromeOptions': {'w3c': True},
    'tb:options': tbOptions,
    'tb:app': 'tb://the-app-url-you-received'
}

self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts)
Copy code
driver = await new webdriver.Builder().withCapabilities({
    "browserName": 'electron',
    "goog:chromeOptions" : { "w3c" : true },
    "tb:options": {
        "key": "api_key",
        "secret": "api_secret"
    },
    'tb:app': 'tb://the-app-url-you-received' // or https://s3.amazonaws.com/...
}).usingServer("https://hub.testingbot.com/wd/hub").build();
Copy code
var chromeOptions = new ChromeOptions()
{
    BrowserVersion = "33",
    BrowserName = "electron",
    PlatformName = "Windows 10",
    UseSpecCompliantProtocol = true
};
var tbOptions = new Dictionary<string, object>
{
    ["key"] = "api_key",
    ["secret"] = "api_secret",
    ["name"] = TestContext.CurrentContext.Test.Name
};

chromeOptions.AddAdditionalCapability("tb:options", tbOptions, true);
chromeOptions.AddAdditionalCapability("tb:app", "tb://the-app-url-you-received", true);

driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"),
    chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600));

Binary location

Next to the required app capability, you will also need to specify a tb:binary_location capability. This is the path to the actual Electron app, so that TestingBot knows which file in the zip file is the Electron app.

For example, if your uploaded zip file is structured like this:

Copy code
TestingBot-Electron-App.zip
[ testingbot-electron-demo-app.app ]
  [ Contents ]
    [ MacOS ]
        -- testingbot-electron-demo-app

The binary_location capability in this case would be:

Copy code
tb:binary_location = "testingbot-electron-demo-app.app/Contents/MacOS/testingbot-electron-demo-app"

Now the capabilities in your test should look like the example below.

Copy code
require 'rubygems'
require 'selenium-webdriver'

caps = Selenium::WebDriver::Remote::Capabilities.new
caps["browserName"] = "electron"
caps["platformName"] = "VENTURA"
caps["browserVersion"] = "31"
caps["tb:binary_location"] = "testingbot-electron-demo-app.app/Contents/MacOS/testingbot-electron-demo-app"
caps["tb:app"] = "https://github.com/testingbot/testingbot-electron-demo-app/releases/download/v1.0.0/testingbot-electron-demo-app-darwin-arm64-1.0.0.zip"

http_client = Selenium::WebDriver::Remote::Http::Default.new

http_client.open_timeout = 200   # Connect timeout
http_client.read_timeout = 200   # Read timeout

driver = Selenium::WebDriver.for(
    :remote,
    :url => "https://key:secret@hub.testingbot.com/wd/hub",
    :options => caps,
    :http_client => http_client)

  # Generate two random numbers between 0 and 9
  num1 = rand(0..9)
  num2 = rand(0..9)

  # Find elements on the page
  first_number = driver.find_element(id: "btn-#{num1}")
  second_number = driver.find_element(id: "btn-#{num2}")
  plus = driver.find_element(id: 'btn-plus')
  equal = driver.find_element(id: 'btn-equal')

  # Perform operations
  first_number.click
  plus.click
  second_number.click
  equal.click

  # Get the result
  result_element = driver.find_element(id: 'calc-display')
  result_text = result_element.text

  driver.quit
  raise "Test failed" unless (num1 + num2) == result_text.to_i
Copy code
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.openqa.selenium.remote.http.ClientConfig;
import org.openqa.selenium.remote.http.HttpClient;
import org.openqa.selenium.remote.http.HttpClient.Factory;

import java.net.URL;
import java.util.Random;

public class SeleniumTest {

    public static void main(String[] args) {
        try {
            DesiredCapabilities caps = new DesiredCapabilities();
            caps.setCapability("browserName", "electron");
            caps.setCapability("platformName", "VENTURA");
            caps.setCapability("browserVersion", "31");
            caps.setCapability("tb:binary_location", "testingbot-electron-demo-app.app/Contents/MacOS/testingbot-electron-demo-app");
            caps.setCapability("tb:app", "https://github.com/testingbot/testingbot-electron-demo-app/releases/download/v1.0.0/testingbot-electron-demo-app-darwin-arm64-1.0.0.zip");

            WebDriver driver = new RemoteWebDriver(
                new URL("https://key:secret@hub.testingbot.com/wd/hub"),
                caps
            );

            // Generate two random numbers between 0 and 9
            Random random = new Random();
            int num1 = random.nextInt(10);
            int num2 = random.nextInt(10);

            WebElement firstNumber = driver.findElement(By.id("btn-" + num1));
            WebElement secondNumber = driver.findElement(By.id("btn-" + num2));
            WebElement plus = driver.findElement(By.id("btn-plus"));
            WebElement equal = driver.findElement(By.id("btn-equal"));

            firstNumber.click();
            plus.click();
            secondNumber.click();
            equal.click();

            WebElement resultElement = driver.findElement(By.id("calc-display"));
            String resultText = resultElement.getText();

            // Verify the result
            if ((num1 + num2) != Integer.parseInt(resultText)) {
                throw new AssertionError("Test failed");
            }

            // Quit the driver
            driver.quit();

        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}
Copy code
from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
import random

# Set capabilities
caps = DesiredCapabilities.CHROME.copy()
caps["browserName"] = "electron"
caps["platformName"] = "VENTURA"
caps["browserVersion"] = "31"
caps["tb:binary_location"] = "testingbot-electron-demo-app.app/Contents/MacOS/testingbot-electron-demo-app"
caps["tb:app"] = "https://github.com/testingbot/testingbot-electron-demo-app/releases/download/v1.0.0/testingbot-electron-demo-app-darwin-arm64-1.0.0.zip"

# Configure HTTP client timeouts
http_client = webdriver.remote.webdriver.WebDriver.DEFAULT_HTTP_CLIENT
http_client.set_timeout(200)

# Initialize the WebDriver connection to TestingBot
driver = webdriver.Remote(
    command_executor="https://key:secret@hub.testingbot.com/wd/hub",
    desired_capabilities=caps,
    options=None,  # Use 'options=None' to pass capabilities directly
    http_client=http_client
)

try:
    # Generate two random numbers between 0 and 9
    num1 = random.randint(0, 9)
    num2 = random.randint(0, 9)

    # Find elements on the page
    first_number = driver.find_element_by_id(f"btn-{num1}")
    second_number = driver.find_element_by_id(f"btn-{num2}")
    plus = driver.find_element_by_id('btn-plus')
    equal = driver.find_element_by_id('btn-equal')

    # Perform operations
    first_number.click()
    plus.click()
    second_number.click()
    equal.click()

    # Get the result
    result_element = driver.find_element_by_id('calc-display')
    result_text = result_element.text

    # Verify the result
    assert (num1 + num2) == int(result_text), "Test failed"

finally:
    driver.quit()
Copy code
const { Builder, By } = require('selenium-webdriver');
const { Options } = require('selenium-webdriver/chrome');

(async function example() {
    let driver;

    try {
        // Set capabilities
        let caps = {
            browserName: 'electron',
            platformName: 'VENTURA',
            browserVersion: '31',
            'tb:binary_location': 'testingbot-electron-demo-app.app/Contents/MacOS/testingbot-electron-demo-app',
            'tb:app': 'https://github.com/testingbot/testingbot-electron-demo-app/releases/download/v1.0.0/testingbot-electron-demo-app-darwin-arm64-1.0.0.zip'
        };

        driver = await new Builder()
            .usingServer('https://key:secret@hub.testingbot.com/wd/hub')
            .withCapabilities(caps)
            .build();

        // Generate two random numbers between 0 and 9
        const num1 = Math.floor(Math.random() * 10);
        const num2 = Math.floor(Math.random() * 10);

        const firstNumber = await driver.findElement(By.id(`btn-${num1}`));
        const secondNumber = await driver.findElement(By.id(`btn-${num2}`));
        const plus = await driver.findElement(By.id('btn-plus'));
        const equal = await driver.findElement(By.id('btn-equal'));

        // Perform operations
        await firstNumber.click();
        await plus.click();
        await secondNumber.click();
        await equal.click();

        // Get the result
        const resultElement = await driver.findElement(By.id('calc-display'));
        const resultText = await resultElement.getText();

        // Verify the result
        if ((num1 + num2) !== parseInt(resultText)) {
            throw new Error('Test failed');
        }

    } catch (error) {
        console.error('Error during test execution:', error);
    } finally {
        if (driver) {
            await driver.quit();
        }
    }
})();
Copy code
using OpenQA.Selenium;
using OpenQA.Selenium.Remote;
using System;

class SeleniumTest
{
    static void Main(string[] args)
    {
        try
        {
            // Set capabilities
            var caps = new DesiredCapabilities();
            caps.SetCapability("browserName", "electron");
            caps.SetCapability("platformName", "VENTURA");
            caps.SetCapability("browserVersion", "31");
            caps.SetCapability("tb:binary_location", "testingbot-electron-demo-app.app/Contents/MacOS/testingbot-electron-demo-app");
            caps.SetCapability("tb:app", "https://testingbot.com/electron/mac-demo.zip");

            var commandTimeout = TimeSpan.FromSeconds(200);

            IWebDriver driver = new RemoteWebDriver(
                new Uri("https://key:secret@hub.testingbot.com/wd/hub"),
                caps,
                commandTimeout
            );

            // Generate two random numbers between 0 and 9
            Random random = new Random();
            int num1 = random.Next(0, 10);
            int num2 = random.Next(0, 10);

            IWebElement firstNumber = driver.FindElement(By.Id($"btn-{num1}"));
            IWebElement secondNumber = driver.FindElement(By.Id($"btn-{num2}"));
            IWebElement plus = driver.FindElement(By.Id("btn-plus"));
            IWebElement equal = driver.FindElement(By.Id("btn-equal"));

            // Perform operations
            firstNumber.Click();
            plus.Click();
            secondNumber.Click();
            equal.Click();

            // Get the result
            IWebElement resultElement = driver.FindElement(By.Id("calc-display"));
            string resultText = resultElement.Text;

            // Verify the result
            if ((num1 + num2) != int.Parse(resultText))
            {
                throw new Exception("Test failed");
            }

            // Quit the driver
            driver.Quit();
        }
        catch (Exception ex)
        {
            Console.WriteLine($"Error during test execution: {ex.Message}");
        }
    }
}

The example above will run the TestingBot Demo Electron app on a remote macOS machine. You can also choose to run the demo app on a remote Windows machine inside TestingBot's grid, please see the capabilities below.

Copy code
caps = Selenium::WebDriver::Remote::Capabilities.new
caps["browserName"] = "electron"
caps["platformName"] = "WIN10"
caps["browserVersion"] = "31"
caps["tb:binary_location"] = "testingbot-electron-demo-app.exe"
caps["tb:app"] = "https://github.com/testingbot/testingbot-electron-demo-app/releases/download/v1.0.0/testingbot-electron-demo-app-win32-x64-1.0.0.zip"

Configuring Electron tests

To configure your Electron test with TestingBot, you need to supply 3 capabilities:

Please use the version or browserVersion capability to indicate to TestingBot which version of Electron was used to create the final app that you are testing.

TestingBot needs to know which version was used, so that we can map it to the Chromium version that is being used for that particular Electron version. For example, if you created your Electron app with Electron 31, you would need to provide browserVersion: 31 as a capability.

Electron Test Results

Test results for each Electron test will be available in the TestingBot member dashboard. For each Electron test, you will get access to a video recording, generated log files and additional meta-data.

These results are also available through the TestingBot REST-API or by using one of the TestingBot CI/CD plugins: Jenkins, Bamboo.