Electron Testing

Electron is a framework that allows you to create cross-platform native applications by using Javascript code. Electron will embed a Chromium engine in a native app format, allowing web developers to create desktop applications by writing HTML and Javascript code.

TestingBot supports running automated WebDriver tests against Electron-based apps on these platforms:

Windows 11
Windows 10
macOS Sequoia
macOS Sonoma
macOS Ventura
macOS Monterey

All Electron versions from version 16 to the most recent version are supported.

TestingBot has created a demo Electron app, which acts as a simple calculator, to showcase automated Electron app testing.

Run your first Electron test

App management

To test an Electron app, you'll first need to upload a zip file of your Electron application to a URL where TestingBot can download it from. You can choose to upload this yourself, for example to AWS S3 or use TestingBot Storage to upload your .zip file.

Note that files uploaded to TestingBot Storage are automatically pruned after 62 days.

$ curl -X POST "https://api.testingbot.com/v1/storage" \
-u key:secret -F "file=@/path/to/app/file/electron-app.zip"

require 'testingbot'
api = TestingBot::Api.new(key, secret)
api.upload_local_file('electron-app.zip')

TestingbotREST restApi = new TestingbotREST(key, secret);
TestingbotStorageUploadResponse uploadResponse = restApi.uploadToStorage(File file);

$api = new TestingBot\TestingBotAPI($key, $secret);
$api->uploadLocalFileToStorage($pathToLocalFile);

import testingbotclient
tb = testingbotclient.TestingBotClient(key, secret)
tb.storage.upload_local_file(localFilePath)

const TestingBot = require('testingbot-api');

const api = new TestingBot({
  api_key: "your-tb-key",
  api_secret: "your-tb-secret"
});

api.uploadFile(localFilePath, function(error, response) {
    // response.app_url
});

TestingBot Storage will return a app_url which you can use in the tb:app capability of your test.

This is how the capabilities in your test should currently look like:

caps = {
  browserName: "electron",
  browserVersion: "33",
  "tb:app": "tb://the-app-url-you-received" # or https://s3.amazonaws.com/...
}

ChromeOptions chromeOpts = new ChromeOptions();
chromeOpts.setExperimentalOption("w3c", true);

MutableCapabilities tbOptions = new MutableCapabilities();
tbOptions.setCapability("key", "api_key");
tbOptions.setCapability("secret", "api_secret");

DesiredCapabilities caps = new DesiredCapabilities();
caps.setCapability(ChromeOptions.CAPABILITY, chromeOpts);
caps.setCapability("tb:options", tbOptions);
caps.setCapability("tb:app", "tb://the-app-url-you-received");
caps.setCapability("browserVersion", "33");
caps.setCapability("browserName", "electron");

tbOptions = {
    'name': 'Electron Sample'
}

chromeOpts =  {
    'browserName': "electron",
    'goog:chromeOptions': {'w3c': True},
    'tb:options': tbOptions,
    'tb:app': 'tb://the-app-url-you-received'
}

self.driver = webdriver.Remote(remote_url, desired_capabilities=chromeOpts)

driver = await new webdriver.Builder().withCapabilities({
    "browserName": 'electron',
    "goog:chromeOptions" : { "w3c" : true },
    "tb:options": {
        "key": "api_key",
        "secret": "api_secret"
    },
    'tb:app': 'tb://the-app-url-you-received' // or https://s3.amazonaws.com/...
}).usingServer("https://hub.testingbot.com/wd/hub").build();

var chromeOptions = new ChromeOptions()
{
    BrowserVersion = "33",
    BrowserName = "electron",
    PlatformName = "Windows 10",
    UseSpecCompliantProtocol = true
};
var tbOptions = new Dictionary<string, object>
{
    ["key"] = "api_key",
    ["secret"] = "api_secret",
    ["name"] = TestContext.CurrentContext.Test.Name
};

chromeOptions.AddAdditionalCapability("tb:options", tbOptions, true);
chromeOptions.AddAdditionalCapability("tb:app", "tb://the-app-url-you-received", true);

driver = new RemoteWebDriver(new Uri("https://hub.testingbot.com/wd/hub"),
    chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600));

Binary location

Next to the required app capability, you will also need to specify a tb:binary_location capability. This is the path to the actual Electron app, so that TestingBot knows which file in the zip file is the Electron app.

For example, if your uploaded zip file is structured like this:

TestingBot-Electron-App.zip
[ testingbot-electron-demo-app.app ]
  [ Contents ]
    [ MacOS ]
        -- testingbot-electron-demo-app

The binary_location capability in this case would be:

tb:binary_location = "testingbot-electron-demo-app.app/Contents/MacOS/testingbot-electron-demo-app"

Now the capabilities in your test should look like the example below.

require 'rubygems'
require 'selenium-webdriver'

caps = Selenium::WebDriver::Remote::Capabilities.new
caps["browserName"] = "electron"
caps["platformName"] = "VENTURA"
caps["browserVersion"] = "31"
caps["tb:binary_location"] = "testingbot-electron-demo-app.app/Contents/MacOS/testingbot-electron-demo-app"
caps["tb:app"] = "https://github.com/testingbot/testingbot-electron-demo-app/releases/download/v1.0.0/testingbot-electron-demo-app-darwin-arm64-1.0.0.zip"

http_client = Selenium::WebDriver::Remote::Http::Default.new

http_client.open_timeout = 200   # Connect timeout
http_client.read_timeout = 200   # Read timeout

driver = Selenium::WebDriver.for(
    :remote,
    :url => "https://key:secret@hub.testingbot.com/wd/hub",
    :options => caps,
    :http_client => http_client)

  # Generate two random numbers between 0 and 9
  num1 = rand(0..9)
  num2 = rand(0..9)

  # Find elements on the page
  first_number = driver.find_element(id: "btn-#{num1}")
  second_number = driver.find_element(id: "btn-#{num2}")
  plus = driver.find_element(id: 'btn-plus')
  equal = driver.find_element(id: 'btn-equal')

  # Perform operations
  first_number.click
  plus.click
  second_number.click
  equal.click

  # Get the result
  result_element = driver.find_element(id: 'calc-display')
  result_text = result_element.text

  driver.quit
  raise "Test failed" unless (num1 + num2) == result_text.to_i

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.openqa.selenium.remote.http.ClientConfig;
import org.openqa.selenium.remote.http.HttpClient;
import org.openqa.selenium.remote.http.HttpClient.Factory;

import java.net.URL;
import java.util.Random;

public class SeleniumTest {

    public static void main(String[] args) {
        try {
            DesiredCapabilities caps = new DesiredCapabilities();
            caps.setCapability("browserName", "electron");
            caps.setCapability("platformName", "VENTURA");
            caps.setCapability("browserVersion", "31");
            caps.setCapability("tb:binary_location", "testingbot-electron-demo-app.app/Contents/MacOS/testingbot-electron-demo-app");
            caps.setCapability("tb:app", "https://github.com/testingbot/testingbot-electron-demo-app/releases/download/v1.0.0/testingbot-electron-demo-app-darwin-arm64-1.0.0.zip");

            WebDriver driver = new RemoteWebDriver(
                new URL("https://key:secret@hub.testingbot.com/wd/hub"),
                caps
            );

            // Generate two random numbers between 0 and 9
            Random random = new Random();
            int num1 = random.nextInt(10);
            int num2 = random.nextInt(10);

            WebElement firstNumber = driver.findElement(By.id("btn-" + num1));
            WebElement secondNumber = driver.findElement(By.id("btn-" + num2));
            WebElement plus = driver.findElement(By.id("btn-plus"));
            WebElement equal = driver.findElement(By.id("btn-equal"));

            firstNumber.click();
            plus.click();
            secondNumber.click();
            equal.click();

            WebElement resultElement = driver.findElement(By.id("calc-display"));
            String resultText = resultElement.getText();

            // Verify the result
            if ((num1 + num2) != Integer.parseInt(resultText)) {
                throw new AssertionError("Test failed");
            }

            // Quit the driver
            driver.quit();

        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
import random

# Set capabilities
caps = DesiredCapabilities.CHROME.copy()
caps["browserName"] = "electron"
caps["platformName"] = "VENTURA"
caps["browserVersion"] = "31"
caps["tb:binary_location"] = "testingbot-electron-demo-app.app/Contents/MacOS/testingbot-electron-demo-app"
caps["tb:app"] = "https://github.com/testingbot/testingbot-electron-demo-app/releases/download/v1.0.0/testingbot-electron-demo-app-darwin-arm64-1.0.0.zip"

# Configure HTTP client timeouts
http_client = webdriver.remote.webdriver.WebDriver.DEFAULT_HTTP_CLIENT
http_client.set_timeout(200)

# Initialize the WebDriver connection to TestingBot
driver = webdriver.Remote(
    command_executor="https://key:secret@hub.testingbot.com/wd/hub",
    desired_capabilities=caps,
    options=None,  # Use 'options=None' to pass capabilities directly
    http_client=http_client
)

try:
    # Generate two random numbers between 0 and 9
    num1 = random.randint(0, 9)
    num2 = random.randint(0, 9)

    # Find elements on the page
    first_number = driver.find_element_by_id(f"btn-{num1}")
    second_number = driver.find_element_by_id(f"btn-{num2}")
    plus = driver.find_element_by_id('btn-plus')
    equal = driver.find_element_by_id('btn-equal')

    # Perform operations
    first_number.click()
    plus.click()
    second_number.click()
    equal.click()

    # Get the result
    result_element = driver.find_element_by_id('calc-display')
    result_text = result_element.text

    # Verify the result
    assert (num1 + num2) == int(result_text), "Test failed"

finally:
    driver.quit()

const { Builder, By } = require('selenium-webdriver');
const { Options } = require('selenium-webdriver/chrome');

(async function example() {
    let driver;

    try {
        // Set capabilities
        let caps = {
            browserName: 'electron',
            platformName: 'VENTURA',
            browserVersion: '31',
            'tb:binary_location': 'testingbot-electron-demo-app.app/Contents/MacOS/testingbot-electron-demo-app',
            'tb:app': 'https://github.com/testingbot/testingbot-electron-demo-app/releases/download/v1.0.0/testingbot-electron-demo-app-darwin-arm64-1.0.0.zip'
        };

        driver = await new Builder()
            .usingServer('https://key:secret@hub.testingbot.com/wd/hub')
            .withCapabilities(caps)
            .build();

        // Generate two random numbers between 0 and 9
        const num1 = Math.floor(Math.random() * 10);
        const num2 = Math.floor(Math.random() * 10);

        const firstNumber = await driver.findElement(By.id(`btn-${num1}`));
        const secondNumber = await driver.findElement(By.id(`btn-${num2}`));
        const plus = await driver.findElement(By.id('btn-plus'));
        const equal = await driver.findElement(By.id('btn-equal'));

        // Perform operations
        await firstNumber.click();
        await plus.click();
        await secondNumber.click();
        await equal.click();

        // Get the result
        const resultElement = await driver.findElement(By.id('calc-display'));
        const resultText = await resultElement.getText();

        // Verify the result
        if ((num1 + num2) !== parseInt(resultText)) {
            throw new Error('Test failed');
        }

    } catch (error) {
        console.error('Error during test execution:', error);
    } finally {
        if (driver) {
            await driver.quit();
        }
    }
})();

using OpenQA.Selenium;
using OpenQA.Selenium.Remote;
using System;

class SeleniumTest
{
    static void Main(string[] args)
    {
        try
        {
            // Set capabilities
            var caps = new DesiredCapabilities();
            caps.SetCapability("browserName", "electron");
            caps.SetCapability("platformName", "VENTURA");
            caps.SetCapability("browserVersion", "31");
            caps.SetCapability("tb:binary_location", "testingbot-electron-demo-app.app/Contents/MacOS/testingbot-electron-demo-app");
            caps.SetCapability("tb:app", "https://testingbot.com/electron/mac-demo.zip");

            var commandTimeout = TimeSpan.FromSeconds(200);

            IWebDriver driver = new RemoteWebDriver(
                new Uri("https://key:secret@hub.testingbot.com/wd/hub"),
                caps,
                commandTimeout
            );

            // Generate two random numbers between 0 and 9
            Random random = new Random();
            int num1 = random.Next(0, 10);
            int num2 = random.Next(0, 10);

            IWebElement firstNumber = driver.FindElement(By.Id($"btn-{num1}"));
            IWebElement secondNumber = driver.FindElement(By.Id($"btn-{num2}"));
            IWebElement plus = driver.FindElement(By.Id("btn-plus"));
            IWebElement equal = driver.FindElement(By.Id("btn-equal"));

            // Perform operations
            firstNumber.Click();
            plus.Click();
            secondNumber.Click();
            equal.Click();

            // Get the result
            IWebElement resultElement = driver.FindElement(By.Id("calc-display"));
            string resultText = resultElement.Text;

            // Verify the result
            if ((num1 + num2) != int.Parse(resultText))
            {
                throw new Exception("Test failed");
            }

            // Quit the driver
            driver.Quit();
        }
        catch (Exception ex)
        {
            Console.WriteLine($"Error during test execution: {ex.Message}");
        }
    }
}

The example above will run the TestingBot Demo Electron app on a remote macOS machine. You can also choose to run the demo app on a remote Windows machine inside TestingBot's grid, please see the capabilities below.

caps = Selenium::WebDriver::Remote::Capabilities.new
caps["browserName"] = "electron"
caps["platformName"] = "WIN10"
caps["browserVersion"] = "31"
caps["tb:binary_location"] = "testingbot-electron-demo-app.exe"
caps["tb:app"] = "https://github.com/testingbot/testingbot-electron-demo-app/releases/download/v1.0.0/testingbot-electron-demo-app-win32-x64-1.0.0.zip"

Configuring Electron tests

To configure your Electron test with TestingBot, you need to supply 3 capabilities:

app capability
binary_location capability
Electron version capability

Please use the version or browserVersion capability to indicate to TestingBot which version of Electron was used to create the final app that you are testing.

TestingBot needs to know which version was used, so that we can map it to the Chromium version that is being used for that particular Electron version. For example, if you created your Electron app with Electron 31, you would need to provide browserVersion: 31 as a capability.

Electron Test Results

Test results for each Electron test will be available in the TestingBot member dashboard. For each Electron test, you will get access to a video recording, generated log files and additional meta-data.

These results are also available through the TestingBot REST-API or by using one of the TestingBot CI/CD plugins: Jenkins, Bamboo.

Cross Browser Testing

Live Testing

Visual Testing

Smart-TV Testing

Automated App Testing

Live App Testing

Codeless Automation

Headless Testing