Azure Devops (VSTS)

Installing Azure Pipelines extension

1. Make sure you are logged in to Azure Devops.
   Then please go to the Visual Studio Marketplace and download the TestingBot extension.

   

2. Click on Get it free to install the extension.

3. Using the Select an Azure DevOps organization drop-down menu, choose the organization and click install.

   

Configure the pipeline

Now that the TestingBot extension has been installed, you'll need to configure the Service Connection.

• Go to Project Settings and select Service connections.

  

• Click New Service Connection.
  You can now add the TestingBot credentials.

  • Your TestingBot API Key as Username: see the TestingBot member dashboard

  • Your TestingBot API Secret as Password: see the TestingBot member dashboard

  

  You can leave the Server URL to the default value: https://api.testingbot.com/v1/

• Add a Service connection name and click Save.

Now that we've set up the Service Connection, we can go ahead and add the custom TestingBot task to the azure-pipelines.yml file.
To do this, you can edit the file in your favorite editor, or through the YAML editor on Azure Devops.

TestingBot Task

In the steps block, you can now add the TBMain task, either via the Task menu in the Azure Devops page or by editing the file:

- task: TBMain@0
  inputs:
    connectedServiceName: '{your service-connection-name}'

Viewing Test Results

The extension allows you to view test results from Azure.
To enable this, make sure you followed the previous steps and modify your existing test or test suite to send an extra capability to TestingBot: build.

The extension makes the following environment variables available for your test cases:

• TB_KEY
• TB_SECRET
• TB_BUILD_NAME

The TB_BUILD_NAME environment variable needs to be passed as a build capability, for example:

DesiredCapabilities capabilities = new DesiredCapabilities();

capabilities.setCapability(CapabilityType.BROWSER_NAME, browser);
capabilities.setCapability(CapabilityType.VERSION, version);
capabilities.setCapability(CapabilityType.PLATFORM, os);
capabilities.setCapability("name", methodName);
// this capability below will make sure the reporting works
capabilities.setCapability("build", System.getenv("TB_BUILD_NAME"));

webDriver.set(new RemoteWebDriver(
  new URL("https://" + System.getenv("TB_KEY") + ":" + System.getenv("TB_SECRET") + "@hub.testingbot.com/wd/hub"),
capabilities));

It is important to pass the TB_BUILD_NAME environment variable as a build capability for the reporting to work.

Once the build is complete, you will see a TestingBot tab. This will show an overview of the tests for this build.

Enable TestingBot Tunnel

TestingBot Tunnel allows you to run tests against web applications hosted on private, staging or development machines.
To enable the TestingBot Tunnel in Azure, make sure the TestingBot Tunnel checkbox is checked.

To use the Tunnel with your test, make sure to edit your test to use the Tunnel IP (localhost:4445):

DesiredCapabilities capabilities = new DesiredCapabilities();

capabilities.setCapability(CapabilityType.BROWSER_NAME, browser);
capabilities.setCapability(CapabilityType.VERSION, version);
capabilities.setCapability(CapabilityType.PLATFORM, os);
capabilities.setCapability("name", methodName);
// this capability below will make sure the reporting works
capabilities.setCapability("build", System.getenv("TB_BUILD_NAME"));

webDriver.set(new RemoteWebDriver(
  new URL("https://" + System.getenv("TB_KEY") + ":" + System.getenv("TB_SECRET") + "@localhost:4445/wd/hub"),
capabilities));

Stop Tunnel

It's important to close the tunnel at the end of your test. To do this, please add the Stop TestingBot Tunnel task in your azure-pipelines.yml file.

Make sure to add the TBStopTunnel@0 after your other tasks.