Features

Your first C# test with SpecFlow

See our SpecFlow example repository for a simple example on how to run SpecFlow tests in parallel on TestingBot.

SpecFlow is an open-source .NET utility which allows you to write tests using Cucumber-compatible Gherkin syntax.

Installation

You can install SpecFlow via the NuGet Gallery

  • Create a new project in Visual Studio.
  • In the Visual Studio Tools menu, go to Library Package Manager > Manage Nuget Package for Solution.
  • This will open the Manage NuGet Packages dialog. Click Online, then Next.
  • In the Search Packages field, enter SpecFlow and click Search.
  • Select SpecFlow from the search results and click Install.
  • For more details, please visit SpecFlow installation Document.


    Sample Test

    This is a sample test case in SpecFlow:

Feature: Google

Scenario Outline: Can find search results
  Given I am on the google page for <profile> and <environment>
  When I search for "TestingBot"
  Then I should see title "TestingBot - Google Search"
  
  Examples:
    | profile | environment |
    | single    | chrome      |


[Binding]
public class SingleSteps
{
  private IWebDriver _driver;
  readonly TestingBotDriver _tbDriver;

  public SingleSteps()
  {
    _tbDriver = (TestingBotDriver)ScenarioContext.Current["tbDriver"];
  }

  [Given(@"I am on the google page for (.*) and (.*)")]
  public void GivenIAmOnTheGooglePage(string profile, string environment)
  {
    _driver = _tbDriver.Init(profile, environment);
    _driver.Navigate().GoToUrl("https://www.google.com/ncr");
  }

  [When(@"I search for ""(.*)""")]
  public void WhenISearchFor(string keyword)
  {
    var q = _driver.FindElement(By.Name("q"));
    q.SendKeys(keyword);
    q.Submit();
  }

  [Then(@"I should see title ""(.*)""")]
  public void ThenIShouldSeeTitle(string title)
  {
    Thread.Sleep(5000);
    Assert.That(_driver.Title, Is.EqualTo(title));
  }
}

Now you need to create the TestingBotDriver we will be using:


[Binding]
public sealed class TestingBot
{
  private TestingBotDriver tbDriver;
  private string[] tags;

  [BeforeScenario]
  public void BeforeScenario()
  {
    tbDriver = new TestingBotDriver(ScenarioContext.Current);
    ScenarioContext.Current["tbDriver"] = tbDriver;
  }

  [AfterScenario]
  public void AfterScenario()
  {
    tbDriver.Cleanup();
  }
}

public class TestingBotDriver
{
  private IWebDriver driver;
  private string profile;
  private string environment;
  private ScenarioContext context;

  public TestingBotDriver(ScenarioContext context)
  {
    this.context = context;
  }

  public IWebDriver Init(string profile, string environment)
  {
    NameValueCollection caps = ConfigurationManager.GetSection("capabilities/" + profile) as NameValueCollection;
    NameValueCollection settings = ConfigurationManager.GetSection("environments/" + environment) as NameValueCollection;

    DesiredCapabilities capability = new DesiredCapabilities();

    foreach (string key in caps.AllKeys)
    {
      capability.SetCapability(key, caps[key]);
    }

    foreach (string key in settings.AllKeys)
    {
      capability.SetCapability(key, settings[key]);
    }

    String key = Environment.GetEnvironmentVariable("TB_KEY");
    if (key == null)
    {
      key = ConfigurationManager.AppSettings.Get("key");
    }

    String secret = Environment.GetEnvironmentVariable("TB_SECRET");
    if (secret == null)
    {
      secret = ConfigurationManager.AppSettings.Get("secret");
    }

    capability.SetCapability("key", key);
    capability.SetCapability("secret", secret);

    driver = new RemoteWebDriver(new Uri("http://" + ConfigurationManager.AppSettings.Get("server") + "/wd/hub/"), capability);
    return driver;
  }

  public void Cleanup()
  {
    driver.Quit();
  }
}

Now we need to create the config file which the code above uses to determine various settings.


<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <configSections>
    <section name="specFlow" type="TechTalk.SpecFlow.Configuration.ConfigurationSectionHandler, TechTalk.SpecFlow" />

    <sectionGroup name="capabilities">
      <section name="single" type="System.Configuration.AppSettingsSection, System.Configuration, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
    </sectionGroup>

    <sectionGroup name="environments">
      <section name="chrome" type="System.Configuration.AppSettingsSection, System.Configuration, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
    </sectionGroup>
  </configSections>

  <appSettings>
    <add key="key" value="key" />
    <add key="secret" value="secret" />
    <add key="server" value="hub.testingbot.com" />
  </appSettings>

  <capabilities>
    <single>
      <add key="screenrecorder" value="true" />
    </single>
  </capabilities>

  <environments>
    <chrome>
      <add key="browser" value="chrome" />
    </chrome>
  </environments>
  
<startup><supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5.1" /></startup><specFlow>
    <!-- For additional details on SpecFlow configuration options see http://go.specflow.org/doc-config -->
  <!-- For additional details on SpecFlow configuration options see http://go.specflow.org/doc-config --><unitTestProvider name="NUnit" /></specFlow></configuration>

Specify Browsers & Devices

To let TestingBot know on which browser/platform/device you want to run your test on, you need to specify the browsername, version, OS and other optional options in the capabilities field.


<environments>
  <tb>
    <add key="platform" value="WINDOWS" />
    <add key="browserName" value="Chrome" />
    <add key="version" value="latest" />
  </tb>
</environments>

To see how to do this, please select a combination of browser, version and platform in the drop-down menus below.



Testing Internal Websites

We've built TestingBot Tunnel, to provide you with a secure way to run tests against your staged/internal webapps.
Please see our TestingBot Tunnel documentation for more information about this easy to use tunneling solution.


The example below shows how to easily run a SpecFlow test with our Tunnel:


1. Download our tunnel and start the tunnel:

java -jar testingbot-tunnel.jar key secret

2. Adjust your test: instead of pointing to 'hub.testingbot.com/wd/hub' like the example above - change it to point to your tunnel's IP address.
Assuming you run the tunnel on the same machine you run your tests, change to 'localhost:4445/wd/hub'. localhost is the machine running the tunnel, 4445 is the default port of the tunnel.


This way your test will go securely through the tunnel to TestingBot and back:


<appSettings>
    <add key="key" value="key" />
    <add key="secret" value="secret" />
    <add key="server" value="localhost:4445" />
</appSettings>

Run tests in Parallel

Parallel Testing means running the same test, or multiple tests, simultaneously. This greatly reduces your total testing time.

You can run the same tests on all different browser configurations or run different tests all on the same browser configuration.
TestingBot has a large grid of machines and browsers, which means you can use our service to do efficient parallel testing. It is one of the key features we provide to greatly cut down on your total testing time.


To run multiple tests at the same time with SpecFlow, modify the config file like this:


// App.config
<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <configSections>
    <section name="specFlow" type="TechTalk.SpecFlow.Configuration.ConfigurationSectionHandler, TechTalk.SpecFlow" />

    <sectionGroup name="capabilities">
      <section name="parallel" type="System.Configuration.AppSettingsSection, System.Configuration, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
    </sectionGroup>

    <sectionGroup name="environments">
      <section name="chrome" type="System.Configuration.AppSettingsSection, System.Configuration, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
      <section name="firefox" type="System.Configuration.AppSettingsSection, System.Configuration, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
      <section name="safari" type="System.Configuration.AppSettingsSection, System.Configuration, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
      <section name="ie" type="System.Configuration.AppSettingsSection, System.Configuration, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
    </sectionGroup>
  </configSections>

  <appSettings>
    <add key="key" value="key" />
    <add key="secret" value="secret" />
    <add key="server" value="hub.testingbot.com" />
  </appSettings>

  <capabilities>
    <parallel>
      <add key="screenrecorder" value="true" />
    </parallel>
  </capabilities>

  <environments>
    <chrome>
      <add key="browser" value="chrome" />
    </chrome>
    <firefox>
      <add key="browser" value="firefox" />
    </firefox>
    <safari>
      <add key="browser" value="safari" />
    </safari>
    <ie>
      <add key="browser" value="ie" />
    </ie>
  </environments>
  
<startup><supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5.1" /></startup><specFlow>
    <!-- For additional details on SpecFlow configuration options see http://go.specflow.org/doc-config -->
  <!-- For additional details on SpecFlow configuration options see http://go.specflow.org/doc-config --><unitTestProvider name="NUnit" /></specFlow></configuration>

// Add to AssemblyInfo.cs
[assembly: Parallelizable(ParallelScope.Fixtures)]

The test cases you want to run in parallel need to be written like this:


// Google Feature for parallel run
Feature: Google

Scenario Outline: Can find search results
  Given I am on the google page for <profile> and <environment>
  When I search for "TestingBot"
  Then I should see title "TestingBot - Google Search"
  
  Examples:
    | profile | environment |
    | parallel  | chrome      |
    | parallel  | firefox     |
    | parallel  | safari      |
    | parallel  | ie          |

You can now run the tests in parallel by running the tests with fixture parallel from Test Explorer.


Queuing

Every plan we provide comes with a limit of parallel tests.
If you exceed the number of parallel tests assigned to your account, TestingBot will queue the additional tests (for up to 6 minutes) and run the tests as soon as slots become available.

Mark tests as passed/failed

To see if a test passed or not in our member area, or to send additional meta-data to TestingBot, you can use our API.


Please see the example below on how to notify TestingBot about the test success state:

[TearDown]
public void CleanUp()
{
    bool passed = TestContext.CurrentContext.Result.Status == TestStatus.Passed;
    try
    {
        // Logs the result to TestingBot
        ((IJavaScriptExecutor)driver).ExecuteScript("tb:test-result=" + (passed ? "passed" : "failed"));
    }
    finally
    {
        // Terminates the remote webdriver session
        driver.Quit();
    }
}

Other C# Framework examples

  • NUnit

    An unit testing framework that is open source written in C#.

  • PNunit

    With PNUnit you can run several tests in parallel.

  • SpecFlow

    SpecFlow allows you to run Automated .NET tests using Cucumber-compatible Gherkin syntax.

  • MSTest

    MSTest framework is a test framework which is included, by default, with Microsoft Visual Studio.

  • MbUnit

    MbUnit is a generative test unit framework, built for C sharp testing.