< Back to Blog Overview

Continuous Deployment Overview

Testing with a Staging Environment - Continuous Deployment
  • Share on Facebook
  • Share on Twitter
  • Share on LinkedIn
  • Share on HackerNews

In this article, we'd like to talk about how we at TestingBot make sure that new code that goes into production is tested thoroughly.

As a cloud testing company, we deal with testing on a daily basis. For us to provide a good service, it's essential that we use the service ourselves.

For this reason, everything at TestingBot is being tested by TestingBot itself (*inception*).

For our website we've got a batch of Selenium tests which test various functionality on several browsers and platforms.
Here's what our tests are verifying:

  • Does the page render correctly on each browser?
  • Do the buttons work, do they trigger the correct action?
  • Does the pricing page work, does upgrading your subscription function correctly?
  • Can we create a new account, can we log in?
  • Does the member dashboard work, can we see and edit tests
  • and a lot more test cases...

We've configured our test runner to be able to run on several environments via a configuration setting:

  • local: tests run on our own computer via TestingBot Tunnel
  • staging: tests run on our own staging environment
  • production: tests run on the live environment (TestingBot.com)

For each environment, we use a separate database, to make sure we do not mix data between environments.

Running tests against code that's about to go into production.

Repository Setup

We make sure that our production webserver syncs code from our master branch.
We've also toggled a setting in our repository that we can only push new code in our development branch, not in our master branch.
The only one with a access to push to master is our Jenkins CI.

We've also added a GIT hook to our repository that will trigger a Jenkins CI Job as soon as we push new code into the repository.
This Jenkins CI job does the following:

  • Sync the development branch on our staging server, so that the staging webserver is serving the latest code, including the code we just pushed.
  • Run a lint checker on the new code, to make sure we don't introduce syntax errors in our code.
    The reason we do this first is because it's very fast to do this, and because it doesn't make sense to first run longer tests when the code has been broken anyway.

  • Run all tests against this staging webserver. Since we use a separate database, the production data is not affected.
  • As soon as one of the steps above fail, the Job fails. If all steps succeed, we continue to the last step:
  • Jenkins will merge the newly pushed code into the master branch.
    When we deploy now, the new code will be included in production.

The process outlined above makes sure that we always know that new changes that go into production pass all tests.

Deploying to production

Once Jenkins informs us that all tests pass (we use a Slack hook), we can deploy the new code.

Once the code is deployed, we will run the same tests again, this time we set the environment variable to production.

If all goes well, these tests should all pass, since they already passed in our staging environment.


Since we're an Automated Testing company, we'd like to improve this process in the future.

When the staging tests pass, we can configure our Jenkins to automatically deploy the new code from the master branch. At this point, we've basically reached the Continuous Deployment level.

How TestingBot can help you

Interested in using this approach for your product? At TestingBot, we provide over 4600 browser combinations to run tests against.
Once you've created some Selenium tests and connected the tests with our grid, you can start testing your code on various platforms, browsers and devices.

TestingBot Logo

Sign up for a Free Trial

Start testing your apps with TestingBot.

No credit card required.

Other Articles

Automated and Live Testing in different Geographical Locations

We've added an option to our Live and Automated Testing platforms to specify from which country you'd like your test to run.

Read more
Automated & Manual Browser Testing on macOS Mojave

We're excited to announce that starting today, we've added macOS Mojave (mac 10.14) beta 5 to our list of available platforms.

Read more
GeoIP, Localised testing in various countries with proxies.

With the most recent release of our TestingBot Tunnel, version 2.8, we now provide GeoIP/localized testing.

Read more

If you're looking to learn more about Selenium, including tips and tricks, then we can recommend selenium.academy.

Read more

Good news for all Slack users: TestingBot now has its own Slack App which can be added to your Slack Workspace. You can immediately head over to...

Read more
TeamCity Plugin Now Available for TestingBot

We're pleased to announce that we've created a Plugin for TeamCity (a continuous-integration system created by Jetbrains).

Read more