Generative AI and Automated Testing

Question 1

Why use Generative AI for Test Data generation?

Answer

Generative AI, which is a subset of artificial intelligence (AI), gives machines the ability to fabricate data that looks like real-world data. By analyzing patterns in existing datasets, Generative AI algorithms generate data points that maintain the underlying characteristics of the original dataset. Because of these capabilities, Generative AI can create diverse and realistic data sets, which covers a wide range of scenarios and edge cases that are difficult to replicate manually.

Question 2

Selenium and Generative AI

Answer

While Selenium excels at emulating user interactions, it often relies on predefined datasets for testing. This means it might not fully capture the complexity of real-world scenarios, for example with testing payment methods such as credit cards, or filling in user credentials such as a street address.

Generative Artificial Intelligence, or Generative AI, can help with generating random but valid looking data to be used in your automated Selenium tests.

One popular solution to generate fake test data is to use a library called Faker, which generates realistic data for tests.

To use generative AI with your tests, you can use various libraries in your test automation code. Below are some examples:

IBM Generative AI SDK

Using IBM Generative AI SDK, you can generate fake data by providing a prompt to the SDK, for example:

const address = await model.call(
  'Generate a valid address for a house located in California, United States',
)
// next, use the 'address' in your test automation code to for example fill in a form

Google AI Generative Language

Another popular solution, from Google, is @google-ai/generativelanguage (PaLM API). You can generate valid looking test data, such as a firstName and lastName with this SDK:

const { TextServiceClient } =
  require("@google-ai/generativelanguage").v1beta2;

const { GoogleAuth } = require("google-auth-library");

const MODEL_NAME = "models/text-bison-001";
const API_KEY = process.env.API_KEY;

const client = new TextServiceClient({
  authClient: new GoogleAuth().fromAPIKey(API_KEY),
});

function getUserDetails() {
	return new Promise((resolve) => {
		const prompt = "Generate a first name and a last name for a person living in Canada";

		client
		  .generateText({
		    model: MODEL_NAME,
		    prompt: {
		      text: prompt,
		    },
		  })
		  .then((result) => {
		    const { firstName, lastName } = result.split(' ')
		    resolve({
		    	firstName,
		    	lastName
		    })
		  });
		}
	})
}


const webdriver = require('selenium-webdriver');
const capabilities = {
 'platform' : 'WIN10',
 'browserName' : 'chrome',
 'version' : 'latest',
 'name': 'NodeJS Sample Test'
}
async function runTest () {
  const { firstName, lastName } = await getUserDetails() // fetch new fake generative AI test data

  let driver = new webdriver.Builder()
    .usingServer('https://hub.testingbot.com/wd/hub')
    .withCapabilities(capabilities)
    .build();
  await driver.get("https://www.google.com/ncr");
  const inputField = await driver.findElement(webdriver.By.name("q"));
  // Search google with the generate firstName and lastName

  await inputField.sendKeys(firstName + ' ' + lastName, webdriver.Key.ENTER);
  await driver.quit();
}
runTest();

The example above will start a new Selenium test session on TestingBot, navigate to Google, and enter a name generated by Google AI's Generative Language in the Google search input box. The test will then quit (close the browser).

Generate testdata with intellinode

This NodeJS SDK allows you to use various AI models, such as OpenAI, Cohere, LLaMa v2, Google PaLM, and others. For example, to use with OpenAI's ChatGPT, you can use the example below, after installing intellinode with npm i intellinode:

const { Chatbot, ChatGPTInput } = require('intellinode');

// set the api key for OpenAI

const chatbot = new Chatbot(apiKey);

const input = new ChatGPTInput('You are helping with creating valid looking data');
input.addUserMessage('Generate a realistic looking name and home address for a person living in the US');

const testdata = await chatbot.chat(input);
let driver = new webdriver.Builder()
    .usingServer('https://hub.testingbot.com/wd/hub')
    .withCapabilities(capabilities)
    .build();
  await driver.get("https://www.google.com/ncr");
  const inputField = await driver.findElement(webdriver.By.name("q"));
  // Search google with the generate firstName and lastName

  await inputField.sendKeys(testData, webdriver.Key.ENTER);
  await driver.quit();

The example above starts a Selenium session and enters data generated with ChatGPT, using a prompt that asks the Chatbot to create realistic looking test data.

The synergy between Generative AI and Selenium Automated Tests is a paradigm shift in software testing. It opens the door to improved testing coverage, enhanced accuracy, and faster testing cycles.