---
title: Test Automation Framework Structure | TestingBot
description: A test automation framework is not a library you download. It is the
  set of conventions and small classes that decide how every test in your repository
  gets…
source_url:
  html: https://testingbot.com/resources/courses/page-object-model/framework-layout
  md: https://testingbot.com/resources/courses/page-object-model/framework-layout.md
---

Course

[Page Object Model and Framework Design](https://testingbot.com/resources/courses/page-object-model)

Chapter 7 of 10

1. [1 What Is the Page Object Model?](https://testingbot.com/resources/courses/page-object-model)
2. [2 Your First Page Object in Selenium](https://testingbot.com/resources/courses/page-object-model/first-page-object)
3. [3 Page Object Model in Playwright](https://testingbot.com/resources/courses/page-object-model/playwright-page-objects)
4. [4 Components and Fluent Page Objects](https://testingbot.com/resources/courses/page-object-model/components-and-fluent-pages)
5. [5 Page Factory vs Plain By Locators](https://testingbot.com/resources/courses/page-object-model/page-factory-vs-by-locators)
6. [6 Test Data, Config and Environments](https://testingbot.com/resources/courses/page-object-model/test-data-and-config)
7. [7 Test Automation Framework Structure](https://testingbot.com/resources/courses/page-object-model/framework-layout)
8. [8 Run the Framework in Parallel on a Grid](https://testingbot.com/resources/courses/page-object-model/parallel-on-a-grid)
9. [9 POM Anti-Patterns and When to Skip POM](https://testingbot.com/resources/courses/page-object-model/anti-patterns)
10. [10 Knowledge check](https://testingbot.com/resources/courses/page-object-model/knowledge-check)

[Exit course](https://testingbot.com/resources/courses)

 Chapter 7 of 10 Intermediate 8 min read 3 quick checks Updated Nov 2026 

# Test Automation Framework Structure
On this page
1. [Layers, and which way they depend](https://testingbot.com#layers-and-which-way-they-depend)
2. [Folder layout](https://testingbot.com#folder-layout)
3. [The driver factory and base test](https://testingbot.com#the-driver-factory-and-base-test)
4. [Reporting: make failures self-explaining](https://testingbot.com#reporting-make-failures-self-explaining)
5. [Retries, and the case against them](https://testingbot.com#retries-and-the-case-against-them)

A test automation framework is not a library you download. It is the set of conventions and small classes that decide how every test in your repository gets a browser, finds its data, reports its result and cleans up. With page objects, config and test data in place, this chapter fixes the layout, the base classes, and reporting, then looks hard at retries.

## Layers, and which way they depend[#](https://testingbot.com#layers-and-which-way-they-depend)

```
tests what to check (assertions live here)
  |
pages/components what the UI offers (locators and user actions)
  |
framework how to drive a browser (driver factory, config, waits, reporting)
  |
data who and with what (users, builders, data providers)
```

Dependencies point down only. Tests import pages and data; pages import the framework's helpers; the framework imports nothing from tests or pages. When a page object imports a test class, or the driver factory knows about `LoginPage`, the layering has broken and changes will start rippling across the suite.

Someone makes `DriverFactory.start()` log in through `LoginPage` so every test begins signed in. What is wrong with that?

 The framework layer now depends on a page object, so dependencies point up and changes ripple Nothing; it saves a line in every test It only breaks in Python

Dependencies point down only: tests use pages, pages use the framework. A factory that knows about `LoginPage` breaks the layering, and every test is forced to log in.

## Folder layout[#](https://testingbot.com#folder-layout)

Java (Maven, TestNG):

```
src/test/java/com/example/
  components/ SiteHeader.java
  data/ User.java
  framework/ Config.java, DriverFactory.java, ScreenshotOnFailure.java
  pages/ BasePage.java, LoginPage.java, LoggedInPage.java, PracticeIndexPage.java
  tests/ BaseTest.java, LoginTest.java, LoginErrorsTest.java, NavigationTest.java
src/test/resources/env/
  practice.properties, staging.properties
testng.xml
pom.xml
```

Python (pytest):

```
config/ practice.toml, staging.toml
pages/ __init__.py, base_page.py, login_page.py
tests/ test_login.py, test_login_errors.py
conftest.py driver, settings and reporting fixtures
settings.py
```

Playwright (TypeScript):

```
components/ site-header.ts
config/ env.ts
pages/ login-page.ts, logged-in-page.ts, practice-index-page.ts
tests/ fixtures.ts, login.spec.ts, login-errors.spec.ts, navigation.spec.ts
playwright.config.ts
testingbot.ts
```

Group by layer first, then by feature once a layer has more than a dozen files (`pages/checkout/`, `pages/account/`). Mirroring the application's navigation in the folder tree makes the right page object easy to find for someone new.

## The driver factory and base test[#](https://testingbot.com#the-driver-factory-and-base-test)

Tests should not create browsers. In Java, a `DriverFactory` owns that decision (local Chrome, local Firefox, or a remote grid, read from `Config`) and a `BaseTest` calls it around every test method:

```java
package com.example.tests;

import java.lang.reflect.Method;

import org.openqa.selenium.WebDriver;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;

import com.example.framework.DriverFactory;

public abstract class BaseTest {
    @BeforeMethod
    public void startDriver(Method method) {
        DriverFactory.start(getClass().getSimpleName() + "." + method.getName());
    }

    @AfterMethod(alwaysRun = true)
    public void stopDriver() {
        DriverFactory.quit();
    }

    protected WebDriver driver() {
        return DriverFactory.get();
    }
}
```

TestNG injects the `Method` into `@BeforeMethod`, so the factory receives a readable test name it can pass to a remote grid as the session name. `alwaysRun = true` makes sure the browser is quit even when setup of a later step failed. The factory itself, and why `driver()` is a method call rather than a field, is the subject of chapter 07.

In pytest, the `driver` fixture in `conftest.py` plays both roles, and in Playwright the built-in `page` fixture does. Neither language needs a base test class, which is a hint: inheritance is only one way to share setup, and in Java it is the conventional one.

Keep `BaseTest` this small. A base test with 40 helper methods is the test-side version of the god object discussed in chapter 08.

## Reporting: make failures self-explaining[#](https://testingbot.com#reporting-make-failures-self-explaining)

A red test in CI is only useful if someone can tell why from the report, without re-running it. Every framework needs, at minimum, a machine-readable result file for CI and a screenshot at the moment of failure.

**TestNG** writes JUnit-style XML into `target/surefire-reports/`. Screenshots come from a listener:

```java
package com.example.framework;

import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;

import org.openqa.selenium.OutputType;
import org.openqa.selenium.TakesScreenshot;
import org.testng.ITestListener;
import org.testng.ITestResult;

public class ScreenshotOnFailure implements ITestListener {
    @Override
    public void onTestFailure(ITestResult result) {
        try {
            byte[] png = ((TakesScreenshot) DriverFactory.get()).getScreenshotAs(OutputType.BYTES);
            Path dir = Path.of("target", "screenshots");
            Files.createDirectories(dir);
            Files.write(dir.resolve(result.getName() + ".png"), png);
        } catch (IOException | RuntimeException e) {
            System.err.println("Could not save screenshot: " + e.getMessage());
        }
    }
}
```

Register it with `@Listeners(ScreenshotOnFailure.class)` on `BaseTest`, or in `testng.xml`. `onTestFailure` runs before `@AfterMethod`, so the browser is still open. The `catch` matters: a reporting failure must never hide the real test failure.

**pytest** writes JUnit XML with `pytest --junitxml=reports/junit.xml`. For screenshots, wrap the report hook in `conftest.py`:

```python
from pathlib import Path

import pytest

@pytest.hookimpl(wrapper=True)
def pytest_runtest_makereport(item, call):
    report = yield
    if report.when == "call" and report.failed:
        drv = item.funcargs.get("driver")
        if drv is not None:
            out = Path("reports/screenshots")
            out.mkdir(parents=True, exist_ok=True)
            drv.save_screenshot(str(out / f"{item.name}.png"))
    return report
```

**Playwright** has the richest built-in story. With `reporter: [['list'], ['html', { open: 'never' }]]` and `trace: 'retain-on-failure'` in `use`, every failed test keeps a trace you can open with `npx playwright show-trace`: DOM snapshots before and after each action, network, console and the source line. Add `['junit', { outputFile: 'results.xml' }]` to the reporter list for CI.

Why does the `ScreenshotOnFailure` listener catch exceptions from taking the screenshot?

 Screenshots fail on every remote grid TestNG requires listeners to catch all exceptions A failure in reporting must never replace or hide the real test failure To retry the screenshot later

If the browser is already gone or the disk is full, the listener logs it and the report still shows the original assertion error.

## Retries, and the case against them[#](https://testingbot.com#retries-and-the-case-against-them)

Every runner can retry failed tests:

```typescript
// playwright.config.ts
retries: process.env.CI ? 1 : 0,
```

```java
package com.example.framework;

import org.testng.IRetryAnalyzer;
import org.testng.ITestResult;

public class RetryOnce implements IRetryAnalyzer {
    private int attempts = 0;

    @Override
    public boolean retry(ITestResult result) {
        if (attempts < 1) {
            attempts++;
            return true;
        }
        return false;
    }
}
```

```bash
pip install pytest-rerunfailures
pytest --reruns 1
```

In TestNG, attach it with `@Test(retryAnalyzer = RetryOnce.class)`. When we ran a deliberately failing test with it, TestNG reported the first attempt as **skipped** and the second as failed, so a retried-then-passed test looks like a pass with one skip. Playwright is more honest: a test that fails and then passes is reported as **flaky** , separately from passed.

Retries make the build green. They do not make the product or the test correct. The arguments against leaning on them:

- **They hide real bugs.** A race condition in the application (double-submit, a slow API that sometimes times out) fails one run in ten. A retry turns that signal into a green tick.
- **They hide test bugs too.** Most flaky UI tests have a concrete cause: a missing wait, shared data, test order dependency. The Selenium docs list "test dependency" among their discouraged practices for exactly this reason. Each of those has a fix; a retry postpones it.
- **They cost time.** A suite with a 5% flake rate and one retry runs noticeably longer, on every build, forever.
- **They erode trust.** Once people know "it usually passes on retry", they stop reading failures.

A reasonable policy:

1. Retries off locally (`retries: 0`). You want to see flakiness while you work.
2. At most one retry in CI, and only so a single flake does not block an unrelated merge.
3. Treat "flaky" as a failure that is tracked. In Playwright, `failOnFlakyTests: !!process.env.CI` in the config (or `--fail-on-flaky-tests` on the command line) fails the run if any test needed a retry; use it on your main branch even if pull requests allow retries.
4. Keep the trace or video from the **first** failing attempt. That is the evidence. `trace: 'retain-on-failure'` keeps it; `on-first-retry` only starts recording on the retry, which is cheaper but loses the original failure.
5. Fix or quarantine flaky tests within a fixed time. A quarantined test that nobody owns is a deleted test.

Our article on [handling test flakiness](https://testingbot.com/resources/articles/handling-test-flakiness) goes deeper into root causes. For how TestingBot records the session video and logs for every remote test, see [test results in the Selenium docs](https://testingbot.com/support/web-automate/selenium/test-results).

With a TestNG `IRetryAnalyzer` that retries once, a test fails and then passes. How does it show up in the TestNG results?

 As flaky, like Playwright As failed As one skip and one pass, so it looks like a pass

TestNG reports the first attempt as skipped. Unless you read the skip count, the flakiness disappears, which is one more reason to track retried tests explicitly.

[Previous · Chapter 6 Test Data, Config and Environments](https://testingbot.com/resources/courses/page-object-model/test-data-and-config) [Next · Chapter 8 Run the Framework in Parallel on a Grid](https://testingbot.com/resources/courses/page-object-model/parallel-on-a-grid)

On this page

1. [Layers, and which way they depend](https://testingbot.com#layers-and-which-way-they-depend)
2. [Folder layout](https://testingbot.com#folder-layout)
3. [The driver factory and base test](https://testingbot.com#the-driver-factory-and-base-test)
4. [Reporting: make failures self-explaining](https://testingbot.com#reporting-make-failures-self-explaining)
5. [Retries, and the case against them](https://testingbot.com#retries-and-the-case-against-them)

Certification

This course has a free, hands-on certificate.

[How it works](https://testingbot.com/resources/courses/page-object-model/certification)