test-automation-framework

name	test-automation-framework
description	Build a scalable test automation framework with shared utilities, reporting, and CI integration. Outputs framework architecture, base classes, fixtures, parallel execution setup, and reporting pipeline.
argument-hint	["language","test types","team size","CI platform","reporting requirements"]
allowed-tools	Read, Write, Bash

Test Automation Framework

A test automation framework provides the infrastructure that makes tests easier to write, maintain, and execute. It standardises patterns, provides shared utilities, handles setup/teardown, and produces consistent reporting. Without it, each team member writes tests differently and shared problems are solved repeatedly.

Framework Architecture

tests/
├── framework/                  # Shared framework code
│   ├── clients/                # API clients, DB helpers
│   │   ├── api_client.py
│   │   └── db_client.py
│   ├── factories/              # Test data factories
│   │   ├── user_factory.py
│   │   └── order_factory.py
│   ├── fixtures/               # Pytest fixtures
│   │   ├── auth.py
│   │   ├── database.py
│   │   └── environment.py
│   ├── assertions/             # Custom assertions
│   │   └── api_assertions.py
│   └── reporting/              # Report generation
│       └── html_reporter.py
│
├── unit/                       # Unit tests
├── integration/                # Integration tests
├── api/                        # API tests
├── e2e/                        # End-to-end tests
│
├── conftest.py                 # Root fixtures
├── pytest.ini                  # Configuration
└── requirements-test.txt       # Test dependencies

Base API Client

# framework/clients/api_client.py
import httpx
import json
import logging
from typing import Optional, Any
from dataclasses import dataclass

logger = logging.getLogger(__name__)

@dataclass
class APIResponse:
    status_code: int
    body: Any
    headers: dict
    elapsed_ms: float
    
    def assert_status(self, expected: int):
        assert self.status_code == expected, \
            f"Expected {expected}, got {self.status_code}. Body: {self.body}"
        return self
    
    def assert_field(self, field: str, expected=None):
        assert field in self.body, f"Field '{field}' missing from response: {self.body}"
        if expected is not None:
            assert self.body[field] == expected, \
                f"Field '{field}': expected {expected!r}, got {self.body[field]!r}"
         
    
     ():
        missing = [f  f  required_fields  f   .body]
          missing, 
         

 :
     ():
        .base_url = base_url.rstrip()
        ._default_headers = default_headers  {: }
        ._timeout = timeout
        ._client = httpx.Client(timeout=timeout)
    
     ():
        ._default_headers[] = 
         
    
     () -> APIResponse:
        url = 
        headers = {**._default_headers, **kwargs.pop(, {})}
        
        logger.debug()
        response = ._client.request(method, url, headers=headers, **kwargs)
        
        :
            body = response.json()
         Exception:
            body = response.text
        
        elapsed = response.elapsed.total_seconds() * 
         APIResponse(response.status_code, body, (response.headers), elapsed)
    
     () -> APIResponse:
         .request(, path, **kwargs)
    
     () -> APIResponse:
         .request(, path, json=json, **kwargs)
    
     () -> APIResponse:
         .request(, path, json=json, **kwargs)
    
     () -> APIResponse:
         .request(, path, json=json, **kwargs)
    
     () -> APIResponse:
         .request(, path, **kwargs)

Test Data Factories

# framework/factories/order_factory.py
from dataclasses import dataclass, field
from typing import Optional
import uuid
from datetime import datetime

@dataclass
class OrderItemData:
    product_id: str = None
    quantity: int = 1
    unit_price: float = 29.99
    
    def __post_init__(self):
        if self.product_id is None:
            self.product_id = f"prod-{uuid.uuid4().hex[:8]}"

@dataclass
class OrderData:
    customer_id: str = None
    items: list = field(default_factory=list)
    shipping_address: str = "123 Test St, Springfield, US 12345"
    status: str = "draft"
    notes: Optional[str] = None
    
    def __post_init__(self):
        if self.customer_id is None:
            .customer_id = 
          .items:
            .items = [OrderItemData()]

 :
     ():
        .api = api_client
        ._created_ids = []
    
     () -> :
        
        data = OrderData(**overrides)
         {
            : data.customer_id,
            : [
                {: i.product_id, : i.quantity}
                 i  data.items
            ],
            : data.shipping_address,
            : data.notes,
        }
    
     () -> :
        
        payload = .build(**overrides)
        response = .api.post(, json=payload)
        response.assert_status()
        order = response.body
        ._created_ids.append(order[])
         order
    
     () -> :
        
        order = .create(**overrides)
        
        .api.post()
         {**order, : }
    
     ():
        
         order_id  ._created_ids:
            .api.delete()
        ._created_ids.clear()

Fixtures

# framework/fixtures/environment.py
import pytest
import os
from framework.clients.api_client import APIClient
from framework.factories.order_factory import OrderFactory

@pytest.fixture(scope="session")
def base_url() -> str:
    return os.environ.get("API_BASE_URL", "http://localhost:8080")

@pytest.fixture(scope="session")
def api_client(base_url) -> APIClient:
    return APIClient(base_url)

@pytest.fixture
def auth_client(api_client) -> APIClient:
    """Authenticated API client. Re-authenticates per test for isolation."""
    response = api_client.post("/auth/login", json={
        "email": os.environ["TEST_USER_EMAIL"],
        "password": os.environ["TEST_USER_PASSWORD"],
    })
    token = response.body["access_token"]
    
    # Create new client instance with auth to avoid polluting shared client
    client = APIClient(api_client.base_url)
    client.set_auth(token)
    return client

@pytest.fixture
def order_factory(auth_client) -> OrderFactory:
    factory = OrderFactory(auth_client)
    yield factory
    factory.cleanup()  # Auto-cleanup after each test

@pytest.fixture
 () -> :
    
     order_factory.create()


 () -> :
    
     order_factory.create_paid()

Parallel Execution

# pytest.ini
[pytest]
addopts =
    -n auto                    # Use all CPU cores
    --dist=loadscope           # Group by module to reduce fixture overhead
    --reruns=2                 # Retry flaky tests
    --reruns-delay=1
    -v
    --tb=short
    --strict-markers

markers =
    smoke: Critical path
    regression: Full suite
    nightly: Slow tests
    serial: Must not run in parallel
    flaky: Known flaky — track but don't block

# Mark tests that must run serially (e.g., modifying shared state)
@pytest.mark.serial
def test_modifies_global_config():
    ...

# Worker-scoped fixtures for parallel safety
@pytest.fixture(scope="function")
def isolated_db(worker_id):
    """Create worker-specific database to avoid parallel test interference."""
    db_name = f"test_db_{worker_id}"
    create_database(db_name)
    yield get_db_connection(db_name)
    drop_database(db_name)

Reporting

# framework/reporting/html_reporter.py
import json
from pathlib import Path
from jinja2 import Template
from datetime import datetime

def generate_report(results: list, output_path: str):
    summary = {
        "total": len(results),
        "passed": sum(1 for r in results if r["outcome"] == "passed"),
        "failed": sum(1 for r in results if r["outcome"] == "failed"),
        "skipped": sum(1 for r in results if r["outcome"] == "skipped"),
        "duration_s": sum(r.get("duration", 0) for r in results),
        "generated_at": datetime.now().isoformat(),
    }
    summary["pass_rate"] = summary["passed"] / summary["total"] if summary["total"] else 
    
    failures = [r  r  results  r[] == ]
    slowest = (results, key= r: r.get(, ), reverse=)[:]
    
    
    Path(output_path).with_suffix().write_text(
        json.dumps({: summary, : failures, : slowest})
    )

Anti-Patterns to Avoid

Anti-Pattern	Problem	Fix
No shared client/factory	Every developer writes their own test setup	Base client and factories in framework/
Tests with global state	Parallel execution causes races	Isolated fixtures; per-test cleanup
No cleanup after tests	Test data accumulates; tests interfere	Factory.cleanup() in fixture teardown
Hardcoded test data	Tests break when data changes	Factories with generated data
Copy-paste test setup	50 tests each with 20 lines of setup	Reusable fixtures; base test classes
No retry for flaky tests	CI noise from transient failures	--reruns=2 with delay; fix underlying flakiness
Reporting only in CI	Local test failures lack context	HTML report generated locally and in CI

10 Rules

1. Framework code is production code — test it, review it, document it.
2. Every test is self-contained — it creates and cleans up all its own data.
3. Factories generate unique data per test — no hardcoded IDs or emails.
4. Authentication is per-test, not per-session — token expiry shouldn't cause cascade failures.
5. Parallel execution is the default — tests that can't run in parallel must be explicitly marked.
6. Cleanup always runs, even when tests fail — use fixtures with yield for guaranteed teardown.
7. Assertions are descriptive — failure messages tell you what went wrong, not just that it did.
8. Framework changes require team approval — breaking shared code breaks everyone's tests.
9. Report generation is automatic — every CI run produces an HTML report as an artifact.
10. Framework version is pinned — automatic updates break tests in surprising ways.