Menu
Module: Testing Strategy and Coverage Standard
The review-first prompt is selected by default. You can switch to a direct command or download a local copy.
Read SKILL.md and any companion files shown by SkillsMP before deciding whether to install.
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
A direct command skips the review prompt. Inspect the source before running it.
npx skills add https://github.com/lebiraja/skills4agents --skill agent-module-testing-strategy-and-coverageThe command stays on one line. Scroll horizontally to inspect it before copying.
Prefer a local copy? Download the files currently available to SkillsMP.
Based on SOC occupation classification
Showing SKILL.md
| name | agent-module-testing-strategy-and-coverage |
| description | Module: Testing Strategy and Coverage Standard |
agent.module.testing-strategy-and-coverage1.0.0productionUse this module to establish a production-grade testing discipline for backend services, frontend applications, and AI integration points.
Apply when:
Do not apply directly when:
All projects adopt this canonical pyramid (bottom = fastest and most, top = slowest and least):
E2E Tests (5–10% of test count)
/‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾\
/ Contract Tests \
/ (Integration) \
/‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾\
/ Service/Unit Tests \
/ (60–70% of test count) \
/________________________________\
Test layer definitions:
Scope: Individual functions, classes, and modules in isolation. Mocking: Mock all external dependencies (DB, services, APIs). Speed: < 100 ms per test; thousands run in seconds. Count: 60–70% of all tests.
Coverage: Business logic, algorithms, edge cases, error paths.
Example domains:
Scope: Multiple components working together (e.g., service + in-process DB, service + mock external API). Mocking: Mock external services; use real or test containers for databases. Speed: 100 ms–2 s per test; hundreds run in seconds to minutes. Count: 20–30% of all tests.
Coverage: Happy paths, failure paths, error handling, contract fulfillment.
Example domains:
Scope: Service API contracts between independent services. Mocking: Mock other services; verify request/response shape and semantics. Speed: 100 ms–1 s per test; tens to hundreds. Count: 5–10% of all tests.
Coverage: Request/response schema, error handling, versioning.
Example domains:
Scope: Full system: frontend → BFF → backend → database → external services. Mocking: Minimize mocking; use real or stubbed external services (with careful handling). Speed: 1–10 s per test; tens run in minutes. Count: 5–10% of all tests.
Coverage: Critical user journeys, cross-service workflows, rollback scenarios.
Example domains:
>= 90% of cyclomatic complexity>= 80% of lines>= 70% of lines>= 50% of linesCalculation: Use coverage tools; configure to fail CI if targets not met.
Use factory libraries (e.g., FactoryBoy in Python, factory_bot in Ruby) to generate realistic, deterministic test data.
import factory
from users.models import User
class UserFactory(factory.django.DjangoModelFactory):
class Meta:
model = User
email = factory.Sequence(lambda n: f"user{n}@example.com")
phone = factory.Sequence(lambda n: f"+1234567890{n % 10}")
first_name = factory.Faker('first_name')
last_name = factory.Faker('last_name')
is_active = True
# Usage:
user = UserFactory()
users = UserFactory.create_batch(10)
Keep fixtures minimal; prefer factories for dynamic generation.
# Good: Small, reusable fixture
@pytest.fixture
def base_user():
return UserFactory(email="base@example.com")
# Avoid: Large fixture databases
@pytest.fixture(scope="session")
def all_test_data():
# Creates 10k records – slow, hard to reason about
...
Ensure all tests are deterministic:
now().import random
from datetime import datetime, timedelta
@pytest.fixture(autouse=True)
def seed_rng():
random.seed(42)
def test_user_created_at():
now = datetime(2026, 4, 4, 12, 0, 0)
user = UserFactory(created_at=now)
assert user.created_at == now
tests/unit/<domain>/test_<module>.pytest_<function>_<scenario>_<expected_result>Example structure:
src/
users/
models.py
validators.py
service.py
tests/
unit/
users/
test_validators.py
test_service.py
Example test:
# tests/unit/users/test_validators.py
import pytest
from users.validators import validate_email
class TestValidateEmail:
def test_valid_gmail_address(self):
# Arrange
email = "user@gmail.com"
# Act & Assert
assert validate_email(email) is True
def test_non_gmail_rejected(self):
# Arrange
email = "user@hotmail.com"
# Act & Assert
with pytest.raises(ValidationError) as exc_info:
validate_email(email)
assert exc_info.value.error_code == "VALIDATION_EMAIL_DOMAIN"
def test_invalid_format_rejected(self):
email = "invalid-email"
with pytest.raises(ValidationError) as exc_info:
validate_email(email)
assert exc_info.value.error_code == "VALIDATION_EMAIL_FORMAT"
Exit criteria:
>= 70% line coverage.tests/integration/test_<endpoint>.pyExample:
# tests/integration/test_user_signup.py
import pytest
from django.test import Client
from users.models import User
@pytest.fixture
def client():
return Client()
@pytest.fixture
def db():
# Use real test database
pass
class TestUserSignupFlow:
def test_signup_success_creates_user(self, client, db):
# Arrange
payload = {
"email": "newuser@gmail.com",
"first_name": "John",
"last_name": "Doe"
}
# Act
response = client.post("/api/v1/users/signup", json=payload)
# Assert
assert response.status_code == 201
assert response.json()["user_id"]
user = User.objects.get(email="newuser@gmail.com")
assert user.first_name == "John"
def test_signup_duplicate_email_rejected(self, client, db):
# Arrange
UserFactory(email="existing@gmail.com")
payload = {"email": "existing@gmail.com", "first_name": "Jane"}
# Act
response = client.post("/api/v1/users/signup", json=payload)
# Assert
assert response.status_code == 409
assert response.json()["error"]["code"] == "CONFLICT_DUPLICATE_EMAIL"
def test_signup_invalid_email_rejected(self, client, db):
# Arrange
payload = {"email": "invalid-email", "first_name": "Jane"}
# Act
response = client.post("/api/v1/users/signup", json=payload)
# Assert
assert response.status_code == 400
assert response.json()["error"]["code"] == "VALIDATION_EMAIL_FORMAT"
Exit criteria:
Example (using Pydantic):
# tests/integration/test_user_api_contract.py
from pydantic import BaseModel, Field, validator
import pytest
class UserSignupRequest(BaseModel):
email: str = Field(..., regex=r"^[a-z0-9]+@gmail\.com$")
first_name: str = Field(..., min_length=1, max_length=100)
last_name: str = Field(..., min_length=1, max_length=100)
class UserResponse(BaseModel):
user_id: str
email: str
created_at: str # ISO 8601
def test_signup_response_matches_schema():
payload = {"email": "user@gmail.com", "first_name": "John", "last_name": "Doe"}
response = client.post("/api/v1/users/signup", json=payload)
# This raises if response doesn't match schema
user_response = UserResponse(**response.json()["user"])
assert user_response.user_id
Exit criteria:
Example:
# tests/e2e/test_user_signup_and_login.py
import pytest
from selenium import webdriver
@pytest.fixture
def browser():
driver = webdriver.Chrome()
yield driver
driver.quit()
def test_signup_and_login_workflow(browser):
# Step 1: Navigate to signup
browser.get("https://test.example.com/signup")
# Step 2: Fill signup form
browser.find_element("email").send_keys("newuser@gmail.com")
browser.find_element("first_name").send_keys("John")
browser.find_element("last_name").send_keys("Doe")
browser.find_element("submit").click()
# Step 3: Verify success
assert browser.find_element("success_message")
# Step 4: Login
browser.get("https://test.example.com/login")
browser.find_element("email").send_keys("newuser@gmail.com")
browser.find_element("password").send_keys("password123")
browser.find_element("submit").click()
# Step 5: Verify dashboard loaded
assert browser.find_element("dashboard")
Exit criteria:
Example:
# tests/integration/test_error_paths.py
import pytest
from unittest.mock import patch
class TestErrorHandling:
def test_external_service_timeout_handled_gracefully(self):
with patch('external_api.call') as mock_call:
mock_call.side_effect = TimeoutError("Service timeout")
response = client.post("/api/v1/enrichment", json={"text": "hello"})
assert response.status_code == 504
assert response.json()["error"]["code"] == "EXTERNAL_SERVICE_TIMEOUT"
assert response.json()["error"]["retryable"] is True
def test_rate_limit_returns_429(self):
with patch('rate_limiter.allow_request', return_value=False):
response = client.post("/api/v1/enrichment", json={"text": "hello"})
assert response.status_code == 429
assert "Retry-After" in response.headers
Exit criteria:
Example (using Locust or k6):
# load_tests/users_api.py
from locust import HttpUser, task, between
class UsersAPIUser(HttpUser):
wait_time = between(1, 3)
@task(2)
def list_users(self):
self.client.get("/api/v1/users")
@task(1)
def create_user(self):
self.client.post("/api/v1/users", json={
"email": "user@gmail.com",
"first_name": "John"
})
Exit criteria:
Example CI configuration:
# .github/workflows/test.yml
name: Test Suite
on: [push, pull_request]
jobs:
unit-tests:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: '3.11'
- run: pip install -r requirements.txt
- run: pytest tests/unit/ -v --cov=src --cov-fail-under=70
integration-tests:
runs-on: ubuntu-latest
services:
postgres:
image: postgres:15
env:
POSTGRES_PASSWORD: password
steps:
- uses: actions/checkout@v3
- run: pytest tests/integration/ -v
e2e-tests:
runs-on: ubuntu-latest
if: github.event_name == 'pull_request'
steps:
- uses: actions/checkout@v3
- run: docker-compose -f docker-compose.test.yml up -d
- run: pytest tests/e2e/ -v
- run: docker-compose -f docker-compose.test.yml down
Exit criteria:
@pytest.fixture
def db_session(db):
# db fixture from pytest-django provides isolated DB
yield db
@pytest.fixture(autouse=True)
def cleanup_after_test(db):
yield
db.session.rollback() # Auto-cleanup
import responses
@responses.activate
def test_payment_processing():
responses.add(
responses.POST,
"https://payment-api.example.com/charge",
json={"charge_id": "ch_123"},
status=200
)
result = payment_service.charge(amount=100)
assert result.charge_id == "ch_123"
| Decision Area | Preferred Option | Alternative | Selection Rule |
|---|---|---|---|
| Test organization | By domain/feature | By test type | Use domain-based organization for discoverability. |
| DB for integration tests | Real test DB | In-memory/SQLite | Use real DB to catch SQL/transaction bugs. |
| External API mocking | Recorded responses (VCR) | Full mock library | Use VCR to maintain realistic payloads. |
| Coverage enforcement | CI gate (fail build if below target) | Advisory-only | Use CI gate for critical paths; advisory for others. |
| E2E framework | Browser-based (Selenium/Playwright) | API-only | Use browser-based for UX-heavy flows. |
| Test data generation | Factories (FactoryBoy) | Fixed fixtures | Use factories for flexibility and realism. |
<= 30 seconds for entire suite<= 5 minutes for entire suite<= 15 minutes for critical journeys>= 90%>= 80%>= 70%<= 0.5% (re-run pass rate >= 99.5%)100% for regressions<= 2 hoursThis module is reusable across all backend and frontend projects. Adapt only: