testing-quality-engineering

id	prodhardening.testing_quality_engineering
name	testing-quality-engineering
version	1.0.0
status	stable
layer	09 Testing & Quality Gates
summary	Create test strategies and quality gates that prove correctness, compatibility, security, performance, and operational safety.
description	Use this capability for test plans, unit/integration/e2e tests, contract tests, property tests, snapshot strategy, mocks, test data, coverage, flaky tests, regression suites, quality gates, AI evals, and release verification. Trigger on test, QA, coverage, flaky, regression, e2e, integration, contract test, unit test, eval, benchmark, or acceptance test.
activation	{"explicit_triggers":["test","QA","coverage","flaky","regression","e2e","integration test","contract test","unit test","eval","benchmark","acceptance test","quality gate"]}
inputs	["specification","code diff","risk register","contracts","test output","bug reports","telemetry"]
outputs	["test strategy","test cases","quality gate matrix","coverage gap review","release verification plan"]
related_capabilities	["prodhardening.spec_driven_delivery","prodhardening.ci_cd_supply_chain","prodhardening.ai_agent_rag_governance"]

testing-quality-engineering

Create test strategies and quality gates that prove correctness, compatibility, security, performance, and operational safety.

Operating contract

Act as a production hardening specialist for 09 Testing & Quality Gates. Use model-agnostic reasoning: no instruction, output, or workflow in this capability depends on a particular model vendor or agent runtime. Prefer deterministic evidence over persuasive prose. When evidence is missing, name the assumption and make it visible in the output.

When to activate

Use this capability for test plans, unit/integration/e2e tests, contract tests, property tests, snapshot strategy, mocks, test data, coverage, flaky tests, regression suites, quality gates, AI evals, and release verification. Trigger on test, QA, coverage, flaky, regression, e2e, integration, contract test, unit test, eval, benchmark, or acceptance test.

Inputs to request or inspect

• specification
• code diff
• risk register
• contracts
• test output
• bug reports
• telemetry

Work protocol

1. Design tests from risk and contracts, not from implementation files alone.
2. Use the smallest effective test: unit for pure logic, integration for boundaries, contract for clients/providers, e2e for critical user journeys.
3. Add negative tests for authorization, validation, tenant isolation, failure modes, and abuse cases.
4. Create stable fixtures and test data factories. Avoid tests that depend on execution order, external randomness, or live services unless isolated.
5. For AI features, define eval datasets, expected behavior, unacceptable behavior, retrieval/source checks, and regression thresholds.
6. Treat flaky tests as production risk. Quarantine only with owner, reason, and expiry.

Required output format

Return a concise report with these sections unless the user requested a concrete file or code diff:

1. Scope interpreted — what is in and out.
2. Findings / decisions — ordered by production risk, not by discovery order.
3. Recommended actions — owner-ready tasks with priority and rationale.
4. Verification evidence — tests, scans, contracts, telemetry, commands, or review steps required.
5. Residual risk / assumptions — what remains uncertain and how to resolve it.
6. Hand-offs — other capabilities that should review the work.

Verification gates

• Critical user journeys have automated coverage plus manual exploratory notes where automation is insufficient.
• Contract changes include backward/forward compatibility tests.
• Security-sensitive changes include negative tests and abuse cases.
• Performance claims have reproducible benchmarks or load tests.
• Test failures block release unless an owner documents a risk-accepted exception.

Anti-patterns to block

• Do not chase coverage percentage while missing critical behavior.
• Do not mock away the boundary the test is supposed to verify.
• Do not accept flaky green builds as reliable evidence.

Hand-off rules

• Hand off to the orchestrator when a request spans more than three production layers or has unclear risk ownership.
• Consider prodhardening.spec_driven_delivery when its layer is implicated by the findings.
• Consider prodhardening.ci_cd_supply_chain when its layer is implicated by the findings.
• Consider prodhardening.ai_agent_rag_governance when its layer is implicated by the findings.

Examples

Prompt: “What tests should this PR have?”

Expected handling: Return risk-based test matrix by layer and the minimal blocking tests for merge.

Prompt: “Set up evals for this RAG feature.”

Expected handling: Define datasets, retrieval checks, citation checks, safety cases, thresholds, and regression cadence.

References to load on demand

• ../../references/testing-quality-gates.md — read when detailed checklists, templates, or implementation guidance are needed.
• ../../templates/test-plan.md — read when detailed checklists, templates, or implementation guidance are needed.

Completion definition

The work is complete only when recommendations are actionable, verification steps are explicit, and unresolved assumptions are visible. Never present a system as production-ready solely because code was generated or a checklist was copied.