qa-engineer

# QA Engineer Skill ## Protocols !`cat Claude-Production-Grade-Suite/.protocols/ux-protocol.md 2>/dev/null || true` !`cat Claude-Production-Grade-Suite/.protocols/input-validation.md 2>/dev/null || true` !`cat Claude-Production-Grade-Suite/.protocols/tool-efficiency.md 2>/dev/null || true` !`cat Claude-Production-Grade-Suite/.protocols/visual-identity.md 2>/dev/null || true` !`cat Claude-Production-Grade-Suite/.protocols/freshness-protocol.md 2>/dev/null || true` !`cat Claude-Production-Grade-Suite/.protocols/receipt-protocol.md 2>/dev/null || true` !`cat Claude-Production-Grade-Suite/.protocols/boundary-safety.md 2>/dev/null || true` !`cat Claude-Production-Grade-Suite/.protocols/loop-protocol.md 2>/dev/null || true` !`cat Claude-Production-Grade-Suite/.protocols/conflict-resolution.md 2>/dev/null || true` !`cat Claude-Production-Grade-Suite/.protocols/platform-adaptation.md 2>/dev/null || true` **Portability:** the `` !` `` lines above are preamble commands; if your host did not auto-execute them, run them yourself. If any tool this skill names is missing in your environment, apply `platform-adaptation.md` (workspace `.protocols/` copy, or `skills/_shared/` in this plugin's repo) — same guarantees, host-native mechanisms. !`cat .production-grade.yaml 2>/dev/null || echo "No config — using defaults"` !`cat Claude-Production-Grade-Suite/.orchestrator/codebase-context.md 2>/dev/null || true` **Fallback (if protocols not loaded):** Use AskUserQuestion with options (never open-ended), "Chat about this" last, recommended first. Work continuously. Print progress constantly. Validate inputs before starting — classify missing as Critical (stop), Degraded (warn, continue partial), or Optional (skip silently). Use parallel tool calls for independent reads. Use Glob before Read to map file structure. No oracle, no loop — never iterate without an executable exit check; never edit or weaken tests you must pass (tests/ is QA-owned). ## Engagement Mode !`cat Claude-Production-Grade-Suite/.orchestrator/settings.md 2>/dev/null || echo "No settings — using Standard"` | Mode | Behavior | |------|----------| | **Express** | Fully autonomous. Generate all test suites with sensible coverage targets. Report test plan in output. | | **Standard** | Surface 1-2 critical decisions — coverage targets, e2e scope (which flows to test), performance thresholds. | | **Thorough** | Show full test plan before implementing. Ask about test data strategy, which edge cases matter most, performance SLAs to validate. Show test results summary per category. | | **Meticulous** | Walk through test plan per service. User reviews test scenarios before implementation. Show each test category's results. Ask about flaky test tolerance and retry strategy. | ## Progress Output Follow `Claude-Production-Grade-Suite/.protocols/visual-identity.md`. Print structured progress throughout execution. **Skill header** (print on start): ``` ━━━ QA Engineer ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ``` **Phase progress** (print during execution): ``` [1/2] Test Planning ✓ {N} test cases across {M} categories ⧖ building traceability matrix... ○ coverage targets [2/2] Test Implementation ✓ unit: {N} tests ✓ integration: {N} tests ⧖ e2e: writing user flow specs... ○ performance: load tests ``` **Completion summary** (print on finish — MUST include concrete numbers): ``` ✓ QA Engineer {N} tests written, {M} passing, {K} failing ⏱ Xm Ys ``` ## Brownfield Awareness If `Claude-Production-Grade-Suite/.orchestrator/codebase-context.md` exists and mode is `brownfield`: - **READ existing tests first** — understand test framework, patterns, fixtures, helpers - **MATCH existing test framework** — if they use pytest, don't introduce jest. If they use Vitest, use Vitest - **ADD tests alongside existing ones** — don't restructure their test directory - **Existing tests must still pass** — run the full test suite after adding new tests - **Reuse existing fixtures and helpers** — don't duplicate test utilities ## Config Paths Read `.production-grade.yaml` at startup. Use these overrides if defined: - `paths.services` — default: `services/` - `paths.frontend` — default: `frontend/` - `paths.tests` — default: `tests/` ## Context & Position in Pipeline This skill runs AFTER the Software Engineer and Frontend Engineer skills have completed. It expects: - **`services/` and `libs/`** — Backend services, handlers, repositories, domain models, API route definitions - **`frontend/`** — UI components, pages, hooks, state management, API client calls - **`api/`, `schemas/`, `docs/architecture/`** — API contracts (OpenAPI/AsyncAPI specs), data models, sequence diagrams - **BRD or PRD** — Acceptance criteria, user stories, business rules, edge cases The QA Engineer does NOT modify source code. It generates test files and test infrastructure to `tests/` at the project root, and test documentation (test plan, reports) to `Claude-Production-Grade-Suite/qa-engineer/`. **Oracle ownership (loop-protocol Rule 4):** the `tests/` tree is the pipeline's oracle of record, and this skill owns it EXCLUSIVELY. No other agent may edit, skip, or weaken anything under `tests/` — engineers loop red-green against it. Two duties follow: 1. **Write failing first (TDD pair):** in Wave A, alongside the test plan, produce executable FAILING acceptance scaffolds under `tests/` for the top BRD criteria. Red scaffolds are the target engineers implement against. 2. **Guard the oracle:** before completion, diff `tests/` against your own scaffolds and flag ANY weakening by another agent (`.skip`, loosened assertion, deleted case) as a CRITICAL finding. ### Graceful Degradation At startup, check whether `frontend/` (or `paths.frontend` from config) exists. If the frontend directory is not found: - Skip all frontend-related test phases (UI E2E, visual regression, frontend contract tests, frontend-specific checks). - Print: `[DEGRADED: frontend not found — skipping frontend tests]` - Continue with all backend test phases normally. --- ## Output Structure This skill produces output in two locations: test deliverables (code, configs, fixtures) at `tests/` in the project root, and workspace artifacts (test plan, reports, findings) in `Claude-Production-Grade-Suite/qa-engineer/`. Never write test files into `services/` or `frontend/` directly. ### Project Root Output (`tests/`) ``` tests/ ├── unit/ │ └── <service>/ # One folder per backend service │ ├── handlers/ │ │ └── <handler>.test.ts # HTTP handler / controller tests │ ├── services/ │ │ └── <service>.test.ts # Business logic / domain service tests │ ├── repositories/ │ │ └── <repo>.test.ts # Data access layer tests (mocked DB) │ ├── validators/ │ │ └── <validator>.test.ts # Input validation tests │ └── mappers/ │ └── <mapper>.test.ts # DTO / domain mapper tests ├── integration/ │ ├── docker-compose.test.yml # Test dependency containers (Postgres, Redis, Kafka, etc.) │ ├── setup.ts # Global integration test setup / teardown │ └── <service>/ │ ├── db/ │ │ └── <repo>.integration.ts # Real DB queries via testcontainers │ ├── cache/ │ │ └── <cache>.integration.ts # Real Redis / cache operations │ ├── messaging/ │ │ └── <queue>.integration.ts # Real message broker publish / consume │ └── api/ │ └── <endpoint>.integration.ts # HTTP-level integration (supertest / httptest) ├── contract/ │ ├── pacts/ │ │ ├── consumer/ │ │ │ └── <consumer>-<provider>.pact.ts # Consumer-driven contract tests │ │ └── provider/ │ │ └── <provider>.verify.ts # Provider verification tests │ ├── schema/ │ │ └── <api>.schema.test.ts # OpenAPI schema validation tests │ └── pact-broker.config.ts # Pact Broker connection config ├── e2e/ │ ├── api/ │ │ ├── flows/ │ │ │ └── <user-flow>.e2e.ts # Multi-step API workflow tests │ │ ├── smoke.e2e.ts # Critical-path smoke tests │ │ └── setup.ts # API E2E auth helpers, base URLs │ └── ui/ │ ├── pages/ # Page Object Models │ │ └── <page>.page.ts │ ├── flows/ │ │ └── <user-flow>.spec.ts # Playwright / Cypress user flow specs │ ├── visual/ │ │ └── <component>.visual.ts # Visual regression snapshot tests │ └── playwright.config.ts # Or cypress.config.ts ├── performance/ │ ├── load-tests/ │ │ └── <scenario>.k6.js # k6 load test scripts (sustained load) │ ├── stress-tests/ │ │ └── <scenario>.k6.js # k6 stress test scripts (breaking point) │ ├── spike-tests/ │ │ └── <scenario>.k6.js # k6 spike test scripts (sudden burst) │ ├── baselines/ │ │ └── <scenario>.baseline.json # Expected p50/p95/p99 latency, throughput │ └── thresholds.js # Shared k6 threshold definitions ├── fixtures/ │ ├── factories/ │ │ └── <entity>.factory.ts # Test data factories (fishery / factory-girl pattern) │ ├── seed-data/ │ │ ├── <entity>.seed.json # Static seed data for integration / E2E │ │ └── seed-runner.ts # Script to load seed data into test DBs │ └── mocks/ │ ├── <external-api>.mock.ts # External API mock servers (MSW / nock) │ └── <service>.stub.ts # Internal service stubs └── coverage/ └── thresholds.json # Per-service and global coverage gates ``` ### Workspace Output (`Claude-Production-Grade-Suite/qa-engineer/`) ``` Claude-Production-Grade-Suite/qa-engineer/ ├── test-plan.md # Master test plan with traceability matrix ├── coverage-report.md # Coverage analysis and findings └── findings.md # QA findings and recommendations ``` --- ## Phases Execute each phase sequentially. Do NOT skip phases. Each phase builds on the outputs of the previous one. ### Parallel Execution Strategy After Phase 1 (Test Planning), Phases 2-6 run in parallel — each test type is independent: ```python # After test plan is written, spawn all test types simultaneously: Agent(prompt="Write unit tests following Phase 2 rules. Read test-plan.md for traceability. Write to tests/unit/.", ...) Agent(prompt="Write integration tests following Phase 3 rules. Read test-plan.md. Write to tests/integration/.", ...) Agent(prompt="Write contract tests following Phase 4 rules. Read test-plan.md. Write to tests/contract/.", ...) Agent(prompt="Write E2E tests following Phase 5 rules. Read test-plan.md. Write to tests/e2e/.", ...) Agent(prompt="Write performance tests following Phase 6 rules. Read test-plan.md. Write to tests/performance/.", ...) ``` Wait for all 5 agents to complete, then run Phase 7 (Test Infrastructure) sequentially — it needs all test files to configure CI. **Why this works:** Each test type reads source code independently and writes to its own directory. No conflicts. The test plan from Phase 1 provides shared context. **Execution order:** 1. Phase 1: Test Planning (sequential — foundational) 2. Phases 2-6: Unit + Integration + Contract + E2E + Performance (PARALLEL) 3. Phase 7: Test Infrastructure (sequential — needs all test files) 4. Phase 8: Execute & Test-Integrity (sequential — the oracle must RUN) --- ### Phase 1 — Test Planning **Goal:** Produce a traceability matrix linking every BRD acceptance criterion to concrete test cases, categorized by test type. **Inputs to read:** - BRD / PRD acceptance criteria (every GIVEN/WHEN/THEN or equivalent) - `api/` API contracts (OpenAPI specs, AsyncAPI specs) - `schemas/` data models and `docs/architecture/` sequence diagrams - `services/` service structure (list all services, handlers, repos) - `frontend/` component and page structure (if frontend exists; otherwise skip frontend inputs) **Actions:** 1. Extract every acceptance criterion and assign a unique ID (AC-001, AC-002, ...). 2. For each criterion, determine which test types are required (unit, integration, contract, e2e, performance). 3. Identify all services, modules, and components that need test coverage. 4. Identify all external dependencies that require mocking or test containers. 5. Identify critical user flows for E2E coverage. 6. Identify performance-sensitive endpoints for load testing. 7. Define coverage thresholds per service (lines, branches, functions). **Output:** Write `Claude-Production-Grade-Suite/qa-engineer/test-plan.md` with the following sections: - **Scope** — What is being tested, what is explicitly out of scope - **Test Strategy** — Test pyramid approach, which test types cover which risk areas - **Traceability Matrix** — Table mapping AC-ID to test case IDs, test type, and priority - **Environment Requirements** — Containers, external services, env vars needed - **Coverage Targets** — Per-service and global coverage gates - **Risk Register** — Areas with high complexity or insufficient testability --- ### Phase 2 — Unit Tests **Goal:** Test each service's business logic, handlers, and repositories in isolation with full mocking of external dependencies. **Inputs to read:** - `services/` source code for each service - The test plan from Phase 1 **Rules:** 1. One test file per source file. Mirror the source directory structure under `tests/unit/<service>/`. 2. Mock ALL external dependencies: databases, caches, message brokers, HTTP clients, other services. 3. Use dependency injection or module mocking — never patch globals. 4. Test the happy path, error paths, edge cases, and boundary values for every public function. 5. For handlers/controllers: test request parsing, validation error responses, correct status codes, response body shape. 6. For services/domain logic: test business rule enforcement, state transitions, calculation correctness. 7. For repositories: test query construction, parameter binding, result mapping (with mocked DB driver). 8. For validators: test every validation rule, including null, empty, boundary, and malformed inputs. 9. Every test must have a descriptive name that reads as a specification: `it("should return 404 when order does not exist for the given user")`. 10. Use factories from `tests/fixtures/factories/` for test data — never inline large object literals. 11. Assert on specific values, not just truthiness. Prefer `toEqual` over `toBeTruthy`. 12. Test error types and messages, not just that an error was thrown. **Output:** Write test files to `tests/unit/<service>/`. Also write factories to `tests/fixtures/factories/` as you discover entity shapes. --- ### Phase 3 — Integration Tests **Goal:** Test service interactions with real dependencies using testcontainers or docker-compose. **Inputs to read:** - `services/` database migrations, schemas, connection configs - `docs/architecture/` infrastructure requirements (which DBs, caches, brokers) - The test plan from Phase 1 **Rules:** 1. Write `tests/integration/docker-compose.test.yml` with containers for every real dependency (PostgreSQL, Redis, Kafka, Elasticsearch, etc.). Pin exact image versions. 2. Write `tests/integration/setup.ts` with global before/after hooks: start containers, run migrations, seed base data, tear down after suite. 3. Each integration test file connects to real containers — no mocks for the dependency under test. 4. Test actual SQL queries against a real database with realistic data volumes (not just 1 row). 5. Test cache read/write/eviction with a real Redis instance. 6. Test message publishing and consumption with a real broker. 7. Test API endpoints with real HTTP calls (supertest / httptest) against a running server. 8. Each test must clean up its own data. Use transactions with rollback, or truncate tables in afterEach. 9. Tests must be parallelizable — use unique identifiers to avoid cross-test data collisions. 10. Test failure modes: connection timeouts, constraint violations, concurrent writes, deadlocks. **Output:** Write test files to `tests/integration/<service>/`. Write `docker-compose.test.yml` and `setup.ts` to `tests/integration/`. --- ### Phase 4 — Contract Tests **Goal:** Verify API consumers and providers agree on request/response schemas and that implementations conform to OpenAPI specifications. **Inputs to read:** - `api/` OpenAPI specs and AsyncAPI specs - `services/` API route definitions, request/response DTOs - `frontend/` API client calls and expected response shapes (if frontend exists; otherwise skip consumer-side frontend contracts) **Rules:** 1. For each API consumer (frontend, other services), write a Pact consumer test that defines the expected interactions. 2. For each API provider, write a Pact provider verification test that replays consumer expectations against the real provider. 3. Write schema validation tests that load the OpenAPI spec and validate every endpoint's actual response against the schema. 4. Test backward compatibility: if there are versioned APIs, verify old consumers still work with new providers. 5. For async APIs (events, messages), write contract tests for message schemas using AsyncAPI specs. 6. Configure Pact Broker connection in `pact-broker.config.ts` (even if the broker URL is a placeholder). 7. Contract tests must fail if a required field is removed, a type changes, or a new required field is added without consumer agreement. **Output:** Write contract tests to `tests/contract/`. --- ### Phase 5 — E2E Tests **Goal:** Test critical user flows end-to-end through the full stack. **Inputs to read:** - BRD / PRD user stories and acceptance criteria (especially the critical path)