design-testing-strategy

name	design-testing-strategy
description	Use before writing any type of tests. Distills 14 industry sources into deterministic decision gates, schemas, and worked test examples.

Design Testing Strategy

A reference manual for designing a fit-for-purpose, fit-for-criticality testing strategy.

This skill is decision-oriented, not philosophical: every gate is deterministic (ON when X / OFF when Y), every schema is enforced (field ordering matters), every example is worked end-to-end.

How To Use This Skill

1. Read Decision Gates in order (Gate 0 -> Gate 6). Each gate is independent — you may finish with any subset of test types ON.
2. Apply Strategic Skip Heuristics to remove ON gates that would yield low ROI for this artifact.
3. For each ON gate, fill the Test Matrix Schema (selected_types entry) — the field order is load-bearing.
4. List rejected types in rejected_types and deliberate skips in deliberately_skipped.
5. Produce a Test Cases to Cover markdown bullet list using ISTQB techniques from Case Design Techniques.
6. Cross-check against the matching Worked Example (A pure function / B HTTP+DB endpoint / C UI component).

---

Decision Gates

Apply gates in numeric order. Each gate produces an independent boolean (applies: true|false). Gates do NOT veto each other — a single artifact may have unit + integration + contract + property-based all ON.

#	Type	ON when	OFF when	Source
0	Skip All	Criticality is NONE (docs-only, comments, formatting, generated code, config without logic, throwaway prototypes)	Anything with branching, computed output, side effects, or user-visible behavior	Pragmatic Programmer — "Test ruthlessly and effectively" implies effective skipping when ROI is zero
1	Unit	Code contains any logic: branches, loops, conditionals, computation, transformation, parsing, validation, formatting	Pure declarative wiring (DI registration, route table) with no behavior	Test Pyramid (Vocke) base layer + Beck TDD Red-Green-Refactor unit
2	Integration	Boundary crossing: HTTP call, DB query, external SDK, message queue, filesystem I/O, OR collaboration with >=2 distinct collaborators where unit doubles distort behavior	Pure function with no I/O and 0-1 stable collaborators	Testing Trophy (Dodds) — integration is the highest-ROI layer; Google "Follow the User"
3	Component or E2E	UI surface AND criticality >= MEDIUM-HIGH AND user-facing critical path (signup, checkout, auth, payment, primary CTA)	Internal admin-only screens, dev tooling, or non-critical UI	Test Pyramid top + ISO/IEC/IEEE 29119 risk ranking + Google e2e principles
4	Contract	Public API consumed by >=1 distinct clients (mobile + web, multiple internal services, external partners) AND independent deploy cadence	API where consumer and provider deploy together	Pact / CDC + Pactflow CDC explainer
5	Smoke	Deployable surface (web app, API, service) AND a deploy/CI pipeline exists where post-deploy validation is meaningful	Library, internal helper, or no deploy pipeline	Google "What Makes a Good End-to-End Test" — smoke = minimal e2e for deploy gate
6	Property-Based	Input domain is large or unbounded (numeric ranges, strings, lists, parsers, serializers, encoders, math) AND invariants are stable (round-trip, idempotency, monotonicity, commutativity) AND criticality >= MEDIUM-HIGH	Small finite input domain, unstable invariants, or LOW criticality	Hypothesis / QuickCheck

Gate Application Algorithm

for gate in [Gate 0, Gate 1, ..., Gate 6]:
    if gate.ON_condition_met(artifact):
        result[gate.type] = applies: true
    else:
        result[gate.type] = applies: false

if Gate 0 is true:
    short-circuit: emit empty selected_types, document criticality=NONE, stop

Criticality Scale (used by Gates 3 and 6):

Level	Definition
NONE	Docs, formatting, generated code, throwaway code, configs without logic
LOW	Internal dev tooling, admin-only screens, logging formatters
MEDIUM	Standard CRUD, internal APIs with a single team consumer, non-critical UI, helpers and utilities
MEDIUM-HIGH	User-facing UI on critical paths, public APIs with multiple consumers, business workflows
HIGH	Money movement, auth/authz decisions, security-critical validation, data integrity, regulated domains

---

Test Type Reference

Type	Use when	Do NOT use when	Frameworks	Typical dependencies	Google Size
unit	Pure logic, single function/method/class, deterministic inputs	Code is just I/O orchestration with no logic	vitest, jest, pytest, go test, JUnit, xUnit, RSpec	None (or in-memory fakes)	Small
integration	Boundary crossing (DB, HTTP, queue, FS); multiple collaborators where mocking distorts behavior	Pure function with no boundary	vitest, jest, pytest, go test, JUnit + Testcontainers, supertest, TestRestTemplate	Real Postgres/Redis/Kafka via Testcontainers, in-process HTTP server, real FS in tmpdir	Medium (single machine, localhost OK)
component	UI rendering + interaction within a single component, no full app context	Backend-only logic; multi-page user flow	React Testing Library, Vue Test Utils, Angular TestBed, Storybook interaction tests	jsdom or happy-dom, mocked network at fetch/axios level	Small to Medium
e2e	Full user path through running app: real browser, real backend, real DB	Internal helper, single component, non-critical UI	Playwright, Cypress, Selenium	Real running app + Testcontainers-backed DB or seeded staging	Large (multi-process, possibly multi-machine)
smoke	Post-deploy go/no-go: hit / health, key endpoints respond, login works	Detailed correctness; smoke is shallow by design	Playwright (1-3 critical paths), HTTP probe scripts, k6 minimal scenarios	Real deployed environment	Large
contract	Public API consumed by 2+ distinct clients with independent deploy cadence	Single-consumer internal API; provider and consumer deploy together	Pact, Spring Cloud Contract, OpenAPI schema validators	Pact broker or contract files in repo	Medium
property-based	Large/unbounded input domain with stable invariants (parser, serializer, encoder, math)	Small finite input space; unstable invariants	Hypothesis (Python), fast-check (TS), QuickCheck (Haskell), jqwik (Java), proptest (Rust)	Same as unit	Small

Google Test Size Mapping

Google Test Sizes (Bland) and SWE at Google Ch.11 classify tests by resources (size), independent of scope (paths covered):

Size	Process model	Network	Filesystem	Time budget	Notes
small	Single process, single thread	None	None (in-memory only)	< 100ms	Fast, hermetic, parallelizable
medium	Single machine, multiple processes allowed	localhost only	tmpdir allowed	< 1s	Testcontainers fits here
large	Multi-machine	External network allowed	Persistent FS allowed	< 15min	Full e2e
enormous	Distributed	Wide network	Anywhere	longer	Cluster / chaos

A test's type (unit/integration/e2e) and size (small/medium/large) are orthogonal: a small integration test (Testcontainers Postgres in same process via JDBC) is legitimate.

Playwright vs Cypress (UI e2e)

Dimension	Playwright	Cypress
Browsers	Chromium, Firefox, WebKit	Chromium, Firefox, WebKit (limited)
Multi-tab / multi-origin	Yes	Limited
Parallelism	Built-in shards	Paid dashboard or external
Network interception	Robust route-level	cy.intercept
Default	Choose Playwright for new projects unless team already standardized on Cypress	Choose Cypress when team has heavy investment

---

Case Design Techniques

Use ISTQB Foundation Level black-box techniques to derive what to test inside each chosen test type. References: ISTQB BVA white paper, ASTQB black-box techniques.

1. Equivalence Partitioning (EP)

Divide input domain into partitions where the system is expected to behave the same way; ONE test per partition is sufficient.

Worked example — discount(orderTotal: number) -> number:

Partition	Range	Representative test input	Expected
Below threshold	0 <= total < 100	50	0% discount
Mid tier	100 <= total < 500	250	5% discount
Top tier	total >= 500	1000	10% discount
Invalid (negative)	total < 0	-1	throw / error

Four tests cover all partitions. EP alone misses boundaries — combine with BVA.

2. Boundary Value Analysis (BVA)

Bugs cluster at boundaries. For every boundary value B, test B-1, B, B+1 (or for floats, the smallest representable step).

Worked example — same discount function, boundary at 100:

Test input	Why	Expected
99 (= B-1)	Last value of "below threshold" partition	0% discount
100 (= B)	First value of "mid tier" partition	5% discount
101 (= B+1)	Confirms not off-by-two	5% discount

Repeat for boundary at 500: test 499, 500, 501. Total: 6 boundary tests + 4 EP tests = 10 cases.

The B-1 / B / B+1 triplet has the same shape across boundaries (vary input, vary expected output, identical assertion); this is a natural fit for a table-driven test (see sub-section 5 below).

3. Decision Tables

When output depends on combinations of conditions. Each column is a rule.

Worked example — canCheckout(cartHasItems, paymentValid, addressOnFile):

Condition / Rule	R1	R2	R3	R4
cartHasItems	T	T	T	F
paymentValid	T	T	F	*
addressOnFile	T	F	*	*
Result	allow	block:address	block:payment	block:cart

Four tests, one per rule (* = don't care, dropped via merging).

4. State Transition

When behavior depends on history. Identify states, events, and forbidden transitions.

Worked example — Order state machine with states {draft, submitted, paid, shipped, cancelled}:

From	Event	To	Test
draft	submit	submitted	happy path
submitted	pay	paid	happy path
paid	ship	shipped	happy path
draft	cancel	cancelled	early cancel
paid	cancel	reject	forbidden — refund flow required, NOT direct cancel
shipped	submit	reject	forbidden

Cover one test per legal transition + one per forbidden transition (negative path).

5. Table-Driven Tests

When EP, BVA, or decision-table analysis yields 3+ cases with the same shape (same setup, same assertion, only inputs and expected outputs differ — e.g., parsing valid/invalid date formats; computing tax across brackets; routing rules) collapse them into a single table-driven test. The cases become rows in a data table; the test body iterates the rows and runs one assertion per row. References: Dave Cheney, Prefer table-driven tests; Go wiki: TableDrivenTests.

Do NOT force a table when setup, framework calls, or the assertion shape varies substantially across cases. Forced uniformity hides real differences behind a single name and produces obscure failure messages — keep those as separate, individually named tests.

Worked example — six EP+BVA cases for discount(orderTotal) (boundary at 100) collapsed into one table-driven unit test (TS / vitest syntax; the same pattern applies to Go t.Run, JUnit @ParameterizedTest, pytest parametrize):

describe("discount", () => {
  const cases: Array<{ name: string; input: number; expected: number }> = [
    { name: "EP: below threshold (typical)", input: 50,  expected: 0    },
    { name: "BVA: B-1 at boundary 100",      input: 99,  expected: 0    },
    { name: "BVA: B at boundary 100",        input: 100, expected: 0.05 },
    { name: "BVA: B+1 at boundary 100",      input: 101, expected: 0.05 },
    { name: "EP: mid tier (typical)",        input: 250, expected: 0.05 },
    { name: "EP: top tier (typical)",        input: 1000, expected: 0.10 },
  ];

  for (const c of cases) {
    it(c.name, () => {
      expect(discount(c.input)).toBe(c.expected);
    });
  }
});

The name column is mandatory: each row must produce an individually addressable test so failures point to the specific case, not "row 3 of 6". Rows that need a different assertion (e.g., the negative-input case throws) stay as separate tests outside the table.

---

Dependency Decision

For Gate 2 (Integration) and Gate 3 (Component/E2E), choose dependencies deliberately. The goal is maximum realism that still runs deterministically in CI.

Dependency style	Use when	Avoid when	Notes
Real infra via Testcontainers	DB/Redis/Kafka/Browser, dev needs real driver behavior, hermetic CI required	Cold-start budget < 1s, no Docker available	Default for integration tests on Postgres / Redis / Kafka / Localstack
In-memory fake	Owned interface, semantics are simple (key-value, list), test speed critical	Fake diverges from real — silent bugs at integration boundary	Acceptable for repository ports in hexagonal architectures, IF the port has its own contract test against real infra
Mock (test double)	Single collaborator with pure interface; test focuses on protocol (was X called with Y)	You're mocking >2 collaborators or mocking data structures (anti-pattern: incomplete mocks)	Mocks are tools to isolate, not things to test
Stubbed HTTP	Calling external SaaS where Testcontainers / Localstack option doesn't exist	When Pact / CDC is needed (use contract tests instead)	nock (Node), responses (Python), WireMock (JVM)
Real external service	Smoke test in staging only	Unit / integration / CI — always non-deterministic	Reserve for smoke tests against staging

Tradeoff summary: Testcontainers > in-memory fake > mock, but cost goes the same direction. Pick the cheapest level that doesn't lie about the boundary's behavior.

---

Strategic Skip Heuristics

Explicit "don't bother" rules. Skipping these is not laziness — it is risk-adjusted ROI per ISO/IEC/IEEE 29119 risk-based testing and Risk-Based Testing.

Skip	Rule
No e2e for internal helpers	If artifact has no UI surface and no user-facing path, skip e2e. Unit + integration is sufficient.
No contract test for bound by deploy consumer API	If only one client consumes the API and they deploy together, contract testing adds maintenance with no decoupling benefit.
No property-based on small finite domains	If input space is enum {A, B, C}, EP + BVA already covers it; property-based adds infra without finding more bugs.
No integration test for pure functions	Adding a Postgres container to test a formatCurrency helper is waste. Unit only.
No component test for static markup	If the component has no state, no events, no conditional rendering, a snapshot is enough — or skip entirely.
No unit test for declarative wiring	DI bindings, route registration, schema declarations: assert at integration level (does the route serve the right handler) instead.
No e2e for things integration covers reliably	Per Google e2e principles: the smaller the test you can use to cover a behavior, the better. e2e is the exception, not the default.
No tests for spike/throwaway code	Per Beck TDD: if the artifact will be deleted within hours, document the exception with the human partner. Then write tests on the kept version.
No "and" tests	If a test name contains "and", split it into separate tests (one assertion per behavior).

---

Test Matrix Schema

Every test strategy MUST be expressed as the YAML block below. Field ordering inside each list entry is load-bearing — judges and downstream tools parse the first key as the critical one (rationale / reason / why), and the second key as the categorical one (type / what).

Schema

test_strategy:
  artifact: "<path or short identifier>"
  rationale: "Why this test strategy is being applied to this artifact (specific, evidence-based)"
  criticality: "NONE | LOW | MEDIUM | MEDIUM-HIGH | HIGH"

  selected_types:
    - rationale: "Why this type is being applied to this artifact (specific, evidence-based)"
      type: "unit | integration | component | e2e | smoke | contract | property-based"
      size: "small | medium | large | enormous"
      framework: "vitest | jest | pytest | go test | JUnit | playwright | cypress | pact | hypothesis | ..."
      dependencies:
        - "List of dependencies: real Postgres via Testcontainers, in-memory fake, mocked HTTP via nock, etc."
      gate: "Gate N (the gate that triggered this selection)"

  rejected_types:
    - reason: "Why this type does NOT apply to this artifact (cite Strategic Skip Heuristic or gate that did not trigger)"
      type: "unit | integration | component | e2e | smoke | contract | property-based"

  deliberately_skipped:
    - why: "Cost / risk justification for skipping despite a partial signal"
      what: "A specific category of test cases being skipped (e.g., 'browser compatibility on IE11', 'load testing beyond 100 RPS')"

Worked YAML Example

test_strategy:
  artifact: "POST /users (user registration endpoint)"
  rationale: "User registration is a critical user-facing path; can be used by web and mobile apps independently of each other."
  criticality: "MEDIUM-HIGH"

  selected_types:
    - rationale: "Endpoint contains validation logic (email format, password rules, uniqueness) — Gate 1 ON for branch coverage"
      type: "unit"
      size: "small"
      framework: "vitest"
      dependencies: ["in-memory user repository fake"]
      gate: "Gate 1"
    - rationale: "Endpoint writes to Postgres and emits user.created event to Kafka — Gate 2 ON, real boundary behavior matters"
      type: "integration"
      size: "medium"
      framework: "vitest + supertest + Testcontainers"
      dependencies: ["Postgres 15 via Testcontainers", "Kafka via Testcontainers"]
      gate: "Gate 2"
    - rationale: "Consumed by mobile app and web app on independent deploy cadences — Gate 4 ON, prevents drift"
      type: "contract"
      size: "medium"
      framework: "Pact"
      dependencies: ["Pact broker"]
      gate: "Gate 4"

  rejected_types:
    - reason: "No UI surface in this artifact — Gate 3 OFF"
      type: "component"
    - reason: "No UI surface — Gate 3 OFF; e2e covered by web/mobile apps separately"
      type: "e2e"
    - reason: "Input domain (email, password) is large but invariants are well-covered by EP+BVA at unit level — property-based ROI is low at MEDIUM-HIGH criticality, only triggers Gate 6 partially"
      type: "property-based"

  deliberately_skipped:
    - why: "Project does not have post-deploy probe pipeline yet; smoke would be no-op"
      what: "Smoke test for /users after deploy"
    - why: "Non-functional load testing is out of scope for this task; tracked separately in performance backlog"
      what: "Load test verifying p99 < 200ms at 1000 RPS"

Field ordering checklist (judges check this verbatim):

• test_strategy: artifact BEFORE rationale BEFORE criticality.
• selected_types[*]: rationale BEFORE type BEFORE size BEFORE framework BEFORE dependencies BEFORE gate.
• rejected_types[*]: reason BEFORE type.
• deliberately_skipped[*]: why BEFORE what.

---

Case Listing Schema

After the matrix, produce a flat markdown bullet list of test cases to be implemented. This is separate from the YAML matrix because:

• a. it lists what to test, not how
• b. it links back to acceptance criteria

Format

## Test Cases to Cover

### AC-N: [criterion title]
- [type] description 
- [type] description 

### AC-N: [criterion title]
- [type] description 
- [type] description 

Where:

• type matches one of selected_types[*].type from the matrix
• description follows AAA / Given-When-Then (Dan North BDD) shape — see Bill Wake AAA (2001)
• AC-N references the acceptance criterion the case verifies (omit if non-AC-bound, e.g., infrastructure smoke)

Worked Example

## Test Cases to Cover

### AC-1: Discount returns the correct percentage based on the total
- [unit] discount returns 0% when total = 0 [EP partition: below threshold]
- [unit] discount returns 0% when total = 99 [BVA: B-1 at boundary 100]
- [unit] discount returns 5% when total = 100 [BVA: B at boundary 100]
- [unit] discount returns 5% when total = 101 [BVA: B+1 at boundary 100]

### AC-2: Discount fails when total is invalid
- [unit] discount throws when total = -1 [EP partition: invalid]

### AC-3: /orders saves the order to the database
- [integration] POST /orders persists order to Postgres and returns 201 with order id

### AC-4: /orders rejects duplicate idempotency key
- [integration] POST /orders rejects duplicate idempotency key with 409

### AC-5: /orders/:id returns order by id
- [contract] GET /orders/:id returns schema matching mobile-app pact

---

Sources & Further Reading

These 14 sources back every gate and rule above. When in doubt, consult the source linked at that gate.

1. Test Pyramid — Mike Cohn (2009, Succeeding with Agile) + Ham Vocke, The Practical Test Pyramid, martinfowler.com.
2. Testing Trophy — Kent C. Dodds (2018), The Testing Trophy and Testing Classifications and Write Tests.
3. Google Test Sizes — Mike Bland (2011), Small / Medium / Large; Software Engineering at Google Ch.11; Test Sizes (Google Testing Blog).
4. Google Testing on the Toilet — What Makes a Good End-to-End Test, Testing UI Logic - Follow the User, Origins (Mike Bland).
5. ISTQB Foundation Level — Black-box techniques: Boundary Value Analysis white paper; ASTQB Black-Box Techniques.
6. ISO/IEC/IEEE 29119 — Risk-based test process standard. Wikipedia overview.
7. Kent Beck — Test Driven Development: By Example (Addison-Wesley, 2002). Publisher page. ISBN 978-0321146533.
8. The Pragmatic Programmer (20th Anniversary Edition) — Hunt & Thomas (2019). pragprog.com.
9. AAA pattern — Bill Wake (2001), 3A — Arrange, Act, Assert. Given-When-Then — Dan North, Introducing BDD.
10. Property-based testing — Hypothesis: What is property-based testing?; QuickCheck (Haskell), fast-check (TS).
11. Contract testing / Consumer-Driven Contracts — Pact docs; Pactflow CDC explainer.
12. Testcontainers — testcontainers.com.
13. Table-driven tests — Dave Cheney, Prefer table-driven tests; Go wiki: TableDrivenTests.
14. Risk-based testing — Risk Management During Test Planning (softwaretestinghelp.com).

---

Worked Examples

Each example shows:

• a. the artifact and acceptance criteria
• b. gate-by-gate walkthrough
• c. test_strategy YAML following the schema
• d. Test Cases to Cover list
• e. commentary on rejected types

---

Example A — Pure Helper Function: formatCurrency(amount: number, code: string): string

Artifact

function formatCurrency(amount: number, code: string): string;
// e.g. formatCurrency(1234.5, "USD") -> "$1,234.50"
//      formatCurrency(1234.5, "EUR") -> "€1.234,50"

Acceptance criteria:

• AC-1: USD output uses $ prefix, comma thousands, period decimal, two decimal places.
• AC-2: EUR output uses € prefix, period thousands, comma decimal, two decimal places.
• AC-3: Throws Error("Unknown currency code") for unsupported codes.
• AC-4: amount = 0 formats as "$0.00" / "€0,00".

Criticality: LOW (helper used in display only, no money movement here).

Gate Walkthrough

Gate	Decision	Reason
0 Skip	OFF	Has logic
1 Unit	ON	Pure logic with branches per currency code — Test Pyramid base
2 Integration	OFF	No I/O, no boundary — Skip Heuristic: no integration for pure functions
3 Component/E2E	OFF	No UI surface
4 Contract	OFF	Not a public API
5 Smoke	OFF	Not deployable
6 Property-Based	ON (partial)	Numeric input is unbounded, but invariants exist (round-trip via parse, monotonicity in amount) — Hypothesis. Promote at MEDIUM-HIGH; here LOW criticality means we apply it sparingly (1-2 properties)

test_strategy YAML

test_strategy:
  artifact: "src/util/formatCurrency.ts"
  rationale: "Pure helper function used in display only; no money movement here."
  criticality: "LOW"

  selected_types:
    - rationale: "Pure logic with currency-specific branches and number formatting; EP+BVA on amount, decision table on currency code"
      type: "unit"
      size: "small"
      framework: "vitest"
      dependencies: []
      gate: "Gate 1"
    - rationale: "Amount domain is unbounded floats; invariant 'parseCurrency(formatCurrency(x, c)) ~= x' is stable; sparingly applied (1-2 properties) at LOW criticality"
      type: "property-based"
      size: "small"
      framework: "fast-check"
      dependencies: []
      gate: "Gate 6"

  rejected_types:
    - reason: "No I/O, no boundary, no collaborators - Gate 2 OFF"
      type: "integration"
    - reason: "No UI surface - Gate 3 OFF"
      type: "component"
    - reason: "No UI surface - Gate 3 OFF"
      type: "e2e"
    - reason: "Internal helper, not consumed across deploys - Gate 4 OFF"
      type: "contract"
    - reason: "Library helper, no deploy pipeline target - Gate 5 OFF"
      type: "smoke"

  deliberately_skipped:
    - why: "Locale list is finite (USD, EUR); exhaustive enumeration via decision table is sufficient and more maintainable than i18n property tests"
      what: "Property-based fuzzing of currency code beyond known list"

Test Cases to Cover

### AC-1: USD output uses `$` prefix, comma thousands, period decimal, two decimal places.
- [unit] formatCurrency(1234.5, "USD") returns "$1,234.50" [EP: typical USD]
- [unit] formatCurrency(0.01, "USD") returns "$0.01" [BVA: B+1 smallest non-zero]
- [unit] formatCurrency(-0.01, "USD") returns "-$0.01" [BVA: B-1 negative side]

### AC-2: EUR output uses `€` prefix, period thousands, comma decimal, two decimal places.
- [unit] formatCurrency(1234.5, "EUR") returns "€1.234,50" [EP: typical EUR]
- [property-based] for any non-NaN finite x in [-1e9, 1e9] and code in {USD, EUR}: parseCurrency(formatCurrency(x, code)) is within 0.005 of x [round-trip invariant]

### AC-3: Throws `Error("Unknown currency code")` for unsupported codes.
- [unit] formatCurrency(1, "XYZ") throws Error("Unknown currency code") [Decision table: unknown code]

### AC-4: `amount = 0` formats as `"$0.00"` / `"€0,00"`.
- [unit] formatCurrency(0, "USD") returns "$0.00" [BVA: B at amount=0]
- [unit] formatCurrency(0, "EUR") returns "€0,00" [BVA: B at amount=0 for EUR]

Why types were rejected: Helper has no boundaries (no integration), no UI (no component/e2e), is internal and library-style (no contract/smoke), and at LOW criticality the cost of additional test types far exceeds the benefit.

---

Example B — HTTP POST Endpoint with DB and Multi-Consumer: POST /users

Artifact

A user-registration endpoint that:

1. Validates request body (email format, password complexity, age >= 13).
2. Checks email uniqueness against Postgres.
3. Inserts user record (transactional).
4. Emits user.created event to Kafka.
5. Returns 201 with {id, email, createdAt}.
6. Returns 400 for invalid input, 409 for duplicate email.

Consumed by: mobile app (iOS/Android) and web app on independent deploy cadences.

Acceptance criteria:

• AC-1: Valid request returns 201 and persists user.
• AC-2: Invalid email format returns 400 with field-level error.
• AC-3: Password not meeting policy returns 400.
• AC-4: Duplicate email returns 409.
• AC-5: Successful registration emits exactly one user.created event.
• AC-6: Response schema is stable for mobile + web consumers.

Criticality: MEDIUM-HIGH (auth surface, identity domain, multi-consumer public API).

Gate Walkthrough

Gate	Decision	Reason
0 Skip	OFF	Has substantial logic
1 Unit	ON	Validators (email, password, age) are pure logic — Test Pyramid base
2 Integration	ON	Boundary crossing: HTTP, Postgres, Kafka — Testing Trophy ROI sweet spot
3 Component/E2E	OFF (here)	No UI in this artifact; UI lives in mobile + web repos and tests itself
4 Contract	ON	Two distinct consumers (mobile + web) on independent deploy cadences — Pact CDC
5 Smoke	ON	Deployable HTTP service; post-deploy probe of /users registration is meaningful — Google e2e
6 Property-Based	OFF	Input domain (email, password, age) is constrained and well-covered by EP+BVA at unit; criticality is MEDIUM-HIGH but Gate 6 OFF on bounded inputs — Skip Heuristic

test_strategy YAML

test_strategy:
  artifact: "POST /users (user registration endpoint)"
  rationale: "User registration is a critical user-facing path; can be used by web and mobile apps independently of each other."
  criticality: "MEDIUM-HIGH"

  selected_types:
    - rationale: "Validators (email, password, age) are pure logic; EP+BVA on each field; one test per partition"
      type: "unit"
      size: "small"
      framework: "vitest"
      dependencies: ["in-memory user repository fake (for service-level unit if needed)"]
      gate: "Gate 1"
    - rationale: "Endpoint writes to Postgres and emits to Kafka; mocking these distorts transactional and ordering behavior - Testcontainers gives real boundary fidelity"
      type: "integration"
      size: "medium"
      framework: "vitest + supertest + Testcontainers"
      dependencies: ["Postgres 15 via Testcontainers", "Kafka via Testcontainers"]
      gate: "Gate 2"
    - rationale: "Public API consumed by mobile + web on independent deploy cadences; contract testing prevents schema drift breaking either consumer"
      type: "contract"
      size: "medium"
      framework: "Pact (provider verification)"
      dependencies: ["Pact broker", "consumer-published pacts from mobile and web"]
      gate: "Gate 4"
    - rationale: "Deployable HTTP service with a post-deploy pipeline; one minimal smoke verifies /users responds 201 in the deployed environment"
      type: "smoke"
      size: "large"
      framework: "Playwright (1 critical path)"
      dependencies: ["deployed environment URL", "test account seeding"]
      gate: "Gate 5"

  rejected_types:
    - reason: "No UI surface in this artifact - Gate 3 OFF; mobile and web repos own their own component tests"
      type: "component"
    - reason: "No UI surface - Gate 3 OFF; consumer e2e lives in mobile/web repos"
      type: "e2e"
    - reason: "Input domain is bounded and EP+BVA at unit level covers it; property-based on this glue endpoint adds infra without finding more bugs - Gate 6 OFF"
      type: "property-based"

  deliberately_skipped:
    - why: "Performance/load testing is out of scope here; tracked in dedicated performance backlog"
      what: "Load test verifying p99 < 200ms at 1000 RPS"
    - why: "Cross-region failover is owned by infrastructure team, not this endpoint"
      what: "Multi-region availability test"

Test Cases to Cover

### AC-1: Valid request returns `201` and persists user.
- [unit] validateEmail accepts "[e-mail]" [EP: well-formed]
- [integration] POST /users with valid body returns 201 and persists row in Postgres
- [smoke] POST /users in deployed environment returns 201 for a synthetic test account

### AC-2: Invalid email format returns `400` with field-level error.
- [unit] validateEmail rejects "alice@" [EP: missing domain]
- [unit] validateEmail rejects "" [BVA: empty boundary]
- [integration] POST /users with invalid email returns 400 and does NOT persist

### AC-3: Password not meeting policy returns `400`.
- [unit] validatePassword rejects 7-char password [BVA: B-1 at min length 8]
- [unit] validatePassword accepts 8-char password meeting policy [BVA: B at min length]
- [unit] validatePassword accepts 9-char password [BVA: B+1]
- [unit] validateAge rejects 12 [BVA: B-1 at boundary 13]
- [unit] validateAge accepts 13 [BVA: B at boundary 13]

### AC-4: Duplicate email returns `409`.
- [integration] POST /users with duplicate email returns 409 and does NOT emit event

### AC-5: Successful registration emits exactly one `user.created` event.
- [integration] POST /users emits exactly one user.created event to Kafka on success
- [integration] POST /users transaction rolls back when Kafka publish fails [State Transition: failure path]

### AC-6: Response schema is stable for mobile + web consumers.
- [contract] Provider satisfies mobile pact: POST /users response shape matches mobile contract
- [contract] Provider satisfies web pact: POST /users response shape matches web contract

Why types were rejected: No UI surface (component/e2e belong to consumer apps), bounded input space (property-based ROI low), out-of-scope concerns (load, multi-region) deliberately skipped with rationale.

---

Example C — UI Form Component: <RegistrationForm /> (web)

Artifact

A React form component:

1. Fields: email, password, confirmPassword, age.
2. Client-side validation: email format, password >= 8 chars with mixed case + digit, passwords match, age >= 13.
3. Submits to POST /users.
4. Shows inline field errors and submit-level errors (network, 409 duplicate).
5. Disables submit button while pending; re-enables on response.
6. WCAG 2.1 AA: labels bound to inputs, errors announced via aria-live, focus moves to first error on validation failure.

Acceptance criteria:

• AC-1: User can submit a valid form and is navigated to /welcome.
• AC-2: Invalid email shows inline "Enter a valid email".
• AC-3: Mismatched passwords show inline "Passwords must match".
• AC-4: Submit is disabled while request is in flight.
• AC-5: 409 response from server shows "This email is already registered" at form level.
• AC-6: Form is keyboard navigable; focus moves to first error on validation failure.
• AC-7: All inputs have programmatic labels; errors are announced via aria-live="polite".

Criticality: MEDIUM-HIGH (registration is a critical user-facing path; accessibility is regulated in many jurisdictions).

Gate Walkthrough

Gate	Decision	Reason
0 Skip	OFF	Behavior + accessibility logic
1 Unit	ON	Validation helpers (validateEmail, passwordsMatch, parseAge) are pure logic
2 Integration	OFF (here)	The component itself does not cross a real boundary; network is mocked at fetch level. Network integration is owned by POST /users (Example B)
3 Component/E2E	ON (component) + ON (e2e for the registration path)	UI surface, criticality MEDIUM-HIGH, user-facing critical path — Test Pyramid top + Follow the User
4 Contract	OFF	UI consumes API; provider-side contract tests live in Example B
5 Smoke	ON	Web app is deployed; smoke for "registration page renders and submits" is meaningful
6 Property-Based	OFF	Bounded form inputs; EP+BVA covers them

test_strategy YAML

test_strategy:
  artifact: "src/components/RegistrationForm.tsx"
  rationale: "React form component used in web app; registration is a business-critical user-facing path."
  criticality: "MEDIUM-HIGH"

  selected_types:
    - rationale: "Validation helpers (validateEmail, passwordsMatch, parseAge) are pure logic; EP+BVA per field"
      type: "unit"
      size: "small"
      framework: "vitest"
      dependencies: []
      gate: "Gate 1"
    - rationale: "UI rendering + interaction within a single component; network mocked at fetch level - tests focus on user-facing behavior per Follow the User"
      type: "component"
      size: "small"
      framework: "vitest + React Testing Library"
      dependencies: ["happy-dom", "msw (mock service worker) for fetch"]
      gate: "Gate 3"
    - rationale: "Registration is a critical user-facing path; one e2e covers the full happy path with real backend (Testcontainers-backed)"
      type: "e2e"
      size: "large"
      framework: "Playwright"
      dependencies: ["app server running locally", "Postgres via Testcontainers", "Kafka via Testcontainers"]
      gate: "Gate 3"
    - rationale: "Web app deploys to staging/prod; smoke verifies /register page loads and form submits in deployed env"
      type: "smoke"
      size: "large"
      framework: "Playwright (1 critical path)"
      dependencies: ["deployed environment URL", "test account seeding"]
      gate: "Gate 5"

  rejected_types:
    - reason: "Component does not own a real boundary; network integration is owned by POST /users (provider) - Gate 2 OFF for this artifact"
      type: "integration"
    - reason: "UI consumes the API; provider contract tests live with the provider (POST /users) - Gate 4 OFF for the consumer"
      type: "contract"
    - reason: "Bounded input space; EP+BVA at unit level is sufficient - Gate 6 OFF"
      type: "property-based"

  deliberately_skipped:
    - why: "Cross-browser e2e on legacy browsers (IE11) is out of support per project browser matrix"
      what: "Browser compatibility e2e on IE11 / Edge Legacy"
    - why: "Visual regression (pixel diff) is owned by a separate Storybook chromatic pipeline"
      what: "Pixel-level visual regression assertions"

Test Cases to Cover

### AC-1: User can submit a valid form and is navigated to `/welcome`.
- [unit] validateEmail accepts "[e-mail]" [EP: well-formed]
- [unit] parseAge rejects 12 [BVA: B-1 at boundary 13]
- [unit] parseAge accepts 13 [BVA: B at boundary 13]
- [e2e] user fills valid form, submits, and lands on /welcome page
- [smoke] /register page loads and form submits in deployed environment

### AC-2: Invalid email shows inline `"Enter a valid email"`.
- [unit] validateEmail rejects "" [BVA: empty boundary]
- [unit] validateEmail rejects "alice@" [EP: missing domain]
- [component] entering invalid email and blurring shows "Enter a valid email" inline

### AC-3: Mismatched passwords show inline `"Passwords must match"`.
- [unit] passwordsMatch returns true when both equal "Abcd1234"
- [unit] passwordsMatch returns false when one is "" [BVA: empty]
- [component] entering mismatched passwords shows "Passwords must match" inline

### AC-4: Submit is disabled while request is in flight.
- [component] submit is disabled when password and confirmPassword differ
- [component] submit click disables button while request is pending [State Transition: idle -> pending]

### AC-5: 409 response from server shows `"This email is already registered"` at form level.
- [component] 409 response shows form-level "This email is already registered"

### AC-6: Form is keyboard navigable; focus moves to first error on validation failure.
- [component] validation failure moves focus to first error field [a11y]

### AC-7: All inputs have programmatic labels; errors are announced via `aria-live="polite"`.
- [component] form renders email, password, confirmPassword, age, submit [happy path render]
- [component] all inputs have programmatic labels and errors live in aria-live="polite" region [a11y]

Why types were rejected: This artifact is a UI consumer — its real boundary is the API, which is tested as integration in Example B (provider side). Property-based testing is not justified for bounded UI input handling. Cross-browser legacy and visual-regression are out of scope and explicitly skipped with rationale.

---

Skill Self-Check

Before declaring a strategy complete, the loading verify:

• All 7 gates evaluated explicitly (ON/OFF + reason).
• selected_types[*] order is rationale -> type -> size -> framework -> dependencies -> gate.
• rejected_types[*] order is reason -> type.
• deliberately_skipped[*] order is why -> what.
• Each AC is referenced by at least one test case.
• BVA cases enumerate B-1, B, B+1 for each numeric boundary.
• Test sizes (small/medium/large) are assigned per Google Test Sizes.
• Test names contain no "and" (per Skip Heuristic).
• At least one Strategic Skip Heuristic was applied or explicitly considered and overridden with rationale.

If any check fails, revise the strategy before delivering.

Coverage Analysis

Mutation testing and other coverage-analysis methods (used after tests are written to assess test-suite quality) are documented in the companion test-coverage skill.