testing

name	testing
description	Comprehensive testing guidance covering the full lifecycle: before/during implementation (TDD, ATDD, example mapping, property-based testing, contract testing) and after implementation (unit, integration, regression, mutation, snapshot, performance, fuzz, smoke, characterization testing). Use when: writing tests first (Red-Green-Refactor), designing test strategy, performing mutation testing, writing integration/regression tests, using Swift Testing (@Test, #expect, parameterized tests), creating mocks/stubs/fakes, refactoring legacy code with tests, doing property-based testing, snapshot testing, performance benchmarking, fuzz testing, or diagnosing test smells and flaky tests. Not for concurrency testing patterns (use the swift skill concurrency domain).
allowed-tools	["Read","Glob","Grep"]

Testing — Full Lifecycle

Overview

This skill covers testing across the entire development lifecycle, organized into two phases:

1. Before & During Implementation — techniques that drive design and catch issues early (TDD, ATDD, example mapping, property-based testing, contract testing, characterization testing)
2. After Implementation — verification techniques that validate correctness and guard against regressions (unit, integration, regression, mutation, snapshot, performance, fuzz, smoke testing)

Decision Tree

Phase 1: Before & During Implementation

Starting a new feature

1. Example Mapping — discover test cases collaboratively → references/test-strategy.md (Example Mapping section)
2. Acceptance Tests (ATDD) — define "done" with executable acceptance tests → references/test-strategy.md (ATDD section)
3. Red-Green-Refactor (TDD) — drive implementation test-first → references/tdd-process.md
4. Property-Based Testing — test invariants with generated inputs → references/property-based-testing.md
5. Contract Testing — verify module/service interface agreements → references/test-strategy.md (Contract Testing section)

Working with legacy code

1. Characterization Testing — discover and lock down existing behavior → references/test-strategy.md (Characterization Testing section)
2. Feathers techniques (sprout, wrap, extract interface) → references/legacy-code.md
3. Establish test doubles → references/test-doubles.md

Fixing a bug

1. Write a failing regression test first → references/regression-testing.md
2. Fix with minimum code → references/tdd-process.md
3. Verify no test smells introduced → references/test-quality.md

Phase 2: After Implementation — Test Types

Unit Testing

• Test individual functions/classes in isolation → references/test-quality.md, references/swift-testing.md
• Use Swift Testing APIs → references/swift-testing.md
• Create test doubles → references/test-doubles.md
• Organize tests properly → references/test-organization.md

Integration Testing

• Verify multiple components work together → references/integration-testing.md
• Mock at system boundaries, use real implementations between your own modules
• Test both success and failure paths across module boundaries

Regression Testing

• Every bug fix gets a permanent regression test → references/regression-testing.md
• Test the exact failure scenario, not just the happy path
• Link tests to issue tracker tickets

Mutation Testing

• Evaluate test suite quality by introducing code mutations → references/mutation-testing.md
• Verify tests actually catch bugs, not just execute code
• Target mutation score > 80% for critical business logic

Snapshot Testing

• Capture and compare rendered output → references/snapshot-testing.md
• Use for UI components, API response formats, view hierarchies
• Ensure deterministic data (no Date(), no UUID())

Performance / Benchmark Testing

• Measure execution time, memory, throughput → references/performance-testing.md
• Establish baselines and detect regressions in CI
• Use for hot paths, data processing, real-time operations

Fuzz Testing

• Feed random/malformed inputs to find crashes → references/test-strategy.md (Fuzz Testing section)
• Target parsers, decoders, validators, and security-sensitive code
• Must never crash, only return errors

Smoke Testing

• Quick, high-level tests that critical paths work at all → references/test-strategy.md (Smoke Testing section)
• First line of defense after deployment
• Test core service initialization and critical user flows

Phase 3: Reviewing & Maintaining Tests

Test quality review

• Check FIRST principles → references/test-quality.md
• Detect test smells → references/test-quality.md
• Verify coverage dimensions → references/test-quality.md
• Run mutation testing to validate test effectiveness → references/mutation-testing.md

Test maintenance

• Test impact analysis → references/test-strategy.md (Test Impact Analysis section)
• Dead test cleanup, flaky test fixes → references/test-strategy.md (Test Maintenance section)

TDD Phase Loop

┌─────────────────────────────────────────────┐
│ 1. RED     │ Write a failing test for ONE   │
│            │ behavior. STOP — confirm fail.  │
├────────────┼────────────────────────────────│
│ 2. GREEN   │ Write MINIMUM code to pass.    │
│            │ STOP — confirm pass.            │
├────────────┼────────────────────────────────│
│ 3. REFACTOR│ Clean up, keep tests green.    │
│            │ STOP — confirm pass, next test. │
└─────────────────────────────────────────────┘

Core Laws:

1. No production code without a failing test
2. Only enough code to make the current test pass
3. One behavior per test
4. Tests are executable documentation
5. Fast feedback loop (tests run in milliseconds)

ATDD Loop (Outer Loop Wrapping TDD)

┌───────────────────────────────────────────────────────────────┐
│ 1. ACCEPTANCE TEST │ Write a failing acceptance test for the  │
│                    │ user-visible behavior. It will fail.     │
├────────────────────┼──────────────────────────────────────────│
│ 2. TDD INNER LOOP │ Use Red-Green-Refactor to implement      │
│                    │ the pieces needed.                       │
├────────────────────┼──────────────────────────────────────────│
│ 3. ACCEPTANCE PASS │ When inner TDD is done, acceptance test │
│                    │ should pass. If not, continue TDD.       │
└───────────────────────────────────────────────────────────────┘

Testing Pyramid

                   /\
                  /  \
                 / E2E\          < 5%  — Slow, critical paths only
                /------\
               /  Smoke \        ~5%   — Quick critical path checks
              /----------\
             / Integration\      ~15%  — Component interaction
            /--------------\
           / Property-Based \    ~5%   — Invariant verification
          /------------------\
         /       Unit         \  ~70%  — Fast, isolated
        /----------------------\

Quick Reference: Swift Testing API

import Testing

@Suite(.tags(.networking))
struct UserServiceTests {
  @Test
  func `fetches user successfully`() async throws {
    let sut = UserService(client: MockHTTPClient(response: .success(json)))
    let user = try await sut.fetch(id: "123")
    #expect(user.name == "Alice")
  }

  @Test(arguments: [
    ("[e-mail]", true),
    ("invalid", false),
    ("", false),
  ])
  func `validates emails`(email: String, expected: Bool) {
    #expect(EmailValidator.isValid(email) == expected)
  }
}

Project Conventions (Dynamic)

Project-specific testing conventions are loaded dynamically based on the current working directory. Check for a project-conventions skill or project-level .claude/skills/ directory that provides local testing rules.

Detection: Look for project convention files in this order:

1. ./.claude/skills/*/references/**/conventions.md (project-level skills)
2. ~/.claude/skills/project-conventions/ (user-level project conventions — activation detected via git remote -v or known project markers in the working directory)
3. Fall back to universal testing principles only

When multiple convention files match, read all of them. Project-level conventions override user-level conventions on conflict. Project conventions take precedence over generic guidance for: factory method naming (makeSUT), test utilities, mock data strategy, and framework-specific testing patterns.

Test Doubles Hierarchy

Type	Purpose	Example
Dummy	Fill parameters, never used	Empty closure
Stub	Canned answers	MockClient(returning: .success(user))
Spy	Records calls	Tracks invocation count
Mock	Expectations on calls	Fails if wrong methods called
Fake	Working but simplified	In-memory database

Test Smell Detection

Smell	Symptom	Cure
Flaky	Random failures	Remove time/order deps, use TestClock
Brittle	Breaks on refactor	Test behavior, not implementation
Slow	> 100ms	Mock I/O, reduce scope
Coupled	Needs other tests	Fresh state per test
Tautological	Never fails	Assert real outcomes; run mutation testing
Obscure	Hard to understand	Better names, less setup
Eager	Tests private methods	Test via public API
Giant	Many asserts	Split into focused tests

Mutation testing is the definitive cure for tautological tests — if a mutant survives, the test isn't asserting anything meaningful. See references/mutation-testing.md.

Test Tags

Use tags to enable selective test execution:

extension Tag {
    @Tag static var unit: Self
    @Tag static var integration: Self
    @Tag static var regression: Self
    @Tag static var acceptance: Self
    @Tag static var smoke: Self
    @Tag static var performance: Self
    @Tag static var fuzz: Self
    @Tag static var contract: Self
    @Tag static var characterization: Self
    @Tag static var snapshot: Self
}

Review Checklist

Before/During Implementation

• Example mapping done for complex features
• Acceptance tests define "done" (ATDD) — written before implementation
• TDD Red-Green-Refactor followed — test written before code
• Each test covers exactly ONE behavior with clear Given-When-Then
• Test won't break on refactor (tests behavior, not implementation)
• Property-based tests for invariants (round-trips, idempotence, commutativity)
• Contract tests verify module interfaces on both sides
• Legacy code characterized before modification

Test Quality (per test)

• Descriptive backtick name starting with function under test (no test prefix)
• #expect assertions (not XCTAssert); #expect(throws:) for errors
• Parameterized tests for input variations
• Happy path, error paths, boundary values, and error-clearing all covered
• ALL initial state properties verified in initialization tests
• Exact equality (full collections, not .first ==)
• All dependencies injected; no real file I/O or network calls
• No sleep/timeout — use async/await and confirmation API
• Spy tracking uses call counts (not var called = false) and array argument capture (not optionals)
• Tests written alongside code (not deferred to follow-up PRs)

Project Conventions (loaded dynamically per project)

• Project-specific factory method convention followed
• Project testing utilities used (not reinvented)
• Shared mock data strategy followed
• Framework-specific patterns applied (Swift Testing, JUnit, MockK, etc.)

After Implementation Verification

• Regression: Every bug fix has a regression test linked to the ticket
• Integration: Critical module boundaries tested with real implementations (mocked only at system boundary)
• Mutation: Critical business logic has mutation score > 80%; surviving mutants reviewed
• Snapshot: UI components snapshot-tested across themes and accessibility sizes
• Performance: Hot paths benchmarked with baselines; CI detects regressions
• Fuzz: Parsers, decoders, and validators tested with random/malformed inputs
• Smoke: Critical user flows verified end-to-end
• Test tags applied (.unit, .integration, .regression, .performance, .fuzz, .smoke)
• Full test suite passes

References

Before/During Implementation

• references/tdd-process.md — Red-Green-Refactor phases, interaction protocol, spike rules
• references/test-strategy.md — Example mapping, ATDD, contract testing, characterization testing, fuzz testing, smoke testing, test impact analysis, test maintenance
• references/property-based-testing.md — Generative testing, invariant properties, round-trip testing, shrinking
• references/legacy-code.md — Sprout, wrap, extract interface techniques
• references/test-doubles.md — Mocks, stubs, spies, fakes, fixtures

After Implementation

• references/mutation-testing.md — Mutation operators, mutation score, manual and automated mutation testing, interpreting survivors
• references/integration-testing.md — Module/subsystem/system integration, strategies (top-down, bottom-up, sandwich), boundaries
• references/regression-testing.md — Regression test process, types, naming, permanence, anti-patterns
• references/snapshot-testing.md — Image/text/hierarchy snapshots, swift-snapshot-testing, deterministic data, perceptual diffing
• references/performance-testing.md — Benchmarks, memory testing, regression detection, Swift Benchmark package

Framework, Quality & Conventions

• references/swift-testing.md — @Test, #expect, tags, parameterized tests, async testing
• references/test-organization.md — Directory structure, naming, test strategy matrix
• references/test-quality.md — FIRST principles, smell detection, coverage dimensions, pyramid
• Project conventions loaded dynamically from project-conventions skill or project-level .claude/skills/
• references/testing.framework.moduleinterface.swift — Swift Interface of Testing framework