test-gen

name	test-gen
description	Generate tests for specified code or changed code. Detects test framework automatically, generates test outlines via local LLM, then implements and verifies tests via Sonnet sub-agent. Use this skill when: asked to generate or add tests; asked to improve test coverage; asked to write tests for specific files or functions.

Test Generation Skill

Generates tests for specified or changed code using a multi-stage pipeline: local LLM for analysis, Sonnet for implementation.

---

Step 1: Scope and Framework Detection

Determine what to test:

User instruction	Scope
Specific file(s) or function(s)	Those targets only
"Test the changes" / no target	Changed files on current branch
"Increase coverage"	Files with low or no test coverage

Auto-detect the test framework from project files:

# Check for test framework indicators
ls package.json pyproject.toml pytest.ini Cargo.toml go.mod 2>/dev/null

Identify the project's test conventions:

• Test file naming pattern (e.g., *.test.ts, *_test.go, test_*.py)
• Test directory structure (e.g., __tests__/, tests/, co-located)
• Existing test examples to follow as patterns

Discover existing test infrastructure (mandatory):

# Scan for shared test helpers, fixtures, and mock utilities
bash ~/.claude/hooks/scan-shared-utils.sh

Search the test directories for:

• Shared test helpers/setup functions (e.g., test-utils, test-helpers, factories, fixtures)
• Common mock patterns already in use (e.g., mockServer, createMock*, fake*)
• Shared setup/teardown hooks (e.g., beforeAll in a shared file)
• Record what exists so the sub-agent reuses them instead of creating new ones

Step 2: Test Outline Generation (Local LLM, Zero Claude Tokens)

Classify files and generate test case outlines using local LLM:

# Classify target files
echo "[file paths, one per line]" | bash ~/.claude/hooks/llm-commands.sh classify-changes

# Generate detailed review focusing on testability
bash ~/.claude/hooks/pre-review.sh code

The pre-review output will highlight:

• Functions without test coverage
• Edge cases and error paths
• Input validation boundaries

If Ollama is unavailable, proceed to Step 3 without outlines.

Step 3: Sonnet Test Implementation (Sub-agent Loop)

Launch a Sonnet sub-agent to generate and verify tests:

You are a test engineer.

Test framework: [detected framework]
Test conventions: [naming pattern, directory structure]
Existing test examples: [sample test file contents for pattern reference]

Shared test infrastructure (MUST reuse — do NOT recreate):
[List of existing test helpers, mock utilities, fixtures from Step 1]

Local LLM analysis (for reference):
[Local LLM output, or "None"]

Source files to test:
[Source file contents]

Task:
1. Generate test files following the project's existing conventions
2. Reuse existing shared test helpers and mock utilities — do NOT create new helpers when equivalent ones already exist
3. Cover these categories:
   - Happy path: normal expected behavior
   - Edge cases: boundary values, empty inputs, null/undefined
   - Error paths: invalid inputs, failure scenarios
   - Integration: key interactions between components (if applicable)
4. Run the tests to verify they pass
5. If tests fail, fix them (max 3 fix iterations)
6. If tests still fail after 3 iterations, report:
   - Which tests failed and why
   - Root cause analysis (test issue vs source code issue)
   - Whether the source code needs fixing (escalate to orchestrator)
7. Check that test assertions are meaningful (not just "doesn't throw")
8. Verify test independence (no shared mutable state between tests)
9. Verify mock-reality consistency:
   - Mock return values must match actual API response shapes (read the real type/interface)
   - Mock/spy resets must be in setup/teardown hooks, not inside test bodies
   - Async functions under test must be awaited before assertions
   - Per-test state must use per-test hooks (beforeEach), not once-before-all (beforeAll)
10. When generating tests: NEVER modify production code to make it easier to test. If the production code uses a safe API variant (e.g., parameterized queries, tagged templates, structured builders), adapt the test mock to match that safe API — do not switch production code to an unsafe escape hatch for testability.
11. **Denial-path tests must assert the side-effect's absence (RT8)** — framework-agnostic: when a generated test exercises a gate's *denial* path — an authorization/permission reject (403), rate-limit reject (429), fail-closed (503), or any "request is blocked" case — it MUST assert BOTH the status/outcome AND that the guarded operation did NOT run. A denial test that asserts only the status is vacuously green: if the gate is removed and the mutation proceeds, the test still passes. Never generate a status-only denial test when a mutation spy/double is in scope. (Illustrative spelling — adapt to the project's framework: Jest/Vitest `expect(deleteMock).not.toHaveBeenCalled()` / `toHaveBeenCalledTimes(0)`; pytest `delete_mock.assert_not_called()`; Go `require.Equal(t, 0, deleteCalls)`.)
12. **Race/concurrency tests must assert both branches occurred (RT4)** — framework-agnostic: when a generated test asserts a cardinality outcome under concurrency ("exactly one winner", "no double-success", a zero-collision count), it MUST also assert that the contested window actually opened — both the success and the failure/contention branch each occurred at least once. A bare zero-cardinality assertion passes vacuously if a setup error short-circuits every iteration. (Illustrative spelling — Jest/Vitest `expect(collisions).toBe(0)` PLUS `expect(successes).toBeGreaterThan(0)` AND `expect(failures).toBeGreaterThan(0)`; adapt the matcher to the project's framework.)
13. **Tolerant-parser tests need a mixed valid+malformed fixture (R40 tolerant-consumer sub-clause)** — framework-agnostic: when the code under test is a best-effort/tolerant parser or filter over a collection (line-oriented file, record stream, "skip unknown shapes" reader), generate a fixture that mixes valid records WITH at least one malformed member, and assert the valid records still come through. An all-valid fixture cannot distinguish per-element fault isolation from a whole-collection catch-all that voids the entire batch on one bad element — the silent-total-data-loss shape passes every clean-input test.

Output: generated test files with pass/fail status.

If sub-agents are unavailable, implement tests directly.

Step 4: Coverage Review

Review generated tests for completeness:

• Identify missing edge cases or error paths

• Check that test assertions are meaningful (not just "doesn't throw")

• Verify test independence (no shared mutable state between tests)

• Verify the sub-agent reused existing test helpers (not reimplemented)

• Audit mock-reality alignment with the local LLM before spot-checking manually:

  { cat [generated-test-file]
    echo '=== OLLAMA-INPUT-SEPARATOR ==='
    cat [source-or-type-definition-file]
  } | bash ~/.claude/hooks/llm-commands.sh verify-mock-shapes
  

  The output is a set of [Severity] test-path:line — Problem — Fix blocks (or No findings). Treat Critical/Major findings as mandatory fixes before reporting completion; Minor findings are informational. Remaining unflagged mocks still warrant a manual spot-check against the actual type definitions — the audit is a filter, not a substitute.

• Run the vacuous-test detectors on the generated tests (closes the generate→verify loop — test-gen is the skill that produces the tests these hooks later catch in review). Pass the same base ref the skill used to scope the run (the merge-base / feature-branch point), not the default — the hooks default to main...HEAD, so a no-arg call silently sees an empty diff when test-gen ran on specific files while on main with no feature branch:

  # BASE_REF = the ref test-gen scoped against (e.g. the merge-base with main).
  # When test-gen ran on specific files with no branch, stage the new test
  # files and pass a ref that includes them, else these hooks no-op.
  # RT8 — denial-path test asserts status but not the mutation's absence
  bash ~/.claude/hooks/check-vacuous-denial.sh "$BASE_REF"
  # RT4 — race/cardinality test with no both-branches-occurred guard
  bash ~/.claude/hooks/check-race-vacuous-guard.sh "$BASE_REF"
  

  Any finding here is a vacuous test the generator just wrote — fix it before reporting completion (add the missing negative / lower-bound assertion), do not defer to a later review round. Scope caveat: both hooks are Jest/Vitest TS/JS only (v1) and are no-ops for pytest / Go / RSpec / other frameworks — for those, the RT8/RT4 obligations (#10/#11 above) are a manual review of the generated denial/race tests, not a mechanical gate. A clean hook run on a non-JS project means "not checked", not "passed".

If gaps are found, delegate additional test generation to Sonnet.

Before reporting completion, check migrations and run ALL three verification steps, plus three cross-skill checks adapted from the triangulate skill (lessons from prior runs that surfaced regressions test-gen alone did not catch):

# Check for pending migrations
bash ~/.claude/hooks/check-migrations.sh

# Run ALL three project-defined checks:
# 1. Lint
[lint command]

# 2. Tests
[test command]

# 3. Production build
[build command]

# 4. Production-code-untouched grep (test-gen specific contract).
# test-gen is supposed to ADD tests, not modify production code. If the diff
# touches files outside the test surface, the sub-agent likely violated the
# "do not modify production code to make it easier to test" rule (Step 3
# obligation #9). Adapt the path globs for the project's test naming conventions.
PROD_DIFF=$(git diff main...HEAD --name-only \
  | grep -vE '(^|/)(__tests__|tests|test|spec|specs)/|\.test\.|\.spec\.|_test\.|test_' || true)
if [ -n "$PROD_DIFF" ]; then
  echo "Production-code modifications detected outside test files:"
  echo "$PROD_DIFF"
  echo "Test-gen must not modify production code. Review and either (a) revert"
  echo "the production changes, OR (b) document why the change is mandatory and"
  echo "escalate to the user — generated tests should adapt to existing API,"
  echo "not the other way around."
fi

# 5. CI gate parity. The local lint/test/build set may be a subset of what CI
# runs. Extract every lint/check/verify command from the project's CI
# configuration and run each locally before declaring the skill complete.
# Surfacing a CI-only failure here costs one iteration; surfacing it after
# push costs one push round plus triage time.
while read -r cmd; do
  [ -z "$cmd" ] && continue
  echo "Running CI gate locally: $cmd"
  eval "$cmd" || { echo "CI gate failed locally: $cmd"; exit 1; }
done < <(bash ~/.claude/hooks/extract-ci-checks.sh)
# Environment-parity caveat: a local pass runs in the LOCAL environment, which
# carries generated artifacts (ORM/RPC codegen, build output) that a "static"/
# "no-generate" CI job omits. If a generated test file (or a helper it imports)
# lands in the input set of a generate-skipping job, the local pass is blind to
# the CI-only missing-artifact failure. Displace the generated artifact
# (`mv <codegen-out>{,.bak}`), re-run the gate, restore. See triangulate phase-2
# Step 2-1 item 7 environment-parity sub-check.

# 6. User feedback memory cross-check (same mechanism as triangulate Phase 2).
# Per-project feedback memories at ~/.claude/projects/<slug>/memory/feedback_*.md
# capture rules the user has previously corrected the orchestrator on; reapplying
# the corrected mistake in generated tests wastes review attention re-litigating
# a settled call. Sub-agents in Step 3 cannot see these memories — the orchestrator
# must enumerate them and cross-check the generated test files before commit.
PROJ_SLUG=$(pwd | sed 's|/|-|g')
MEM_DIR="$HOME/.claude/projects/$PROJ_SLUG/memory"
if [ -d "$MEM_DIR" ]; then
  for f in "$MEM_DIR"/feedback_*.md; do
    [ -f "$f" ] || continue
    echo "=== $(basename "$f") ==="
    cat "$f"
    echo
  done
fi
# For each feedback rule with a grep-able pattern, run
# `git diff main...HEAD | grep -nE '<pattern>'`. Direct hits MUST be fixed in
# this skill's session; do not defer to a later review. Non-grep-able rules are
# manual review obligations.

All must pass. Fix any failures before proceeding.

IMPORTANT: Tests and build alone are insufficient. Lint catches unused imports, style violations, and other issues that neither tests nor builds detect. The production build catches issues that only surface during full compilation/bundling — module resolution failures, type errors in non-test code, and bundler/packager-specific failures — that test runs do not exercise. All three must pass.

IMPORTANT: Fix ALL errors found by lint/test/build — including pre-existing errors in files not touched by the current task. Never dismiss failures as "unrelated to our changes." We are building the whole project, not just a diff.

Final report:

=== Test Generation Complete ===
Test files created: [list]
Test cases: [total]
  Happy path: [n]
  Edge cases: [n]
  Error paths: [n]
Tests passing: [n/total]
Lint: [pass/fail]
Build: [pass/fail]
Production-code untouched: [confirmed / N file(s) outside test surface — see findings]
CI gate parity: [N gates extracted, all pass locally / N extracted, M failed and resolved / no CI config detected]
Memory cross-check: [N feedback rules enumerated, no regressions / N enumerated, M direct hits resolved / no memory dir]
Coverage: [if measurable]