Canon TDD for humans and AI agents. Use for production code changes by writing tests first, proving RED, implementing minimal GREEN, and refactoring safely. 2026 edition adds TDP, flakiness gate, ralph-loop integration, memory-search in Step 0, PBT as Step 5.5, mutation testing gate in Step 4, and CJS LSP warning.
Canon TDD for humans and AI agents. Use for production code changes by writing tests first, proving RED, implementing minimal GREEN, and refactoring safely. 2026 edition adds TDP, flakiness gate, ralph-loop integration, memory-search in Step 0, PBT as Step 5.5, mutation testing gate in Step 4, and CJS LSP warning.
version
1.4.0
model
sonnet
invoked_by
both
user_invocable
true
tools
["Read","Write","Edit","Bash","Glob","Grep"]
aliases
["testing-expert"]
best_practices
["Keep a visible test scenario backlog and execute one scenario at a time","Prove RED before code changes and keep evidence in command output","Implement smallest GREEN patch that satisfies current failing test only","Use bounded repair loops and anti-test-hacking checks before completion"]
error_handling
strict
streaming
supported
source
builtin
trust_score
100
provenance_sha
290a7fd993785874
Test-Driven Development (TDD)
Overview
This skill implements Canon TDD with AI-specific guardrails:
Build or update a scenario list.
Execute exactly one scenario as a runnable test.
Prove RED.
Implement minimum change for GREEN.
Optionally refactor.
Repeat until scenario list is empty.
When to Use
Use for:
New features
Bug fixes
Behavior changes
Repository-scale patching driven by tests
AI-assisted code generation where tests are executable specifications
Ask human approval before bypassing only for:
Throwaway prototypes
Purely declarative config edits with no execution path
One-off migration scripts that will not be maintained
The Iron Law
NO PRODUCTION CODE WITHOUT A FAILING TEST FIRST
If code was written first, discard and restart from RED.
Canon Loop
Step 0: Create/refresh scenario backlog
Before building the backlog, query memory for past failure signatures and reusable test templates:
Skill({ skill: 'memory-search' }); // query: "<feature-name> test failure signatures"
Read .claude/context/memory/learnings.md for recurring anti-patterns relevant to this task.
Then:
Keep a short ordered list of test scenarios for this task.
Prioritize by design signal and risk, not by implementation convenience.
Add discovered scenarios during execution.
Reuse templates from memory — do not repeat failure patterns already documented.
Step 1: Pick exactly one scenario and write one runnable test
One behavior per cycle.
Use clear behavior names.
Favor real collaborators; mock only external boundaries.
Step 2: Prove RED
Run the narrowest test command.
Failure must be due to missing behavior, not syntax or setup errors.
Record red evidence (test file and failing assertion message).
Step 3: Implement minimum GREEN patch
Implement only what current red test requires.
No speculative APIs or unrelated cleanup.
Keep patch bounded to current scenario.
Step 4: Prove GREEN
Re-run narrow test command.
Run impacted suite (or package-level test set).
Confirm no regressions.
Flakiness Gate (mandatory for async, hook, or nondeterministic tests):
For tests that involve async I/O, stop hooks, timers, or file system operations, a single pass is insufficient. Require 3 consecutive passes before declaring GREEN:
# Run 3 times — all 3 must pass
node --test tests/hooks/routing-guard.test.cjs && \
node --test tests/hooks/routing-guard.test.cjs && \
node --test tests/hooks/routing-guard.test.cjs
A test that passes once and fails on the second run is RED, not GREEN. Do not advance to Step 5 until 3 consecutive passes are confirmed.
For security hooks, routing validators, auth logic, and any code path that controls access or trust decisions, run Stryker mutation testing after achieving GREEN to verify that tests genuinely catch faults and are not vacuously passing.
# Run Stryker mutation testing (threshold: 85%)
npx stryker run
# Require mutationScore >= 85 in stryker.config.json
For fast-check-based property tests on security hooks, the fail-closed property is the mutation-equivalent gate:
// fast-check fail-closed property — must hold for any input
fc.assert(
fc.property(fc.anything(), input => {
const result = securityHook(input);
// Hook must NEVER return allow=true for malformed/unexpected inputexpect(result.allow).not.toBe(true);
})
);
Skip this gate for non-security application code (Step 4 → Step 5 directly).
Step 5: Optional refactor
Refactor only with green tests.
Re-run the same test set after refactor.
Step 5.5: Property-Based Testing (recommended for utility functions and security hooks)
After refactor (or after Step 4 for security-critical code), consider supplementing example-based tests with property-based tests. PBT achieves 23.1–37.3% pass@1 improvement over example-based TDD alone for LLM code generation (arXiv:2506.18315) by breaking the self-deception cycle.
PBT is a supplement to Canon TDD, not a replacement. Canon RED/GREEN/REFACTOR completes first; PBT runs after GREEN is confirmed.
Step 6: Repeat until backlog empty
AI-Assisted Guardrails
Use tests as executable prompt context; keep prompts short and test-focused.
Prefer deterministic tests (stable fixtures, no nondeterministic ordering).
Use bounded repair loops: max 3 repair attempts per scenario before redesign.
Run anti-test-hacking checks:
Verify changed assertions still express original requirement.
Add at least one negative test for bug-fix tasks.
Ensure code does not branch on test-only artifacts.
Memory Acceleration Layer
Use lightweight memory only to reduce repeated setup and triage:
preferred repo-local test/lint/format commands
recurring failure signatures and short fix summaries
recurring anti-pattern reminders
reusable scenario templates
Reference: references/tdd-memory-profile.md
Hard rules:
memory never bypasses RED proof
memory never changes Canon sequence
keep profile bounded and low-noise
Test-Driven Prompting (TDP) — 2026 Standard Pattern
TDP is the dominant 2026 pattern for multi-agent TDD: inject the verbatim failing test output into the developer agent spawn prompt. This eliminates interpretation errors — the developer sees exactly what the test runner sees.
Pattern
Instead of describing the failure in prose, capture stdout/stderr and inject it directly:
// Step 1: Run test and capture raw outputconst { execSync } = require('child_process');
let testOutput = '';
try {
execSync('node --test tests/hooks/routing-guard.test.cjs', { encoding: 'utf-8' });
} catch (e) {
testOutput = e.stdout + e.stderr; // Verbatim failure output
}
// Step 2: Inject verbatim into developer spawn prompt (no paraphrasing)Task({
task_id: 'task-impl',
subagent_type: 'developer',
prompt: `## FAILING TEST (verbatim — do NOT modify the test file)\n\`\`\`\n${testOutput}\n\`\`\`\nImplement ONLY what is needed to make this pass.`,
});
Why TDP Works
Eliminates paraphrased failure descriptions (telephone game effect)
Developer has the full assertion context: line number, actual vs expected values
Forces minimal implementation — developer can only implement what the test demands
Prevents specification drift between QA agent's test intent and developer's interpretation
TDP + Multi-Agent TDD Decomposition
Step
Agent
Action
1
qa
Write failing test, commit test-only, capture raw output
2
Router
Extract test output, build TDP spawn prompt
3
developer
Implement to GREEN using verbatim test output as spec
4
reflection-agent
Verify no test assertions were modified (git diff check)
Source: Simon Willison (2026) — "Red/Green TDD for agents: failing test output IS the specification"; TDFlow arXiv:2510.23761.
Autonomous TDD with ralph-loop (Session-Persistent Iteration)
For repository-scale TDD where sessions may be interrupted, wire ralph-loop (Mode 2 — router-managed) to maintain the TDD scenario backlog across interruptions:
TDD State Schema
Maintain a TDD-specific state file at .claude/context/runtime/tdd-state.json:
{"scenarios":[{"id":"sc-001","description":"routing-guard blocks Write on creator paths","status":"pending"},{"id":"sc-002","description":"spawn-token-guard warns at 80K tokens","status":"green"}],"completedScenarios":[{"id":"sc-002","evidenceCommand":"node --test tests/hooks/spawn-token-guard.test.cjs","passedAt":"2026-03-12T10:00:00Z"}],"currentScenario":"sc-001","evidenceLog":[{"scenarioId":"sc-001","phase":"red","output":"AssertionError: expected exit code 2, got 0","timestamp":"..."}]}
Resume Pattern
At the start of each iteration, read the TDD state file:
// Step 0 — before building/refreshing backlogconst state = JSON.parse(
fs.readFileSync('.claude/context/runtime/tdd-state.json', 'utf-8') || '{}'
);
const completedIds = (state.completedScenarios || []).map(s => s.id);
const remaining = (state.scenarios || []).filter(s => !completedIds.includes(s.id));
// Pick next scenario from remaining — never re-run completed ones
Integration with ralph-loop Mode 2
Router spawns qa agent with { task_id, subagent_type: 'qa', prompt: TDP_PROMPT + verbatim state }
Mock MemoryRecord. Test confidence gate (threshold 0.7). Use atomic writes.
Property-Based Testing
Use fast-check (and @fast-check/vitest for vitest integration) for any function with invariants — not just routing. fast-check 3.x (2025) adds improved unicode, date, and bigint arbitraries.
Before writing a failing test, verify the API contract exists to prevent "fails due to wrong API" rather than "fails due to missing behavior":
# Step 1: Find the target function's file + line
pnpm search:code "functionName"# Step 2: Verify signature with LSP hover
lsp_hover({ filePath: "/abs/path/to/file.ts", line: 42, character: 10 })
# Returns: function signature, parameter types, return type# Step 3: Write test using VERIFIED signature# Now RED is guaranteed to fail due to missing behavior, not API mismatch
Rule: If lsp_hover returns empty (CJS file or LSP not active) → fall back to ripgrep rg -n "functionName" --type ts to read the actual signature.
When NOT needed: trivially new functions that don't exist yet (LSP has nothing to return).
Contract Testing (Hook Boundaries — Expanded)
Hook contracts define the stdin/stdout JSON protocol. Test at the boundary:
routing-guard.cjs: blocks Task without task_id (exit 2)
unified-creator-guard.cjs: blocks Write to .claude/skills/**/SKILL.md (exit 2)
spawn-token-guard.cjs: warns at 80K tokens (exit 0 + message)
Test Runner Selection (node --test vs Vitest 4)
Agent Studio uses node --test (built-in Node.js test runner) as the default for all .cjs CommonJS files (hooks, lib, scripts). Vitest 4 is the recommended runner for ESM/TypeScript files.
Runner
Use When
Command
node --test
.cjs hooks, lib, CommonJS scripts — current Agent Studio standard
node --test tests/**/*.test.cjs
vitest
.ts, .mts, ESM .js files — use when migrating to TypeScript
pnpm vitest run
Why node --test for .cjs: Vitest requires Vite configuration and ESM-compatible modules. Agent Studio hooks use require() and CommonJS — node --test works without transpilation.
Why Vitest 4 for .ts/ESM: Boot time drops from ~8s (Jest) to ~1.2s (Vitest). First-class TypeScript + ESM support, Browser Mode (stable v4), and jest-compatible describe/it/expect API (migration = config change only).
Anti-pattern: Do NOT use Jest for new files. Vitest is the 2025-2026 standard for ESM/TypeScript.
# Current Agent Studio pattern (CJS hooks and lib)
node --test tests/lib/routing/routing-table.test.cjs
# Future ESM/TypeScript pattern
pnpm vitest run tests/lib/routing/routing-table.test.ts
AI Output Evaluation Testing (Non-Deterministic Agents)
LLM/agent outputs are non-deterministic — binary pass/fail assertions are insufficient. Use score-based evaluation and tool-call sequence validation instead.
For agent tests, validate the sequence and count of tool calls, not just the final output:
// Spy on tool calls and assert orderingconst toolCallLog = [];
const mockTaskUpdate = jest.fn(args => {
toolCallLog.push({ tool: 'TaskUpdate', args });
});
const mockBash = jest.fn(args => {
toolCallLog.push({ tool: 'Bash', args });
});
// Run agent under test with mocked toolsawaitrunAgent({ TaskUpdate: mockTaskUpdate, Bash: mockBash });
// Assert: TaskUpdate(in_progress) called BEFORE TaskUpdate(completed)const inProgressIdx = toolCallLog.findIndex(
c => c.tool === 'TaskUpdate' && c.args.status === 'in_progress'
);
const completedIdx = toolCallLog.findIndex(
c => c.tool === 'TaskUpdate' && c.args.status === 'completed'
);
expect(inProgressIdx).toBeLessThan(completedIdx); // Ordering enforcedexpect(inProgressIdx).toBeGreaterThanOrEqual(0); // Must have been calledexpect(completedIdx).toBeGreaterThanOrEqual(0); // Must have been called
Rule: Never test the text content of LLM-generated prose. Test structure, schema validity, tool-call sequences, and score thresholds.
Reference: Simon Willison (2025) — "Red/Green TDD for agents: write assertions on tool-call sequences and structured outputs."
MSW v2 HTTP Mocking (API Boundary Testing)
Use MSW (Mock Service Worker) v2 to test skills and agents that make external HTTP calls. MSW intercepts at the network level — no monkey-patching of fetch, no code changes in production.
Tests exercise real HTTP client code paths (not mocked abstractions)
onUnhandledRequest: 'error' catches unintentional external calls during tests
Handlers define request/response contracts — doubles as documentation
Agent-Studio targets for MSW boundary tests:
researcher skill → WebSearch/WebFetch HTTP calls
github-ops skill → GitHub API calls
Any agent using mcp__Exa__web_search_exa or WebFetch
Mutation Testing (Stryker JS)
Mutation testing validates test QUALITY, not just coverage. Run after achieving 100% line coverage:
Stryker + Vitest (2026 Standard — ESM/TypeScript projects)
# Install (once per project) — use vitest-runner for ESM/TypeScript
pnpm add -D @stryker-mutator/core @stryker-mutator/vitest-runner vitest
// stryker.config.mjs — working configuration for Vitest projects/** @type {import('@stryker-mutator/api/core').PartialStrykerOptions} */exportdefault {
testRunner: 'vitest',
vitest: {
configFile: 'vitest.config.ts', // optional: path to your vitest configrelated: true, // default: run only tests related to mutated file
},
thresholds: { high: 80, low: 60, break: 50 },
reporters: ['html', 'progress'],
};
# Run mutation tests (use incremental to speed up local loops)
pnpm stryker run --incremental
# Target threshold: >80% mutation score# Score = (killed mutations / total mutations) × 100
Survived — test suite MISSED this code path (add assertion)
No coverage — no test exercises this line at all (add test)
When to run: after completing a TDD cycle for security-critical code (hooks, validators, routing logic). Not required for all code — prioritize by risk.
Agent-Studio priority targets for mutation testing:
.claude/hooks/routing/routing-guard.cjs
.claude/hooks/safety/unified-creator-guard.cjs
.claude/lib/routing/routing-table.cjs
Validation Phase: TDAD Dependency Map (P0)
Before committing, agents MUST identify which test files cover the changed source files. Use compiler-assisted reference discovery or targeted grep:
# Find tests that import the changed file
grep -r "import.*changedFile\|require.*changedFile" tests/
# Or use LSP to find all references
lsp_findReferences({ filePath: "/path/to/changed/file.ts", line: 1, character: 1 })
Build a dependency map and run ONLY those tests first (70% faster regression detection per arXiv:2603.17973):
# 1. Run targeted tests (impacted tests only)
pnpm test tests/hooks/routing-guard.test.cjs
# 2. Verify no regressions in targeted scope# 3. Only then run full suite
pnpm test
Rationale: Full test suites can exceed 5 minutes on large repos. Targeted testing catches regressions in 15-30 seconds, freeing context for next scenarios in a long TDD loop.
Validation Phase: Spec-Gaming Detection (P0)
In the Validate phase, verify the implementation hasn't gamed test assertions:
Checklist:
Tests assert behavior, not implementation details (no testing private variables or class internals)
No hardcoded expected values were copied from test to implementation
Mutation score ≥80% indicates test quality is sufficient for detecting regressions
Review: could the code pass tests while being fundamentally wrong?
Run mutation testing if available:
# Test suite strength validation — mutations should be caught
pnpm stryker run
# If mutation score < 80%, tests are too weak:# - Add negative tests# - Add boundary condition tests# - Verify assertions are on behavior, not mocks
Spec-gaming examples to catch:
✗ Implementation hardcodes return 42 to pass test expecting 42 → mutation testing catches this
✗ Test mocks behavior instead of asserting it → mutation testing shows 0% mutation killed
✗ Test checks log message instead of behavior → flip the assertion, implementation still passes
Agent-Studio targets: After completing security-critical hook or routing changes, run mutation testing before marking task complete.
