test

name	test
description	TDD test writer. Writes failing tests FIRST (red), then verifies they pass after implementation (green). Covers unit, integration, and e2e tests.
metadata	{"author":"runedev","version":"1.4.0","layer":"L2","model":"sonnet","group":"development","tools":"Read, Write, Edit, Bash, Glob, Grep","emit":"tests.passed, tests.failed","listen":"code.changed, db.migrated"}

test

Tests define the EXPECTED BEHAVIOR. They MUST be written BEFORE implementation code. If tests pass without implementation → the tests are wrong. Rewrite them. The only exception: when retrofitting tests for existing untested code.

THE IRON LAW: Write code before test? DELETE IT. Start over.

• Do NOT keep it as "reference"
• Do NOT "adapt" it while writing tests
• Do NOT look at it to "inform" test design
• Delete means delete. git checkout -- <file> or remove the changes entirely. This is not negotiable. This is not optional. "But I already wrote it" is a sunk cost fallacy.

ROLE BOUNDARY: Test writes TEST FILES only. NEVER modify source/implementation files.

• Do NOT "quickly fix" a broken import in source to make tests run
• Do NOT refactor source code to be "more testable"
• Do NOT add missing exports to source files
• If source needs changes → hand off to rune:fix. Test's job ends at the test file. This separation ensures test never writes code biased toward passing its own tests.

VERTICAL SLICING (Iron Law extension): one test → GREEN → one test → GREEN. Never bulk.

• bulk_test_count MUST stay <= 1 before the first GREEN in a session
• After each GREEN, bulk_test_count resets to 0; writing 2+ tests before the next GREEN = HORIZONTAL VIOLATION
• Each cycle MUST emit a commit pair: test(scope): <behavior> + feat(scope): <behavior>
• Claim "I did TDD" is verified by completion-gate against git log --oneline — no paired commits = REJECTED
• Horizontal slicing produces tests-of-imagination, not tests-of-behavior. See references/vertical-tdd.md.
• Emit signal tdd.horizontal.violation when triggered; preflight blocks merge until cycles are unwound. Exceptions (narrow, must be documented in test header): retrofitting characterization tests for legacy untested code; spec-driven scaffolding where the contract is external (OpenAPI, wire protocol).

Instructions

Phase 1: Understand What to Test

1. Read the implementation plan or task description carefully
2. Use Glob to find existing test files: **/*.test.*, **/*.spec.*, **/test_*
3. Use Read on 2-3 existing test files to understand:
   • Test framework in use
   • File naming convention (e.g., foo.test.ts mirrors foo.ts)
   • Test directory structure (co-located vs __tests__/ vs tests/)
   • Assertion style and patterns
4. Use Glob to find the source file(s) being tested

TodoWrite: [
  { content: "Understand scope and find existing test patterns", status: "in_progress" },
  { content: "Detect test framework and conventions", status: "pending" },
  { content: "Write failing tests (RED phase)", status: "pending" },
  { content: "Run tests — verify they FAIL", status: "pending" },
  { content: "After implementation: verify tests PASS (GREEN phase)", status: "pending" }
]

Phase 2: Detect Test Framework

Use Glob to find config files and identify the framework:

• jest.config.* or "jest" key in package.json → Jest
• vitest.config.* or "vitest" key in package.json → Vitest
• pytest.ini, [tool.pytest.ini_options] in pyproject.toml → pytest
  • Async check: If pytest detected AND source files contain async def:
    • Check if pytest-asyncio is in dependencies (pyproject.toml [project.dependencies] or [project.optional-dependencies])
    • Check if asyncio_mode is set in [tool.pytest.ini_options] (values: auto, strict, or absent)
    • If async code exists but no asyncio_mode configured → WARN: "pytest-asyncio not configured. Async tests may silently pass without executing async code. Recommend adding asyncio_mode = \"auto\" to [tool.pytest.ini_options] in pyproject.toml."
• Cargo.toml with #[cfg(test)] pattern → built-in cargo test
• *_test.go files present → built-in go test
• cypress.config.* → Cypress (E2E)
• playwright.config.* → Playwright (E2E)

Verification gate: Framework identified before writing any test code.

Phase 3: Write Failing Tests

Use Write to create test files following the detected conventions:

1. Mirror source file location: if source is src/auth/login.ts, test is src/auth/login.test.ts

2. Structure tests with clear describe / it blocks (or language equivalent):

   • describe('Feature name')
     • it('should [expected behavior] when [condition]')

3. Cover all three categories:

   • Happy path: valid inputs, expected success output
   • Edge cases: empty input, boundary values, large input
   • Error cases: invalid input, missing data, network failure simulation

4. Use proper assertions. Do NOT use implementation details — test behavior:

   • Jest/Vitest: expect(result).toBe(expected)
   • pytest: assert result == expected
   • Rust: assert_eq!(result, expected)
   • Go: if result != expected { t.Errorf(...) }

5. For async code: use async/await or pytest @pytest.mark.asyncio

Python Async Tests (pytest-asyncio)

When writing tests for async Python code:

1. Verify setup before writing tests:

   • Confirm pytest-asyncio is in project dependencies
   • Confirm asyncio_mode is set in pyproject.toml [tool.pytest.ini_options] (recommend "auto")
   • If neither is configured, warn the caller and suggest setup before proceeding

2. Writing async test functions:

   • With asyncio_mode = "auto": just write async def test_something(): — no decorator needed
   • With asyncio_mode = "strict": every async test needs @pytest.mark.asyncio
   • Without asyncio_mode set: always use @pytest.mark.asyncio decorator explicitly

3. Async fixtures:

   • Use @pytest_asyncio.fixture (NOT @pytest.fixture) for async setup/teardown
   • Scope rules: async fixtures default to function scope — use scope="session" carefully with async

4. Common pitfalls:

   • Tests that pass without await — they run but don't execute the async path
   • Missing pytest-asyncio makes async def test_* silently pass as empty coroutines
   • Mixing sync and async fixtures can cause event loop errors

Phase 3.5: Cycle Discipline (Vertical Slicing Gate)

Before running the new test in Phase 4, count what's about to be added to the working tree:

bulk_test_count = (test files staged + test files unstaged but new) since the last GREEN commit

Gate: bulk_test_count MUST be exactly 1.

State	Action
bulk_test_count == 1	Proceed to Phase 4 (run RED).
bulk_test_count >= 2 AND no prior GREEN this session	HORIZONTAL VIOLATION — pause, keep one test, defer the rest.
bulk_test_count >= 2 AND last GREEN exists in git log	HORIZONTAL VIOLATION — same.
bulk_test_count == 0	No test to run; this phase is a no-op.

Cycle audit log — append to TEST.md (or create) one line per cycle:

cycle 1 — RED: test_validates_email_rejects_empty | GREEN: validateEmail handles empty | commit: 4f3a1c
cycle 2 — RED: test_validates_email_requires_at_sign | GREEN: validateEmail checks @ | commit: a92e0d

The audit log is the receipt. completion-gate reads it.

When the violation fires, do NOT delete tests automatically — surface to the calling agent: "horizontal slicing detected (N tests before first GREEN). Recommend keeping test K, deferring N-1 to subsequent cycles."

Phase 4: Run Tests — Verify They FAIL (RED)

Use Bash to run ONLY the newly created test files (not full suite):

• Jest: npx jest path/to/test.ts --no-coverage
• Vitest: npx vitest run path/to/test.ts
• pytest: pytest path/to/test_file.py -v (if async tests and no asyncio_mode in config: add --asyncio-mode=auto)
• Rust: cargo test test_module_name
• Go: go test ./path/to/package/... -run TestFunctionName

Hard gate: ALL new tests MUST fail at this point.

• If ANY test passes before implementation exists → that test is not testing real behavior. Rewrite it to be stricter.
• If tests fail with import/syntax errors (not assertion errors) → fix the test code, re-run

Phase 5: After Implementation — Verify Tests PASS (GREEN)

After rune:fix writes implementation code, run the same test command again:

1. ALL tests in the new test files MUST pass
2. Run the full test suite with Bash to check for regressions:
   • npm test, pytest, cargo test, go test ./...
3. If any test fails: report clearly which test, what was expected, what was received
4. If an existing test now fails (regression): escalate to rune:debug

Verification gate: 100% of new tests pass AND 0 regressions in existing tests.

Phase 6: Coverage Check

After GREEN phase, call verification to check coverage threshold (80% minimum):

• If coverage drops below 80%: identify uncovered lines, write additional tests
• Report coverage gaps with file:line references

Phase 6.5: Diff-Aware Mode (optional)

When invoked with mode: "diff-aware" or by cook after implementation:

1. Run git diff main --name-only to get changed files
2. For each changed file, trace its blast radius: what imports it? what routes does it serve? what components render it?
3. Map changed files → affected routes/endpoints/pages
4. Prioritize tests: files with most downstream dependents get tested first
5. Generate targeted test commands that cover ONLY affected paths — skip unchanged modules

This mode is valuable for large codebases where running the full suite is slow. It answers: "what could this diff have broken?"

Input:  git diff main --name-only
Output: Prioritized test plan targeting only affected paths

Test Types — 4-Layer Methodology

Tests are organized in 4 layers. Each layer catches a different failure class. Higher layers are slower but catch integration issues lower layers miss.

Layer	Type	What It Catches	Framework	Speed
L1	Unit	Logic bugs, boundary violations, pure function errors	jest/vitest/pytest/cargo test	Fast
L2	Integration	API contract breaks, DB query errors, service interaction failures	supertest/httpx/reqwest	Medium
L3	True Backend	Real tool/service output correctness (not just exit 0)	Same + real software invocation	Medium-Slow
L4	E2E / Subprocess	Full workflow from user/agent perspective, installed app works	Playwright/Cypress/subprocess	Slow

Layer rules:

• L1 (Unit): Synthetic data, no external deps. Every function tested in isolation. Fast, deterministic, CI-friendly
• L2 (Integration): Tests service boundaries — API endpoints, DB operations, message queues. May need test DB or mock server
• L3 (True Backend): Invokes the REAL tool/service and verifies output programmatically. No graceful degradation — if the dependency isn't installed, tests FAIL (not skip). Verify: magic bytes, file size > 0, content structure. Print artifact paths for manual inspection
• L4 (E2E/Subprocess): Tests the installed command/app via subprocess or browser automation. Full user workflow: input → process → output → verify

"No graceful degradation" rule (L3/L4): Hard dependencies MUST be installed. Tests MUST NOT skip or produce fake results when the dependency is missing. A silently skipping test is worse than a loudly failing test.

Additional modes:

Type	When	Speed
Regression	After bug fixes	Fast
Diff-aware	After implementation, large codebases (Phase 6.5)	Fast (targeted)

TEST.md — Test Plan + Results Document

For non-trivial features (3+ test files or 20+ test cases), create a TEST.md in the test directory. This is BOTH a planning doc (written BEFORE tests) and results doc (appended AFTER tests pass).

Before writing tests — write the plan:

# Test Plan: [Feature Name]

## Test Inventory
- `test_core.py`: ~XX unit tests planned (L1)
- `test_integration.py`: ~XX integration tests planned (L2)
- `test_e2e.py`: ~XX E2E tests planned (L3/L4)

## Unit Test Plan (L1)
| Module | Functions | Edge Cases | Est. Tests | Req IDs |
|--------|-----------|------------|------------|---------|
| `core/auth.py` | login, register, refresh | expired token, invalid creds, rate limit | 12 | REQ-001, REQ-003 |

## E2E Scenarios (L3/L4)
| Workflow | Simulates | Operations | Verified | Req IDs |
|----------|-----------|------------|----------|---------|
| User signup | New user onboarding | register → verify → login | Token valid, profile created | REQ-005 |

## Realistic Workflow Scenarios
- **[Name]**: [Step 1] → [Step 2] → verify [output properties]

After tests pass — append results:

## Test Results
[Paste full `pytest -v --tb=no` or `npm test` output]

## Summary
- Total: XX | Passed: XX | Failed: 0
- Execution time: X.Xs | Coverage: XX%

## Requirement Coverage
| Req ID | Test File(s) | Status |
|--------|-------------|--------|
| REQ-001 | `test_auth.py::test_login` | ✅ Covered |
| REQ-002 | — | ❌ Not covered |

## Gaps
- [Areas not covered and why]

Why TEST.md: Planning tests before code catches missing edge cases early. Appending results creates permanent evidence. One document = complete testing story.

Skill Behavior Tests (Eval Scenarios)

For testing SKILL.md behavior (not code), use Eval Scenarios — unit tests for skill files, not code files.

Eval Scenario Format

## Eval: E[NN] — [scenario name]

### Prompt
[The exact situation/message an agent receives]

### Expected Reasoning
[Step-by-step reasoning the agent SHOULD follow]

### Must Include
- [Assertion 1: what the output MUST contain or do]
- [Assertion 2]

### Must NOT
- [Anti-pattern 1: what the output MUST NOT do]
- [Anti-pattern 2]

### Category
happy-path | adversarial | edge-case | jailbreak | credential-leak

Eval Coverage Requirements

A skill is behavior-tested when it has evals covering:

Category	Min Evals	Purpose
Happy path	1	Core workflow executes correctly
Edge case	1	Empty input, missing context, unusual state
Adversarial	1	Time pressure, sunk cost, authority pressure
Jailbreak / injection	1	Prompt injection attempt, "ignore instructions"

Minimum: 4 evals per skill (1 per category). Security-critical skills (sentinel, safeguard): 8+ evals.

Eval Storage

Save eval files as skills/<name>/evals.md. Each eval is a numbered scenario (E01–E24 range). skill-forge Phase 7 checks for evals presence before ship.

Error Recovery

• If test framework not found: ask calling skill to specify, or check package.json devDependencies
• If Write to test file fails: check if directory exists, create it first with Bash mkdir -p
• If tests error on import (module not found): check that source file path is correct, adjust imports
• If Bash test runner hangs beyond 120 seconds: kill and report as TIMEOUT

Called By (inbound)

• cook (L1): Phase 3 TEST — write tests first
• fix (L2): verify fix passes tests
• review (L2): untested edge case found → write test for it
• deploy (L2): pre-deployment full test suite
• preflight (L2): run targeted regression tests on affected code
• surgeon (L2): verify refactored code
• launch (L1): pre-deployment test suite
• safeguard (L2): writing characterization tests for legacy code
• review-intake (L2): write tests for issues identified during review intake
• scaffold (L1): generate initial test suite for new project
• graft (L2): write integration tests for grafted code
• skill-forge (L2): write tests for new skill functionality
• mcp-builder (L2): write tests for MCP server tools
• debug (L2): write regression test capturing the bug
• plan (L2): reference test requirements in implementation plan

Calls (outbound)

• verification (L3): Phase 6 — coverage check (80% minimum threshold)
• browser-pilot (L3): Phase 4 — e2e and visual testing for UI flows
• debug (L2): Phase 5 — when existing test regresses unexpectedly

Data Flow

Feeds Into →

• cook (L1): test results (pass/fail/coverage) → cook's Phase 5 quality gate evidence
• completion-gate (L3): test runner stdout → evidence for "tests pass" claims
• fix (L2): failing test output → fix's target (what to make green)

Fed By ←

• plan (L2): phase file test tasks → test's RED phase targets (what to test)
• review (L2): untested edge cases found during review → new test targets
• fix (L2): implemented code → test's GREEN phase verification target

Feedback Loops ↻

• test ↔ fix: test writes failing tests (RED) → fix implements to pass → test verifies (GREEN) → if new failures emerge, loop continues
• test ↔ debug: test discovers regression → debug diagnoses root cause → test writes regression test to prevent recurrence

Anti-Rationalization Table

Excuse	Reality
"Too simple to need tests first"	Simple code breaks. Test takes 30 seconds. Write it first.
"I'll write tests after — same result"	Tests-after = "what does this do?" Tests-first = "what SHOULD this do?" Completely different.
"I already wrote the code, let me just add tests"	Iron Law: delete the code. Start over with tests. Sunk cost is not an argument.
"Tests after achieve the same goals"	They don't. Tests-after are biased by the implementation you just wrote.
"It's about spirit not ritual"	Violating the letter IS violating the spirit. Write the test first.
"I mentally tested it"	Mental testing is not testing. Run the command, show the output.
"This is different because..."	It's not. Write the test first.
"I'll batch the tests since they're related"	Batched tests = tests of imagination. Each cycle reacts to the prior. Write one, GREEN it, then the next.
"All five tests are already written, let me just review them with you"	Same fallacy as code-before-test. Keep the first one, defer the others to subsequent cycles.

Advanced: Oracle-Injection E2E Testing

For data pipelines, AI workflows, and multi-stage processing where comparing full output structures is impractical, use oracle injection:

1. Generate a UUID oracle token: const oracle = crypto.randomUUID()
2. Inject into synthetic input: embed the oracle in realistic test data that flows through the pipeline
3. Run the full pipeline: input → all stages → output
4. Search for oracle in output: if found → data flowed end-to-end correctly

// Example: testing a document processing pipeline
const oracle = "ORACLE-" + crypto.randomUUID();
const testDoc = `Meeting notes: discussed ${oracle} integration timeline`;
const result = await pipeline.process(testDoc);
assert(result.output.includes(oracle), "Oracle not found — pipeline lost data");

When to use: E2E tests for pipelines with 3+ stages, LLM-based processing, ETL workflows, or any system where output structure is complex/non-deterministic but data preservation is critical.

When NOT to use: Unit tests, simple CRUD, or when exact output comparison is feasible.

Spec→Test Traceability

When a spec or plan with acceptance criteria exists (.rune/features/<name>/requirements.md, plan.md, or phase file), every criterion MUST map to at least one test case.

Spec Acceptance Criteria → Test Case → Implementation

US-1/AC-1.1: "User can reset password via email" → test_password_reset_sends_email()
US-1/AC-1.2: "Rate limit: max 3 reset attempts/hour" → test_password_reset_rate_limit()
FR-3: "Expired tokens rejected" → test_expired_reset_token_rejected()

Validation step (after writing tests): Cross-check acceptance criteria against test names. For each criterion:

• Has test → OK
• No test → flag as UNTESTED REQUIREMENT (more serious than uncovered lines)

Cross-boundary minimum: every user story whose path crosses the UI↔data boundary (story has both a UI task and an endpoint/data task) needs ≥1 L2 integration test exercising handler → service/endpoint with the REAL route wired (contract-level is fine; mocking the ENTIRE chain does not count). A story covered only by unit tests with the handler mocked = the story is NOT covered — that's how dead buttons reach 80% coverage.

Contract tests first: when plan emitted .rune/features/<name>/contracts/, each contract gets a failing contract test (request/response shape, error cases) BEFORE the endpoint is implemented — the RED phase for the API surface.

Why this is stronger than coverage: Coverage checks that lines were EXECUTED. Traceability checks that INTENT was VERIFIED. You can have 100% coverage but miss a requirement if the test doesn't assert the right behavior.

Skip if: No spec AND no plan exists (ad-hoc fix), or neither has an acceptance criteria section. A requirements.md with US/AC IDs makes this section MANDATORY — "no plan file" alone is not a skip reason.

Browser click-through (advisory): when a UI story completes, SUGGEST a browser-pilot run of the story's Independent Test (from requirements.md) — one real click-path beats ten mocked renders. Advisory, not a gate: skip freely in headless/CI-only environments.

Eval-Driven Development

Define capability evals and regression evals BEFORE writing implementation code. Evals go beyond unit tests — they verify that the agent/system can handle the feature's intent, not just its mechanics.

Two Eval Types

Type	Purpose	Pass Criteria	When
Capability eval	Can the system do this new thing?	pass@k: ≥1 success in k attempts (k=3-5)	Before implementation
Regression eval	Did we break existing behavior?	pass^k: ALL k attempts must pass	After implementation

pass@k (capability): At least 1 of k runs succeeds. Used for new features where some variance is acceptable. Threshold: ≥90% pass@3 for standard features, ≥95% pass@5 for critical paths.

pass^k (regression): ALL k runs must pass. Used for existing behavior that must never break. If ANY run fails, it's a regression. Threshold: 100% pass^3.

Eval File Format

Store evals in .rune/evals/<feature>.md:

# Eval: <feature name>

## Capability Evals (pass@k)
| ID | Description | k | Threshold | Status |
|----|-------------|---|-----------|--------|
| CAP-1 | [what the system should be able to do] | 3 | 90% | pending |

## Regression Evals (pass^k)
| ID | Description | k | Status |
|----|-------------|---|--------|
| REG-1 | [existing behavior that must not break] | 3 | pending |

Anti-Pattern: Eval Overfitting

Do NOT overfit evals to specific prompts or known examples. Evals should test the capability, not the exact input.

• BAD: "When user says 'hello', respond with 'Hi there!'" — tests exact string match
• GOOD: "When user greets, respond with a greeting" — tests capability

Integration with TDD

1. Write eval definitions (capability + regression) → .rune/evals/<feature>.md
2. Write unit/integration tests (RED phase) → test files
3. Implement feature (GREEN phase) → source files
4. Run evals to verify capability achieved + no regressions
5. Preflight checks eval results as part of quality gate

Red Flags — STOP and Start Over

If you catch yourself with ANY of these, delete implementation code and restart with tests:

• Code exists before test file
• "I already manually tested it"
• "Tests after achieve the same purpose"
• "It's about spirit not ritual"
• "This is different because..."
• "Let me just finish this, then add tests"

All of these mean: Delete code. Start over with TDD.

Constraints

1. MUST write tests BEFORE implementation code — if tests pass without implementation, they are wrong
2. MUST cover happy path + edge cases + error cases — not just happy path
3. MUST run tests to verify they FAIL before implementation exists (RED phase is mandatory)
4. MUST NOT write tests that test mock behavior instead of real code behavior
5. MUST achieve 80% coverage minimum — identify and fill gaps
6. MUST use the project's existing test framework and conventions — don't introduce a new one
7. MUST NOT say "tests pass" without showing actual test runner output
8. MUST delete implementation code written before tests — Iron Law, no exceptions
9. MUST show RED phase output (actual failure) — "I confirmed they fail" without output is REJECTED
10. MUST NOT modify source/implementation files — test writes test files ONLY, hand off source changes to rune:fix
11. MUST NOT write a 2nd test until the 1st test reaches GREEN — vertical slicing only, bulk_test_count <= 1 enforced
12. MUST emit commit pair per cycle (test: then feat:) — git log is the audit trail for "I did TDD" claims

Mesh Gates

Gate	Requires	If Missing
RED Gate	All new tests FAIL before implementation	If any pass, rewrite stricter tests
GREEN Gate	All tests PASS after implementation	Fix code, not tests
Coverage Gate	80%+ coverage verified via verification	Write additional tests for gaps

Output Format

## Test Report
- **Framework**: [detected]
- **Files Created**: [list of new test file paths]
- **Tests Written**: [count]
- **Status**: RED (failing as expected) | GREEN (all passing)

### Test Cases
| Test | Status | Description |
|------|--------|-------------|
| `test_name` | FAIL/PASS | [what it tests] |

### Coverage
- Lines: [X]% | Branches: [Y]%
- Gaps: `path/to/file.ts:42-58` — uncovered branch (error handling)

### Regressions (if any)
- [existing test that broke, with error details]

Testing Anti-Patterns (Gate Functions)

Before writing tests, check yourself against these 5 anti-patterns. Each has a gate function — a question you MUST answer before proceeding.

Anti-Pattern 1: Testing Mock Behavior

Asserting that a mock exists (e.g., testId="sidebar-mock") instead of testing real component behavior. You're proving the mock works, not the code. Gate: "Am I testing real component behavior or just mock existence?" → If mock existence: STOP. Rewrite to test real behavior.

Anti-Pattern 2: Test-Only Methods in Production

Adding destroy(), reset(), or __testSetup() methods to production classes that are ONLY called from test files. Production code should not know tests exist. Gate: "Is this method only called by tests?" → If yes: STOP. Move to test utilities or test helper file, not production class.

Anti-Pattern 3: Mocking Without Understanding Side Effects

Mocking a function without first understanding ALL its side effects. The real function may write config files, update caches, or emit events that downstream code depends on. Gate: Before mocking, STOP and answer: "What side effects does the REAL function have? Does this test depend on any of those?" → Run with real implementation first, observe what happens, THEN add minimal mocking.

Anti-Pattern 4: Incomplete Mocks

Partial mock missing fields that downstream code consumes. Your test passes because it only checks the fields you mocked, but production code reads fields your mock doesn't have → runtime crash. Iron Rule: Mock the COMPLETE data structure as it exists in reality, not just fields your immediate test uses. Examine actual API response / real data shape before writing mock.

Anti-Pattern 5: Mock Setup Longer Than Test Logic

If mock setup is 30 lines and the actual test assertion is 3 lines, the test is testing infrastructure, not behavior. This is a code smell that indicates wrong abstraction level. Gate: "Is my mock setup longer than my test logic?" → If yes: test at a higher level (integration) or extract mock factories.

Anti-Pattern 6: Test Slop (Framework-Behavior Tests)

Tests that verify the framework works rather than YOUR code works. If the test would still pass with an empty component/function, it's testing infrastructure. Gate: "Would this test pass if I deleted my business logic?" → If yes: STOP. Rewrite to test behavior that YOUR code introduces.

Examples of test slop:

• "renders without crashing" (tests that React works, not your component)
• "route responds with 200" without checking response body (tests Express, not your handler)
• Asserting a mock was called N times without checking the RESULT of those calls
• Type existence tests (typeof result === 'object') when you should test the actual value
• Component renders + handler fully mocked, presented as story coverage — the click path was never exercised; the story's dead button ships with "green" tests

Red flags — any of these means STOP and rethink:

• Mock setup longer than test logic
• *-mock test IDs in assertions
• Methods only called in test files
• Can't explain in one sentence why a mock is needed
• Test would pass with empty implementation (test slop)

Returns

Artifact	Format	Location
Test files	Source files	Co-located or __tests__/ per project convention
Test plan + results	Markdown	TEST.md in test directory (non-trivial features only)
Eval scenarios	Markdown	skills/<name>/evals.md (for skill behavior testing)
Coverage report	Inline stdout	Shown in Test Report
Test Report	Markdown (inline)	Emitted to calling skill (cook, fix, review)

Chain Metadata

Append to Test Report when invoked standalone. Suppress when called as sub-skill inside an L1 orchestrator (cook, team, etc.) — the orchestrator emits a consolidated block. See docs/references/chain-metadata.md.

chain_metadata:
  skill: "rune:test"
  version: "1.2.0"
  status: "[DONE]"
  domain: "[area tested]"
  files_changed:
    - "[test files created/modified]"
  exports:
    test_results: { passed: [N], failed: [N], coverage: [N] }
    test_files: ["[paths to test files]"]
    status: "[RED | GREEN]"  # RED = TDD failing (expected), GREEN = all pass
  suggested_next:  # status-aware — pick based on RED or GREEN
    # When GREEN:
    - skill: "rune:preflight"
      reason: "[grounded in results — e.g., 'All 15 tests GREEN, check edge case completeness']"
      consumes: ["test_results", "test_files"]
    # When RED (TDD expected):
    - skill: "rune:fix"
      reason: "[grounded in failures — e.g., '3 tests RED as expected, implement to make them pass']"
      consumes: ["test_results", "test_files"]

Sharp Edges

Known failure modes for this skill. Check these before declaring done.

Failure Mode	Severity	Mitigation
Tests passing before implementation exists	CRITICAL	RED Gate: rewrite stricter tests — passing without code = not testing real behavior
Skipping the RED phase (not confirming FAIL)	HIGH	Run tests, confirm FAIL output before calling cook/fix to implement
Testing mock behavior instead of real code	HIGH	Anti-Pattern 1 gate: "Am I testing real behavior or mock existence?"
Mocking without understanding side effects	HIGH	Anti-Pattern 3 gate: run with real impl first, observe side effects, THEN mock minimally
Incomplete mocks missing downstream fields	HIGH	Anti-Pattern 4 iron rule: mock COMPLETE data structure, not just fields your test checks
Coverage below 80% without filling gaps	MEDIUM	Coverage Gate: identify uncovered lines and write additional tests
Introducing a new test framework instead of using existing one	MEDIUM	Constraint 6: detect framework first, use project's existing one always
Modifying source files to make tests work	HIGH	Role boundary: test writes test files ONLY — source changes go to rune:fix
Test-only methods leaking into production code	MEDIUM	Anti-Pattern 2 gate: if method only called by tests → move to test utilities
Bulk test writing before first GREEN (horizontal slicing)	CRITICAL	Phase 3.5 gate — bulk_test_count <= 1, defer additional tests to subsequent cycles
Test names describe shape (returns boolean, has property) instead of behavior	MEDIUM	Scan names for shape-words; rewrite to behavior verbs (accepts/rejects/produces). See references/test-quality.md
Mocking internal collaborators in same module	HIGH	System-boundary rule from references/mocking-policy.md — mock only what you don't own
Layering old shallow-module tests on top of new deepened-interface tests	MEDIUM	After improve-architecture deepens a module, REPLACE tests don't ADD them; one interface = one test surface

Self-Validation

SELF-VALIDATION (run before emitting Test Report):
- [ ] Every test file has at least one assertion — no empty test bodies
- [ ] RED phase output shows actual failures (not "0 tests") — tests were real, not stubs
- [ ] No test modifies source code — test files only, source changes belong to fix
- [ ] Test names describe behavior, not implementation ("should reject expired token" not "test function X")
- [ ] No mocks of the thing being tested — only mock external dependencies
- [ ] If BA requirements exist (REQ-xxx), every requirement has at least one test — check plan's Traceability Matrix

Done When

• Test framework detected from project config files
• Tests cover happy path + at least 2 edge cases + error case
• All new tests FAIL (RED phase — actual failure output shown)
• After implementation: all tests PASS (GREEN phase — actual pass output shown)
• Coverage ≥80% verified via verification
• Test Report emitted with framework, test count, RED/GREEN status, and coverage
• Self-Validation: all checks passed
• Cycle audit trail intact — every RED has a paired GREEN in git log, no orphan tests pre-first-GREEN
• Test names use behavior-words (accepts/rejects/produces/...) not shape-words (returns/has property/is defined)

Cost Profile

~$0.03-0.08 per invocation. Sonnet for writing tests, Bash for running them. Frequent invocation in TDD workflow.

Scope guardrail: Do not modify source or implementation files to make tests pass unless explicitly delegated by the parent agent.