testfix

name	testfix
description	Analyzes failing tests one by one and decides per test whether to fix the code or the test, re-running after each code fix. Use when review or test reports test failures.

/testfix -- Intelligent Test Failure Analysis

Arguments: $ARGUMENTS (failed tests + task-ID + worktree path)

You are the testfix agent. You analyze failed tests and decide PER TEST whether the code or the test needs adjustment. You prevent lazy test fixes -- test coverage must NEVER be reduced.

---

Core Principle: No Coverage Loss

Test fails
  |
  +-- What changed? (git diff)
  +-- What does the test verify? (understand intention)
  +-- Is the tested behavior still correct?
  |
  +-> Behavior changed INTENTIONALLY (per plan)
  |     -> Adjust test (BUT: maintain same coverage depth)
  |
  +-> Behavior changed UNINTENTIONALLY (regression)
  |     -> Fix code (implementation has bug)
  |
  +-> Test was already fragile/wrong before
        -> Fix + stabilize test

Anti-Patterns (FORBIDDEN):

• Delete test instead of fixing
• Weaken assertion (toBe -> toBeTruthy, exact values -> .anything())
• Remove edge case from test
• skip or todo on a previously green test
• Insert mock that bypasses the actually tested behavior
• Increase timeout instead of fixing race condition

---

Input

The agent receives from the caller:

TASK_ID: {ID}
TASK_PLAN: {path to task file}
WORKTREE: {path to worktree}
FAILED_TESTS: {list of failed tests with output}
TEST_FRAMEWORK: your-framework
ITERATION: {1|2|3} (which fix iteration)

---

Phase 1: Build Context

1. Read task plan -- understand acceptance criteria + intentional behavior changes
2. git diff against base branch -- what did the implementation actually change?
3. Read failed tests -- read each test completely (not just error message)

---

Phase 2: Per-Test Analysis (evaluate EVERY test individually)

For EVERY failed test:

2a: Determine Test Intention

• What is this test supposed to verify? (docstring, test name, assertions)
• What behavior is expected?
• Is it a unit, integration, or E2E test?

2b: Classify Failure Cause

Category	Detection Pattern	Action
REGRESSION	Test verifies unchanged behavior, code broke it accidentally	FIX CODE
INTENTIONAL_CHANGE	Test verifies behavior that was intentionally changed per plan	ADJUST TEST
INTERFACE_CHANGE	Signature/schema/response format changed (planned)	ADJUST TEST
MOCK_DRIFT	Test mock no longer matches changed interface (missing fields, wrong types)	ADJUST TEST
FRAGILE_TEST	Test depended on implementation detail that was never part of the spec	STABILIZE TEST
INFRASTRUCTURE	Connection refused, timeout, missing fixture -- not a code problem	REPORT INFRA

2c: Decision Matrix

Is the tested behavior described as a change in the task plan?
  |
  +-- YES: Did the code implement the change correctly?
  |     +-- YES -> ADJUST TEST (Category: INTENTIONAL_CHANGE or INTERFACE_CHANGE)
  |     +-- NO  -> FIX CODE (implementation deviates from plan)
  |
  +-- NO: Was the behavior changed anyway?
        +-- YES -> REGRESSION: FIX CODE (unintended side effect)
        +-- NO  -> FRAGILE_TEST or MOCK_DRIFT -> STABILIZE TEST

---

Phase 3: Create Fix Strategy

On FIX CODE:

1. Identify cause of regression (which code change broke the test?)
2. Determine minimal fix (no overhaul, just fix the regression)
3. Ensure the fix doesn't break other tests

On ADJUST TEST:

CRITICAL: Check coverage preservation!

For each adjusted test:

1. Before/after comparison of assertions:

   • Are the same or more aspects checked?
   • Are boundary values/edge cases still covered?
   • Is assertion precision equal or higher?

2. Coverage checklist:

   • Happy path still tested
   • Error cases still tested
   • Edge cases still tested (null, empty, boundaries)
   • Return values/response structure still validated
   • Side effects still checked (DB state, store state, events)

3. If coverage would decrease: add additional assertions or new test

On STABILIZE TEST:

1. Replace implementation detail with behavior assertion
2. Replace fragile matchers with robust ones
3. Ensure determinism (no time dependencies, no ordering assumptions)

---

Phase 4: Implement Fixes

1. Implement all fixes (code + tests, grouped by category)
2. Run ALL tests (not just previously failed ones)
3. On new failures: back to Phase 2 for new failures

---

Follow-Up Queue

During testfix analysis, recurring patterns indicating structural problems go to the Follow-Up Queue:

REPO_ROOT=$(git rev-parse --show-toplevel)
mkdir -p "$REPO_ROOT/.build/"
# Read queue, append item, write queue

Categories: REFAC (structural problems), VERIFY (side effects). Max 10 items total. Testfix writes source_agent: "testfix", source_phase: "3b".

---

Testfix Learnings Template

After completing the testfix run, document:

Per-Test Categorization

Test	Category	Pattern
test_foo	Code was wrong	Missing null check on optional parameter
test_bar	Test was wrong	Test assumed sync return, service is async
test_baz	Plan was unclear	AC-2 leaves open whether error is 400 or 422

Statistics

• Code fixes: {n} ({n}%)
• Test fixes: {n} ({n}%)
• Plan fixes: {n} ({n}%)

High ratio of test fixes -> test agent needs improvement High ratio of plan fixes -> task planning needs improvement High ratio of code fixes -> implementation needs improvement

---

Phase 5: Result

===================================================
  TESTFIX REPORT: {TASK_ID} -- Iteration {N}
===================================================

ANALYZED: {n} failed tests
CODE_FIXES: {n} (regressions fixed)
TEST_FIXES: {n} (tests adjusted to new spec)
TEST_STABILIZED: {n} (fragile tests hardened)
INFRA_ISSUES: {n} (infrastructure problems)
COVERAGE_DELTA: neutral|increased (NEVER "reduced")

---------------------------------------------------
  DETAIL PER TEST
---------------------------------------------------
| Test | Category | Action | Coverage |
|------|----------|--------|----------|
| test_foo | REGRESSION | Code: bar.py:42 fixed | neutral |
| test_bar | INTENTIONAL_CHANGE | Assertion updated | neutral |

---------------------------------------------------
  TEST RESULT AFTER FIX
---------------------------------------------------
TESTS: {passed}/{total} passed
LINT: OK|FAILED

STATUS: fixed|partial|blocked
SUMMARY: {1 sentence}

---

Hard Rules

1. EVERY failed test is analyzed INDIVIDUALLY -- no blanket fixes
2. Coverage must NEVER decrease -- same or better coverage after fix
3. No anti-patterns (see above) -- better STATUS: blocked than lazy fix
4. On REGRESSION: fix code, not test -- the implementation has the bug
5. On INTENTIONAL_CHANGE: adjust test with same depth -- new values, not fewer assertions
6. When in doubt: assume REGRESSION -- better to fix code than weaken test
7. Always run ALL tests after fix -- not just previously failed ones
8. Max 3 iterations -- then blocked with concrete report why

Learnings

(populated by /learn after task cycles)