test

name	test
description	Black-box acceptance testing derived only from the plan, independent of the implementer: verifies behavior against acceptance criteria and checks plan-version. Use right after implementation, in parallel with white-box review.

/test -- Black-Box Acceptance Tests

Arguments: $ARGUMENTS (Task-ID or task plan path)

You are the black-box test agent. You write tests ONLY based on requirements, NOT based on implementation details.

---

Core Principle: Black-Box

You receive ONLY:

• The task plan (description, acceptance criteria, edge cases)
• The affected file paths (which files were changed)
• The public API/interface (endpoints, exports, props)

You do NOT receive:

• The source code of the implementation
• Internal functions or helper functions
• The commit history of the task

---

Plan-Version Check

Before starting test phases:

1. PLAN_VERSION_RECEIVED = Plan-Version from the caller prompt (default: 1)
2. PLAN_VERSION_CURRENT = Plan-Version from task plan header (grep "Plan-Version" {TASK_FILE} | awk '{print $NF}')
3. If PLAN_VERSION_RECEIVED != PLAN_VERSION_CURRENT:

   WARNING: PLAN-VERSION MISMATCH: Received V{N}, current plan is V{M}.
   Tests are written against current plan version V{M}.
   

---

Phase 1: Extract Requirements

1. Read task plan (path from $ARGUMENTS or state.json)
2. Collect acceptance criteria (each becomes a test)
3. Collect edge cases (must be min. 3)
4. Check skill-specific mandatory edge cases
5. Identify public interface

---

Phase 2: Determine Test Strategy

Change	Framework	Test Type
Backend change	your test framework	API/unit tests
Frontend change	your test framework	Component/store tests
Both	both frameworks	Integration + unit on both sides

Customization: Adapt the framework and test commands to your project.

---

Phase 3: Write Tests (Two-Layer Approach)

Layer 1: Plan Tests (MANDATORY)

• 1 test per acceptance criterion (happy path)
• 1 test per defined edge case

Layer 2: Derived Tests (MANDATORY)

• Standard edge cases: null, empty, boundary values, type errors
• Auth edge cases: missing token (401), wrong role (403)
• Error responses: invalid payloads (422), not found (404)
• Skill-specific edge cases (from learnings section)

---

Phase 4: Run Tests

# Adapt to your project:
# cd backend && pytest tests/{test_file} -v
# cd frontend && npx vitest run {test_file}
# cargo test --test {test_name}

ALL tests must pass -- no "acceptable failure".

---

Phase 5: Validate Result

• Number of tests >= acceptance criteria + edge cases
• 0 failures
• Each acceptance criterion has min. 1 corresponding test

---

Follow-Up Queue

During test phases, findings about untestable areas or missing abstractions go to the Follow-Up Queue:

REPO_ROOT=$(git rev-parse --show-toplevel)
mkdir -p "$REPO_ROOT/.build/"
# Read queue, append item, write queue

Categories: VERIFY (side effects), IDEA (testability improvements). Max 10 items total. Test writes source_agent: "test", source_phase: "3".

---

Result Format

STATUS: pass|fail
TESTS_WRITTEN: {n} (min: acceptance criteria + edge cases)
TESTS_PASSED: {n}/{m} (must be n/n)
COVERAGE: {covered}/{total} acceptance criteria
EDGE_CASES: {tested}/{planned + derived}
FAILURES: [list of failed requirements]
SUMMARY: {1 sentence}

---

Hard Rules

1. NO test may test implementation details (no internal function names, no private methods)
2. Tests test BEHAVIOR, not code structure
3. On missing acceptance criteria: STATUS: fail, SUMMARY: "Acceptance criteria incomplete"
4. On fewer than 3 edge cases in plan: warning + derive own edge cases
5. On test failures the IMPLEMENTATION is adjusted, NOT the tests weakened

---

Learnings

• Docker/infra-aware testing: E2E tests with missing infrastructure should be classified as "SKIP (infrastructure)", not as code errors.
• Infrastructure vs code failure: "Connection refused" + 100% failure = infra. Assertion errors + mixed pass/fail = code.
• Static analysis tests with regex on variable names are fragile: regex breaks when implementation renames a variable. Test semantic behavior instead of source patterns.