| name | forge-verification |
| description | WHEN: About to claim implementation is complete or working. HARD-GATE: Run verification, see output, THEN claim success. Never "should pass", "confident", "should work". |
| type | rigid |
| version | 1.0.1 |
| preamble-tier | 3 |
| triggers | ["run verification","verify implementation","check implementation is complete"] |
| allowed-tools | ["Bash","Read"] |
Verification Before Completion
Iron Law: Evidence before assertions. Always.
Anti-Pattern Preamble: Why Agents Skip Verification
| Rationalization | The Truth |
|---|
| "I'm confident the code works, the logic is sound" | Confidence is not evidence. Production requires proof from actual test runs. |
| "The previous test run passed, this shouldn't have changed" | Previous runs prove nothing about current state. Dependencies, cache, timing all matter. |
| "The changes are small, I should verify manually" | Manual spot-checks miss edge cases. Automated verification is the only complete record. |
| "Unit tests pass, that's sufficient proof" | Unit tests verify components in isolation. Full verification tests integration, real conditions, end-to-end flows. |
| "It should work based on my understanding of the code" | Understanding is subjective. Output is objective. Never substitute belief for evidence. |
| "Verification takes too long, I'll assume it passes" | The time cost of verification is negligible vs. the cost of undetected failures in production. |
| "The code review looked good, that's verification enough" | Code review is about correctness; verification is about observable behavior. They are orthogonal. |
| "I tested locally, it works there" | Local testing != production verification. Environment differences, ordering, concurrency, state matter. |
| "These are boring infrastructure tests, probably fine" | Boring tests catch boring bugs that break production in boring ways. Skip none. |
| "I can just describe what passed instead of showing logs" | Logs are the only source of truth. Descriptions are filtered through human interpretation. |
| "Running verification a second time would just duplicate effort" | Running verification confirms repeatability. Flaky tests need investigation. Do not skip. |
Iron Law
RUN THE VERIFICATION COMMAND AND OBSERVE THE OUTPUT BEFORE CLAIMING SUCCESS. CONFIDENCE IS NOT EVIDENCE — OBSERVED OUTPUT IS EVIDENCE.
Cross-cutting: brain codebase scans
When work depends on ~/forge/brain/products/<slug>/codebase/ (council, tech plans, eval paths): if SCAN.json exists, run python3 tools/verify_scan_outputs.py <codebase-dir> and require exit 0 (retry up to 3× with 1s delay before claiming the scan is load-bearing). The forge_scan.py CLI already verifies after each run unless FORGE_SCAN_SKIP_VERIFY=1. Treat verify failure like missing tests — no success claims until the brain tree is whole or the gap is explicitly waived in writing.
When a task-id has ~/forge/brain/prds/<task-id>/tech-plans/*.md and you are about to claim plans are merge-ready or REVIEW_PASS: run python3 tools/verify_forge_task.py --task-id <id> --brain ~/forge/brain --strict-tech-plans (or python3 tools/verify_tech_plans.py alone) and require exit 0 — see docs/forge-task-verification.md. Teams that hit structural PASS / semantic thin slips should add --strict-0c-inventory in CI (inventory GAP rows + required citations when Confluence mirror, touchpoints/, or QA CSV exist). This catches missing Section 0c anchors and misplaced ### 1b.2a; it does not replace reading prd-locked.md end-to-end, but it blocks the common “chat-only self-review” slip.
Red Flags — STOP
If you notice any of these, STOP and do not proceed:
- Agent says "should pass", "confident it works", or "looks correct" — These are beliefs, not evidence. STOP. Run the verification and show actual output before any claim of success.
- Verification output is summarized in words instead of shown as logs — Summaries hide failures. "Tests passed" is not evidence. STOP. Show the raw test runner output.
- Test count drops between runs without explanation — Tests are being silently skipped or filtered. A lower count is not a better result. STOP. Investigate why tests disappeared.
- Agent runs only a subset of tests ("these are the relevant ones") — Selective verification misses cross-cutting regressions. STOP. Run the full suite or explicitly justify each exclusion.
- Infrastructure is unreachable and verification is skipped — "Can't connect to DB" means the test ran in an invalid environment. STOP. Fix infrastructure and re-run from scratch.
- A "flaky" test is dismissed without investigation — Flakiness is a real bug. A flaky test means the system has nondeterministic behavior. STOP. Investigate and fix before claiming pass.
- Verification is claimed complete but checklist items are unmarked — Partial verification is not verification. STOP. Every checklist item must be independently confirmed.
Detailed Workflow
Phase Authority Check (before running any verification command):
Read ~/forge/brain/prds/<task-id>/conductor.log tail to confirm the current phase:
- Verification during TDD RED phase: conductor.log should show
[P4.0-TDD-RED] logged for the repo (tests written and confirmed failing — no production feature code yet)
- Verification for post-dispatch TDD: conductor.log should show
[P4.1-DISPATCH] for the relevant repo (implementation started, not yet eval)
- Verification for eval: conductor.log should show
[P4.0-SEMANTIC-EVAL] or [P4.1-DISPATCH] complete
- Verification for pre-PR: conductor.log should show
[P4.4-EVAL-PASS] before forge-verification is used as final gate
Do not claim verification results as eval evidence if conductor.log shows the task is still in a TDD phase ([P4.0-TDD-RED] present, [P4.1-DISPATCH] absent).
Staleness Check (run FIRST, before any verification work)
Before beginning a new verification run, check whether a PASS verdict already exists for the current commit and is less than 7 days old. If so, you may skip re-running and cite the existing evidence — but only if no code has changed since that commit.
TASK_ID="${FORGE_TASK_ID:-unknown}"
BRAIN="${FORGE_BRAIN:-${FORGE_BRAIN_PATH:-$HOME/forge/brain}}"
VERDICT_FILE="$BRAIN/prds/$TASK_ID/review-verdicts.jsonl"
CURRENT_COMMIT=$(git rev-parse HEAD 2>/dev/null || echo "")
NOW_EPOCH=$(date -u +%s)
SEVEN_DAYS=$((7 * 24 * 3600))
if [ -f "$VERDICT_FILE" ] && [ -n "$CURRENT_COMMIT" ]; then
MATCH=$(grep "\"$CURRENT_COMMIT\"" "$VERDICT_FILE" | grep "\"PASS\"" | tail -1)
if [ -n "$MATCH" ]; then
TS=$(echo "$MATCH" | sed 's/.*"timestamp":"\([^"]*\)".*/\1/')
ENTRY_EPOCH=$(date -u -d "$TS" +%s 2>/dev/null || date -u -j -f "%Y-%m-%dT%H:%M:%SZ" "$TS" +%s 2>/dev/null || echo 0)
AGE=$(( NOW_EPOCH - ENTRY_EPOCH ))
if [ "$AGE" -lt "$SEVEN_DAYS" ]; then
echo "VERIFICATION PASS (cached — verified $(( AGE / 3600 ))h ago for commit $CURRENT_COMMIT)"
echo "Stop here. Re-run /forge-verification explicitly to force a fresh check."
else
echo "WARNING: Last PASS for this commit is $(( AGE / 86400 )) days old — re-verification required."
fi
fi
fi
If the output prints "VERIFICATION PASS (cached…)" — stop. If it prints nothing or the warning — proceed with the full workflow below.
Identify What to Verify
- Input: Task completed, code written, tests written
- Action: Enumerate all verification targets (unit tests, integration tests, e2e scenarios, performance baselines)
- Output: Verification checklist (tasks below depend on this list)
Execute Verification
For each verification target:
-
Run the command exactly as written (no shortcuts, no assumptions)
# Example: full test suite
npm test -- --coverage --verbose
# Example: integration suite
cargo test --release -- --test-threads=1
# Example: e2e eval
/invoke forge-eval-gate
-
Observe and capture output
- Take note of test framework output (PASS/FAIL counts)
- Capture error messages (stack traces, assertions)
- Note timing and performance metrics
- Record environment info (Node version, database state, network availability)
-
Do NOT interpret — just record facts:
- ✗ "This probably works"
- ✓ "9/9 tests passed, 0 skipped, 0 flaky"
Surface-Specific Verification Commands
Run the command appropriate for the surface being verified. Capture full output — do not paraphrase.
API / Backend (HTTP):
curl -f -s -o /dev/null -w "%{http_code}" http://localhost:<PORT>/health
curl -X POST http://localhost:<PORT>/api/<endpoint> \
-H "Content-Type: application/json" \
-d '{"key": "value"}' | jq .
Web Frontend:
curl -f http://localhost:<PORT>/ | grep -o '<title>[^<]*</title>'
node -e "const CDP=require('chrome-remote-interface'); CDP(async c=>{ await c.Page.navigate({url:'http://localhost:<PORT>'}); const s=await c.Page.captureScreenshot(); require('fs').writeFileSync('verify-screenshot.png', Buffer.from(s.data,'base64')); process.exit(0); })"
Android:
adb shell dumpsys activity activities | grep -i "<package>"
adb logcat -d -s "<TAG>" | tail -50
adb shell am start -n "<package>/<activity>" && echo "Launch OK"
iOS:
xcrun simctl list devices | grep "Booted"
xcrun simctl launch booted <bundle-id> && echo "Launch OK"
Database:
psql $DATABASE_URL -c "\dt" | grep "<expected_table>"
psql $DATABASE_URL -c "SELECT version FROM schema_migrations ORDER BY version DESC LIMIT 1;"
Cache (Redis):
redis-cli -h localhost -p 6379 PING
redis-cli -h localhost -p 6379 INFO server | grep redis_version
HARD-GATE: Paste the actual command output in your verification report. "It works" without output is not verification.
Validate Against Expectations
Claim Success (Evidence-Backed)
-
Output statement format:
VERIFICATION PASS
- Test suite: 47/47 passed, 0 skipped
- Coverage: 92% lines, 88% branches
- E2E eval: all 8 scenarios green
- Performance: p95 latency 145ms (SLA: 200ms)
Evidence: [link to full test output log]
-
Persist the verdict to the review dashboard (REQUIRED on every PASS):
TASK_ID="${FORGE_TASK_ID:-unknown}"
BRAIN="${FORGE_BRAIN:-${FORGE_BRAIN_PATH:-$HOME/forge/brain}}"
VERDICT_FILE="$BRAIN/prds/$TASK_ID/review-verdicts.jsonl"
mkdir -p "$(dirname "$VERDICT_FILE")"
COMMIT=$(git rev-parse HEAD 2>/dev/null || echo "unknown")
TIMESTAMP=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
echo "{\"commit\":\"$COMMIT\",\"timestamp\":\"$TIMESTAMP\",\"verdict\":\"PASS\",\"task_id\":\"$TASK_ID\"}" >> "$VERDICT_FILE"
Edge Cases & Fallback Paths
Case 1: Flaky Tests (Intermittent Failures)
- Symptom: Test passes on 1st run, fails on 2nd run
- Do NOT: Ignore flakiness, run once and claim success
- Action:
- Run verification 3 times in succession
- If all 3 pass: document flakiness in brain (decision link)
- If any fail: investigate timing, isolation, state leaks
- Fix root cause or quarantine test (don't hide it)
- Re-run 3x again after fix
Case 2: Verification Infrastructure Down (CI/CD, test runner)
- Symptom: "Cannot reach test server", "Docker daemon not running"
- Do NOT: Claim verification passed because "it should work"
- Action:
- Restore infrastructure (restart service, reconnect VPN, etc.)
- Re-run verification after restoration
- If cannot restore: escalate as BLOCKED
- Document the infrastructure dependency in brain
Case 3: Test Timeouts or Resource Exhaustion
- Symptom: Tests hang, OOM, or exceed timeout
- Do NOT: Reduce timeout, remove resource-heavy tests
- Action:
- Investigate: is this a real bottleneck or a test environment issue?
- If test is too slow for product: refactor code or test
- If environment is too constrained: upgrade infrastructure
- Do not reduce verification rigor to fit constraints
Case 4: Verification Passes but Contradicts Earlier Assumptions
- Symptom: "Tests pass but I thought this would fail", or "More tests pass than I expected"
- Do NOT: Assume you miscounted or misunderstood earlier
- Action:
- Re-examine the actual test code (not names, not descriptions)
- Verify the test environment hasn't changed unexpectedly
- Update your mental model based on evidence
- Link this decision in brain (why assumption was wrong)
Case 5: Partial Verification (Some Suites Pass, Others Blocked)
- Symptom: Unit tests pass, but integration tests can't run (DB unavailable)
- Do NOT: Claim PASS based on unit tests alone
- Action:
- Restore the blocking constraint (DB, service, API)
- Re-run all suites
- Output is PASS only if ALL relevant suites pass
- If blocking constraint cannot be restored: escalate to BLOCKED
Verification Checklist
Before claiming success, complete all items:
Additional Edge Cases
Edge Case 1: Verification Tool Is Broken (False Positives, Crashes, Invalid Output)
Situation: Verification tool itself is buggy or invalid. It crashes, produces nonsensical output, or has false positives.
Example: Test framework hangs on invalid assertion syntax. Verification script crashes mid-test. Coverage report shows 200% coverage (mathematically impossible).
Do NOT: Trust output from broken tools or claim verification passed based on suspect output.
Action:
- Identify: is the tool broken or is the code broken?
- Run tool on known-good code (previous commit, simple example)
- Does tool work on that code? Then your code is broken (not the tool)
- Does tool still fail on known-good code? Then tool is broken
- If tool is broken:
- Do NOT use it to verify your code
- Escalate as BLOCKED (verification tool unavailable)
- Find alternate verification method or wait for tool fix
- Document tool failure in brain
- If code is broken:
- Fix code
- Re-run verification
- Proceed normally
- Never ship based on verification from broken tools
Edge Case 2: Verification Output Is Unreadable (Logs Too Large, Format Unclear, Summary Insufficient)
Situation: Verification runs, produces output, but output is hard to parse or understand.
Example: Test output is 50MB log with no summary. Test names are cryptic (test_0001, test_0002). Output format differs from expected (JSON instead of TAP, etc.).
Do NOT: Summarize output in words ("looks like it passed"). Output must be human-readable.
Action:
- Examine raw output:
- Is it parseable? (valid JSON, TAP format, etc.)
- Is it complete? (no truncation, all assertions visible)
- Is it meaningful? (test names describe what they test)
- If output is too large:
- Filter to relevant sections (failures, summary, stats)
- Or re-run with verbose=false, summary-only
- Document what was filtered and why
- If output format is unclear:
- Re-run with standard format (TAP, JSON, human-readable)
- Or write parser to convert to readable format
- If output is still unclear:
- Escalate as BLOCKED (verification output uninterpretable)
- Tool maintainers must improve output readability
- Document readable output in brain (with context)
Edge Case 3: Verification Passes but Humans Spot Issues (Tool Missed Bugs)
Situation: Verification suite passes all tests, but human testing or production finds bugs. Tool missed real issues.
Example: All unit tests pass. Integration tests pass. Manual testing discovers "user can't export data" feature doesn't work.
Do NOT: Ignore human findings. Verification tools are not infallible.
Action:
- Investigate: why did verification miss this?
- Was the scenario not tested? (missing test case)
- Was test written incorrectly? (false positive)
- Was behavior regression not caught? (old test, changed code)
- Add test case:
- Write test that reproduces the human-found issue
- Verify test fails on current code
- Fix code
- Verify test passes
- Re-run all verification (not just new test)
- Document in brain:
- What was missed by verification
- Why test was missing
- Action taken (added test, fixed code)
- Review verification coverage:
- Are there other scenarios humans would test that verification misses?
- Add them to verification suite
- Key lesson: Verification catches what you test. Humans catch what you didn't test.
Output: PASS (with evidence from working tools) or BLOCKED (verification tool broken, output unreadable, infrastructure down, unresolvable flakiness)
Edge Case 4: Test Count Drops Between Runs (Silent Test Deletion)
Symptom: Verification passes on run N (47 tests, all pass). On run N+1 after a code change, 44 tests pass — but the implementer claims "nothing was deleted." The 3 missing tests were silently removed.
Do NOT: Accept a passing run if the test count has decreased without a documented reason.
Action:
- Compare test counts between the prior run and the current run — flag any decrease
- Ask the implementer: which tests were removed, and why? Require a commit that shows the deletion
- If tests were removed because they were "duplicates" or "flaky," require evidence before accepting the removal
- If tests were removed because the feature was descoped, verify against the spec that the descope was intentional
- A decrease in test count is a red flag, not a neutral fact — treat it as BLOCKED until explained
- Escalation: BLOCKED if test count decreases without justification committed to git
Edge Case 5: Verification Passes Locally but CI Shows Different Results
Symptom: Local npm test shows 47/47 passing. CI pipeline shows 41/47 with 6 failures. The implementer argues "it works on my machine."
Do NOT: Accept local verification as evidence when CI contradicts it.
Action:
- CI is authoritative — it runs in a clean environment without local dev state
- Identify the failing tests in CI output — they often fail due to environment differences (missing env vars, different Node version, different file permissions)
- Require the implementer to reproduce CI failures locally:
CI=true npm test or use the same env vars as CI
- Do not approve or merge until CI passes — "works locally" is not verification evidence
- Common CI-only failures: hardcoded localhost references, missing
.env entries, implicit dependency on global tools not in CI
- Escalation: BLOCKED until CI passes; NEEDS_COORDINATION if CI infrastructure itself is broken (not the code)
Checklist
Before claiming a task complete:
Post-Implementation Checklist