메뉴
WHEN: Phase 4.4 needs a final pass/fail verdict from qa/semantic-eval-manifest.json outcome + qa/semantic-eval-run.log (semantic CSV execution). Renders GREEN/RED/YELLOW.
Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.
SOC 직업 분류 기준
WHEN: You need to detect performance regressions before merge — TTFB, response time, bundle size. Run as part of forge-eval-gate or standalone before raising a PR.
WHEN: A decision is being superseded, deprecated, or has aged out. Archive it without deletion — marks as warm→cold→archived with full audit trail.
WHEN: You are writing a new decision, superseding an old one, or querying relationships across decisions. Create semantic edges between decisions and link concepts across products/projects/time.
WHEN: You are about to make a decision and need to check if prior art or past learnings exist. Recall decisions, patterns, and gotchas from the brain before proceeding.
WHEN: You need to trace the full provenance of a specific decision — who made it, when, why, and what alternatives were considered. Shows why, when, by whom, evidence, alternatives, outcome.
WHEN: You need to record a decision, lock a spec, log an eval run, or document learnings in the brain.
| name | eval-judge |
| description | WHEN: Phase 4.4 needs a final pass/fail verdict from qa/semantic-eval-manifest.json outcome + qa/semantic-eval-run.log (semantic CSV execution). Renders GREEN/RED/YELLOW. |
| type | rigid |
| requires | ["brain-read"] |
| version | 2.0.0 |
| preamble-tier | 3 |
| triggers | ["judge eval results","get eval verdict","final eval verdict","eval pass or fail"] |
| allowed-tools | ["Bash","Read","Write"] |
Forge machine-eval evidence is only the semantic path: qa/semantic-eval-manifest.json and qa/semantic-eval-run.log after qa-semantic-csv-orchestrate / tools/run_semantic_csv_eval.py with stack up (eval-product-stack-up). Host tools map qa/semantic-automation.csv Surface rows to eval-driver-*** skills per docs/semantic-eval-csv.md.
No human, agent, or rationalization overrides a RED verdict.
THE JUDGE NEVER ISSUES GREEN WITHOUT A RECORDED outcome IN qa/semantic-eval-manifest.json AND CONSISTENT qa/semantic-eval-run.log LINES FOR THE PHASE 4.4 RUN — outcome pass ⇒ GREEN; fail ⇒ RED; yellow ⇒ YELLOW. IF outcome IS ABSENT OR CONTRADICTS THE LOG, VERDICT RED (INCOMPLETE_DATA).
IF manifest outcome = pass BUT semantic-eval-run.log contains CONTEXT_GAP or BLOCKED_DEPENDENCY entries → verdict is YELLOW (incomplete execution, not full pass).
IF manifest outcome = fail AND run.log shows all failures are BLOCKED_DEPENDENCY (no CODE_BUG) → verdict is YELLOW with note "dependencies unresolved — not a product code failure".
When drivers never ran: Do not invoke this skill to manufacture GREEN. If automation was not executed, verdict is RED or invoke qa-pipeline-orchestrate static-only paths per that skill.
| # | Rationalization | The Truth |
|---|---|---|
| 1 | "Most steps passed, one failure is minor" | A single critical-path failure is RED. Partial pass is not pass. |
| 2 | "I'll skip eval-judge because there are no YAML scenarios" | Invalid. Evidence is manifest + log — always run this skill when Phase 4.4 claims completion. |
| 3 | "The manifest says pass but the log shows FAILED" | RED — inconsistent evidence (INCOMPLETE_DATA or failure — treat as RED until reconciled). |
| 4 | "Manifest says pass, so it's GREEN" | If run.log has CONTEXT_GAP or BLOCKED_DEPENDENCY entries, execution was incomplete. Incomplete execution is YELLOW, not GREEN — it proves nothing about the skipped steps. |
| 5 | "All BLOCKED_DEPENDENCY means RED" | BLOCKED_DEPENDENCY is a dependency resolution failure, not a product code failure. Classify YELLOW and note which upstream steps must be fixed first. |
~/forge/brain/prds/<task-id>/qa/semantic-eval-manifest.json — includes outcome: pass, fail, or yellow after the stack-up run.~/forge/brain/prds/<task-id>/qa/semantic-eval-run.log — append-only JSON lines per step for the same run.outcome: pass and log has no contradictory hard failures for required steps AND no CONTEXT_GAP or BLOCKED_DEPENDENCY entries → GREEN.outcome: pass BUT log contains CONTEXT_GAP or BLOCKED_DEPENDENCY entries → YELLOW (incomplete execution — skipped steps unverified).outcome: fail AND log contains at least one CODE_BUG failure → RED (self-heal per Phase 4.5).outcome: fail AND all failures in log are BLOCKED_DEPENDENCY (zero CODE_BUG entries) → YELLOW, note "dependencies unresolved — not a product code failure".outcome: yellow → YELLOW (document gaps before merge).outcome or empty/invalid manifest after claimed run → RED, reason INCOMPLETE_DATA.Emit structured YAML or markdown with:
verdict: GREEN | RED | YELLOWtimestamp: ISO-8601evidence: pointers to manifest path, log path, and any cited step ids from semantic-automation.csvreason: for RED/YELLOW — cite manifest outcome and log linesRecord RED and YELLOW in brain via brain-write when your workflow requires an audit id.
semantic-eval-run.log contains FAILED lines that contradict pass.qa-semantic-csv-orchestrate run.qa/semantic-eval-run.log (no silent skips).PASS, FAIL, BLOCKED_DEPENDENCY, SKIPPED (with reason), CONTEXT_GAP — no unclassified results.qa/semantic-eval-manifest.json written with kind: semantic-csv-eval and a non-placeholder outcome.python3 tools/verify/verify_forge_task.py --task-id <id> --brain <brain> exits 0.| Related | Role |
|---|---|
qa-semantic-csv-orchestrate | Produces manifest + run log. |
docs/semantic-eval-csv.md | CSV schema and conductor [P4.0-SEMANTIC-EVAL]. |
eval-product-stack-up | Stack before execution. |
eval-driver-* | Host execution mapping from Surface — not a separate YAML scenario layer. |
self-heal-locate-fault | Downstream on RED — use semantic-eval-run.log JSON lines as primary evidence. |
forge-eval-gate | Parent workflow containing this judge. |