メニュー
Run FactHarbor post-change validation on benchmark families and analyze results. CAUTION — submits real analysis jobs. Use after pipeline or prompt changes to verify no regressions.
Codex または Claude でインストール この Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。
SOC 職業分類に基づく
Keep repository documentation lean, clear, and readable. Use automatically before adding a new document, substantially expanding or rewriting Markdown/xWiki content, adding explanatory/background sections, adding FAQs/glossaries/templates, introducing repeated framing, or reviewing a documentation diff for clutter, filler, duplication, over-broad prose, stale links, or readability problems.
MUST be used automatically for any FactHarbor bugfix, regression fix, failing test/build fix, failed-validation recovery, or defect repair — BEFORE editing code. Decides whether the best fix is reverting or amending a previous change, quarantining/deleting obsolete code, or adding new code. Also use when about to add a fallback, flag, retry, helper, guard, diagnostic, or parallel mechanism — in response to a failure or proactively (even with no failing test) — or when reviewing a diff for workaround stacking / additive repair drift.
Analyze local job reports from the most recent build/commit (HEAD by default), detect quality issues, and propose AGENTS.md-compliant fixes. Scopes to HEAD by default, or to specific job URLs / job IDs / commit / input slug passed as argument. Uses RAG (static register + handoffs + machine-readable benchmark expectations, plus dynamic follow-up scans) and runs an adaptive multi-agent debate before emitting recommendations. Complements `/prompt-diagnosis` (which is narrower — prompt provenance only) by covering evidence, boundaries, verdict reasoning, and warning severity end to end.
Structured adversarial debate on any proposition. Runs an intake/structural audit first, then spawns pipeline-mirroring roles (Advocate, Challenger, Reconciler) with optional Consistency Probes and Validator. Use when a decision needs adversarial pressure before committing — root-cause attribution, architecture choices, fix selection, design tradeoffs. Other skills invoke /debate for their adversarial synthesis steps. Adapts role count and model tiers to proposition complexity.
Consolidate all documentation in Docs/WIP/. Investigates implementation status of WIP topics using helper agents when the tool supports delegation, ensures all open items are well-described and present in the backlog, updates and cleans up backlog and status files, and moves historical/completed content to the Archive. Use this skill whenever the user says "consolidate WIP", "clean up WIP", "tidy WIP docs", "audit WIP", or asks to update the backlog from WIP — even if phrased differently. Note: this skill covers Docs/WIP/ only. To clean up the entire Docs/ tree use docs-update.
Analyze debug-analyzer.log and run tests to identify pipeline issues after a change. Standard post-change check per Captain's protocol.
| name | validate |
| description | Run FactHarbor post-change validation on benchmark families and analyze results. CAUTION — submits real analysis jobs. Use after pipeline or prompt changes to verify no regressions. |
| allowed-tools | Bash Read |
| disable-model-invocation | true |
ultrathink
Run validation for change: $ARGUMENTS
Step 1 — Understand what changed:
Run git log --oneline -5 and git diff HEAD~1 --stat to identify the scope of recent changes.
Before submitting jobs, AGENTS.md §Live Job Submission Discipline applies: commit relevant source/prompt/config changes, refresh/reseed/restart affected runtime state as needed, and monitor live jobs for failures, stale state, warnings/errors, and delays.
Step 2 — Run validation batch:
From the project root run:
npm run validate:run -- post-$ARGUMENTS
This submits 16 benchmark families and writes JSON summaries to test-output/validation/.
Step 3 — Compare to baseline (if a prior run exists):
List available runs: ls test-output/validation/
If a baseline exists, run: npm run validate:compare -- <baselineDir> <newDir>
Step 4 — Analyze results: Apply the Captain's principle: "improve the system not the data" — all fixes must be generic, never input-specific.
Pay particular attention to these priority families (Captain's benchmark protocol):
For each regression: identify which pipeline stage and which recent change caused it. For each improvement: confirm it is generic and not overfitted to these inputs. Recommend next steps.
