メニュー
Use when an LLM or agent emits decisions or claims and you need to turn fuzzy output into a verifiable, regression-protected signal.
Codex または Claude でインストール この Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。
SOC 職業分類に基づく
Use when shipping or handing off code — treat docs as a first-class deliverable — present-tense, diagrammed-as-code, verified against reality, reader-tested.
Use when making a load-bearing or security-relevant claim, or reusing inherited or unfamiliar code — treat recalled facts and unread code as unverified until re-grounded against the live source, and disposition every dangerous surface rather than eyeballing it.
Use when containing untrusted or agent-generated execution, sandboxing a worker with write/exec tools, or judging whether a guard is actually a security boundary.
Use when building automation that edits, tests, or deploys itself — an agent that regenerates its own code, a self-updating pipeline, any loop that modifies the thing running it.
Use when the human's request rests on a premise you can check, contradicts evidence or a recorded principle, or proposes a direction you have not independently weighed — weigh the strongest alternative before taking a stance, challenge once with evidence before complying; their decision still wins.
Use when a live result contradicts the hoped-for story or a metric could be gamed — state what really works and never fake a green.
| name | grounded-verifiable-gates |
| description | Use when an LLM or agent emits decisions or claims and you need to turn fuzzy output into a verifiable, regression-protected signal. |
Never let a model's self-assessment be the decision. Force every claim to cite source text that literally matches, compute the outcome with a pure tested function over only the grounded claims, and protect that function with an eval harness that runs the real production path and fails CI on regression.
Reach for this whenever a model's output influences what happens next: a judge that approves or flags, an extractor that fills a record, a classifier that routes, an agent that decides to act. The higher the cost of a wrong, confidently-stated answer, the more this applies.
Red-flag thoughts that mean STOP and apply this skill:
ungrounded.min_recall 0.70, max_over_flag 1.0, min_grounding 0.80.An LLM's stated confidence and its actual correctness are only loosely correlated, and a fluent wrong answer is more dangerous than an obviously broken one. If the model's own number is the gate, a hallucination flows straight to the outcome and you have no tripwire. Grounding gives you a mechanical filter — unverifiable claims are inert. A pure verdict function gives you something you can unit-test and reason about. An eval harness on the real path gives you regression protection that actually reflects production; one on a re-implementation gives you false comfort that breaks silently when the prompt or pipeline drifts. And calibrated, version-controlled gates convert "it seemed fine in the demo" into a signal that blocks a merge. Skip these and quality becomes invisible until a user hits the failure — at which point you're debugging vibes, not a contract.
On the project this was distilled from — a hexagonal, human-in-the-loop AI agent (an LLM judge reviewing documentation, shipped to a cloud runtime) — docs/ARCHITECTURE.md states: "A finding whose quote is not literally present — a hallucination — is marked ungrounded and is ignored by the decision gate …" (the source continues "… and excluded from the proposal prompt"). The eval harness was built to reuse the production evaluation entrypoint and the domain decide gate unchanged, so, per docs/EVALUATION.md, "it measures exactly the judge the agent runs in production" — fixtures normalized through the same text pipeline as live input. The CI gates lived in evals/golden/labels.yaml beside the golden labels as config: min_recall_deterministic: 0.70, max_over_flag_good: 1.0, min_grounding_rate: 0.80, and the run exited non-zero on breach. Crucially, the team reframed precision honestly: because the model graded its own output, precision was reported as a bound with the bias disclosed, while recall and grounding — independent of the judge's self-opinion — carried the headline. You can apply all of this without knowing that project: substring-check quotes, gate on grounded findings only, eval the real code path, pin thresholds in CI, calibrate against a labeled run, and be honest about which numbers the model could game.
This skill's harness is its own enforcement: a CI job that runs the deterministic gate over the fixed corpus on every change and fails on regression. What a machine additionally holds: grounding invariants as assertions (every cited id exists, every span matches its source), thresholds living in config so a calibration change is a reviewable diff, and fail-closed semantics — an empty corpus or a skipped eval job is a FAIL, never a quiet green. A gate's own deny/fail branch carries its own regression test, mutation-checked — flip the fail-closed return and the suite must go red; a still-green suite means the deny path is unguarded and can silently regress to fail-open.