| name | no-overclaim-merge-gate |
| description | Run at the human-at-the-PR end-of-line, BEFORE approving/merging or before recommending a merge in this multi-agent repo — and whenever a session reports "N PRs landed", "throughput", "merged per hour", "ready to merge", "gate is green", "training run done", or a benchmark/uplift number. Many concurrent advisors (Claude/Copilot/GLM/human) optimize for merged-PRs-per-hour; this is the feature-factory overclaim trap. This skill re-anchors the merge decision on verified OUTCOMES (real gate receipts, decontam CLEAN, judge-independence not lexical-fallback, canClaimAGI still false) and reasserts that a human holds merge authority. Use even when the PR is green and a prior session already said "safe to merge" — green checks are necessary, not sufficient.
|
| metadata | {"short-description":"End-of-line PR inspection: verify outcomes (gate receipts, decontam, judge independence, no AGI claim); human merges"} |
No-overclaim merge gate (the human end-of-line)
The unit of value in this repo is a verified outcome, never a merged PR. A tree full of
green PRs that quietly loosened a threshold, reused a lexical judge as two "families", trained on a
decontam-dirty pack, or nudged a candidate toward "validated" is negative value — it spends the
repo's only real asset, its measurement credibility. This skill is the deliberate, unhurried
inspection a human runs at the point of merge. It complements git-discipline (merge mechanics —
stale? blocked? duplicated?) and ci-artifact-drift (regenerate the artifacts); this skill governs
whether the claim the merge encodes is earned.
Merge authority is a human's, always. An agent may verify, stage, and recommend; it does not
self-approve the merge of a claim-bearing change. If any check below is not green-on-evidence, the
agent stops and hands the human a plain GO/NO-GO with the receipts — it does not merge to keep a
throughput number up.
0. Reframe before you look (defeat the factory metric)
Ask out loud: what does merging this PR let us claim that we could not claim before? If the answer
is "nothing — it's tooling/refactor/docs", the bar is low: skip to §4 (still confirm no claim crept
in). If the answer is a status change (a number, a GO/NO-GO, "validated", a new RESULTS row, a
promoted adapter), every section below is mandatory. Do not measure yourself by PRs/hour; a
correct NO-GO that stays unmerged is a win.
1. The measurement contract must pass on THIS head — with receipts
Green CI on the PR page can be stale (main moved) or cancelled. Re-run the contract locally against
the PR's current head in your own throwaway worktree off origin/main:
make claim-check
Read the receipts, not just the exit code:
claim_gate.py --prefix … shows GO on real numbers — not a lowered threshold. A flip to
NO-GO is a real result: it is labelled candidate and gets a agi-proof/failure-ledger.md entry.
Never lower a gate bar to turn a PR green (ci-artifact-drift §1).eval_stats — the uplift has power/MDE + bootstrap CI, not a single lucky seed.
2. Decontam must be CLEAN — no eval/holdout prompt in any training row
python tools/assert_decontam.py
A dirty pack makes every downstream number meaningless. CLEAN is a merge precondition for anything
that touches a pack or a benchmark; a leak is NO-GO, not a "fix later".
3. Judge independence must be real, not lexical-fallback
A "≥2 judge families" claim is void if the judges collapsed to the lexical fallback — that is one
non-independent judge wearing two hats (see the judge-lane-independence-gate memory). Confirm the
independence receipt, not just the family count:
python tools/run_independence_eval.py
If a keyed judge lane silently degraded to lexical matching, the independence claim is NO-GO.
4. canClaimAGI stays false — no claim crept in
This repo makes no capability/uplift/AGI claim. lint_claims.py (inside make claim-check) is
the fail-closed guard; still eyeball the diff for prose that upgrades a verdict (candidate →
validated, "measures" → "proves", a headline number without its CI/seeds/families). canClaimAGI
must remain false everywhere it appears. If the PR's copy overclaims, that is the trap this whole
skill exists to catch — NO-GO until the language is brought back to the measurement ceiling.
5. The change was actually produced under the invariants
- One-GPU-job invariant: any cert/train receipt behind this PR must have run without a competing
heavy GPU load stomping it (a stray load silently corrupts an in-flight run —
spark-cluster-ops).
Cross-check the run against SESSION-COORDINATION.md (the live CLAIM/OWNS/DONE ledger) and, for
training receipts, the cluster dashboard / TrainWatch history — a "done" train that never shows
a real loss/ETA/progress trace is unverified, not done.
- Isolated changes:
git diff --cached --stat shows only this PR's files — the shared worktree
is often contaminated by other advisors' WIP (git-discipline §4).
6. Verdict — hand the human a receipt, let the human merge
Produce a one-screen GO/NO-GO:
- Each gate: PASS + the receipt line (or FAIL + why). No "should be fine".
- What the merge would newly let us claim, stated at the measurement ceiling (candidate/GO/NO-GO/
validated — never above).
- Open caveats → a
agi-proof/failure-ledger.md line for anything measured-but-not-fully-validated
(single-seed, no third-party, self-authored cases).
Then stop: the human performs the merge (main-protection requires fast + ci-complete green
and all review threads resolved — git-discipline). An agent never merges a claim-bearing PR to
keep a throughput metric moving.
Red flags that force a NO-GO (any one)
- "Merged N PRs this session" offered as the accomplishment, with no outcome verified.
- A gate turned green after a threshold, seed set, or judge config was edited in the same PR.
- "Judges agree" without an independence receipt; "decontam passed" without the assert run on this head.
- A prose verdict stronger than its evidence; any
canClaimAGI: true or new capability/AGI claim.
- A training/cert receipt with no TrainWatch/dashboard trace or no
SESSION-COORDINATION.md owner.
