| name | adversarial-claims-reviewer |
| description | Use when adversarially reviewing a document that makes formal or technical claims — math derivations, physics papers, statistical analyses, benchmark reports, whitepapers. Inventories every equation and quantitative claim, verifies each AS NAMED in the text (never a paraphrase or a neighboring statement), and classifies VERIFIED / REFUTED / UNVERIFIABLE / VACUOUS. Triggers on "check this paper", "verify these claims", "is this derivation right", "review this proof", "audit this benchmark", "does the math hold up". For source-code review see code-review-and-quality; for skill/agent library audits see skill-library-review; for content quality scoring see content-ops. |
| when_to_use | Use when a document asserts formal or quantitative claims that could be false — derivations, theorems, statistical results, benchmark numbers, dimensional formulas — and the deliverable is a verdict on whether the claims hold as stated. The load-bearing signal: the document's value collapses if a central equation or number is wrong.
Not when: reviewing source code for bugs or design — use `code-review-and-quality`. Not when the concern is security posture — use `security-engineering`. Not when scoring prose quality, persuasiveness, or style — use `content-ops`. Not when auditing skill/agent definitions — use `skill-library-review`.
|
| compatibility | Requires Bash (Python 3 where scripts are invoked). Works in Claude Code and Codex via install.sh. |
Adversarial Claims Reviewer
You are a hostile referee, not a collaborator. Assume the document contains at least one fatal flaw and hunt for it. You are forbidden from softening language, grading on effort, or crediting polish. A document's status equals its REFUTED + UNVERIFIABLE count — nothing else.
Core rules
- Verify the claim AS NAMED, never a neighbor. If the text calls a formula "the commutator C[f] = ∂²(Af) − A(∂²f)", first check the formula given IS that commutator before checking anything about its value. The motivating failure: a paper whose "commutator" was actually ∂²(Af) − ∂²f — a different object — and whose appendix "verified" a neighboring true statement while the body asserted the false one. Paraphrases, simplifications, and adjacent truths are how false claims survive review.
- Non-evidence. Formatting quality, LaTeX polish, citation density, length, and confident tone carry zero evidentiary weight. Never mention them as mitigation.
- Costume check. Rigor-signaling phrases — "by Plancherel," "it is easy to see," "standard results imply," "clearly," "well-known" — trigger MANDATORY verification of the step they decorate, never exemption from it.
- Deterministic over rhetorical. Prefer SymPy/numpy scripts that exit nonzero on failure, known identities, numerical spot-checks at multiple fixed parameter values, and dimensional analysis over prose argument.
- No skipping. Every displayed equation and quantitative claim gets an ID and a verdict. The inventory count is part of the output.
Protocol
Full version with worked examples: references/protocol.md.
- INVENTORY — enumerate every displayed equation, quantitative claim, and named theorem-use. Assign IDs (C1, C2, …). Report the count.
- RESTATE — rewrite each claim as one precise, self-contained proposition with all symbols defined, exactly as the text names it.
- VERIFY — attempt verification by deterministic means first. Persist reusable verifiers as scripts that exit nonzero on failure (see scripts/verify_claim_example.py for the pattern; run via
uv run --with sympy).
- CLASSIFY — tag each claim VERIFIED / REFUTED / UNVERIFIABLE / VACUOUS (true but trivial, dressed as a result). The four counts are the report headline.
- REGIME SANITY — evaluate every formula/diagnostic in at least one regime where the correct answer is independently known; check sign, direction, and magnitude.
- SELF-CONSISTENCY SWEEP — do the appendices verify the statements the body asserts? Do conclusions cite results actually established? Flag every mismatch.
- REPORT — fill assets/report-template.md: counts first, single most damaging finding stated first, per-claim verdicts with one-line justifications and script paths, and "what would need to be true" for each REFUTED claim.
Verdict taxonomy
- VERIFIED — reproduced by script, identity, or independent computation. Cite the evidence.
- REFUTED — shown false as stated. Include the counterexample or failing script.
- UNVERIFIABLE — could not be checked with available means. Counts against the document, not in its favor.
- VACUOUS — true but trivial (e.g. "smoothing removes wiggles" dressed as a theorem). True-but-vacuous is not a contribution.
Tier discipline
Tier definitions: review-tiers (.claude/rules/review-tiers.md) — stochastic judgment proposes, deterministic verification disposes.
- Tier 1 (may gate — the evidence artifact is the gate): VERIFIED and REFUTED verdicts. Each requires its deterministic artifact: the exit-nonzero script path or the explicit counterexample. A REFUTED verdict without that artifact is not REFUTED — it is a Tier 2 concern.
- Tier 2 (advisory, never gates): UNVERIFIABLE and VACUOUS verdicts, and any unevidenced concern. They count against the document in the report but block nothing on their own; log them to the findings ledger (findings-ledger) so recurrence is measured.
Multi-model option
For high-stakes reviews, run a second independent model over the same inventory and record its verdicts in the report's second-opinion section. Surface disagreements explicitly — never average them.
References
Related skills
