quality-gate

name	quality-gate
description	Verify that paper claims match analysis outputs before submission.

Quality Gate: Paper ↔ Analysis Consistency

Cross-check every numerical claim in the paper against analysis output files. Reports only — never edits.

Input: $ARGUMENTS — path to the paper draft, or leave blank to auto-detect.

---

Step 1: Locate the Paper Draft

If $ARGUMENTS is provided, use that path. Otherwise glob for:

• manuscripts/**/*.tex
• manuscripts/**/*.qmd
• manuscripts/**/*.md

If multiple drafts are found, use request_user_input in Plan mode for a single-choice prompt when there are 2-3 clear candidates; otherwise ask conversationally. For the question UI:

• header: "Paper"
• question: "Multiple drafts found. Which manuscript should I check?"
• options: list up to 3 found files (label: filename, description: path and file size). If more than 3, ask conversationally instead.

---

Step 2: Extract Numerical Claims

Read the full manuscript and extract every quantitative claim:

• Coefficients and standard errors — e.g., "The effect is 0.23 (SE = 0.04)"
• Sample sizes — e.g., "N = 4,521 observations"
• Percentages and proportions — e.g., "42% of firms..."
• Means, medians, ranges — e.g., "average income of $45,000"
• Table and figure references — e.g., "Table 2 shows...", "as seen in Figure 1"
• p-values and significance statements — e.g., "statistically significant at the 1% level"

Record location (section, paragraph, line number if available) for each claim.

---

Step 3: Inventory Output Files

Glob for all output files:

• output/tables/**/*.tex — regression and summary tables
• output/tables/**/*.html — HTML versions
• output/figures/**/*.pdf, output/figures/**/*.png — figures
• output/**/*.rds, output/**/*.pkl, output/**/*.parquet, output/**/*.csv — saved objects

Build an inventory with file paths and sizes.

---

Step 4: Run the Verifier Protocol

Read agents/verifier.md and use it as the base verification protocol.

By default, perform the verification inline. If the user explicitly requested delegated verification, you may run the verifier as a separate subagent. In either case, use:

• The full list of numerical claims extracted in Step 2 (with locations)
• The output file inventory from Step 3
• The paper draft path
• The bibliography file path

Verifier prompt: "You are the verifier agent. Paper draft: [path].
Bibliography: [bib path].

CLAIMS TO VERIFY:
[paste the full claims list from Step 2]

OUTPUT FILE INVENTORY:
[paste the inventory from Step 3]

Verify each claim against the output files. Then do a reverse check —
find output files NOT referenced in the paper. Then check all citation
keys against the bibliography. Follow the verifier agent instructions
and return your full verification report."

After the verification pass completes, collect its results:

• Claim verification table (MATCHED / UNVERIFIED / MISSING FILE per claim)
• Unreferenced output files list
• Missing citation keys

---

Step 5: Save Report

Create quality_reports/ if it is missing before saving.

Save to quality_reports/quality_gate_[YYYY-MM-DD]_[paper-name].md:

# Quality Gate Report: [Paper Name]
**Date:** [YYYY-MM-DD]
**Paper:** [file path]

## Verdict: PASS / CONDITIONAL PASS / FAIL

PASS = all claims matched, no missing citations, no unexplained unreferenced outputs
CONDITIONAL PASS = minor unverified claims or informational unreferenced outputs
FAIL = unverified critical claims or missing citations

---

## Claim Verification

| Claim | Location | Found in Output? | Source File | Status |
|-------|----------|-----------------|-------------|--------|
| β = 0.23 (SE = 0.04) | Section 4, para 2 | Yes | output/tables/main_regs.tex | MATCHED |
| N = 4,521 | Table 2 note | Yes | output/tables/main_regs.tex | MATCHED |
| 42% of firms | Intro, para 1 | No | — | UNVERIFIED |

---

## Unreferenced Outputs

Files in output/ not referenced in the paper:

| File | Size | Recommended Action |
|------|------|-------------------|
| output/tables/robustness_het.tex | 4.2 KB | Reference in Section 7 or explain exclusion |

---

## Missing Citations

| Key | Used At | Status |
|-----|---------|--------|
| SmithJones2021 | Section 3, para 1 | NOT IN BIBLIOGRAPHY — CRITICAL |

---

## Summary

- Claims verified: N / M total
- Claims unverified: K (see table above)
- Unreferenced outputs: J
- Missing citations: L

## Recommended Actions (Priority Order)
1. [BLOCKING] ...
2. [RECOMMENDED] ...

---

Key Rules

• Report only — never edit. All fixes are the user's responsibility after reviewing the report.
• Tolerance: For inline numbers, a claim is MATCHED if the value appears in any output file within reasonable display rounding (±0.005 for 2-decimal numbers).
• False positives are OK. Flag uncertainties as UNVERIFIED rather than guessing MATCHED.
• Missing files are always BLOCKING — a figure reference pointing to a non-existent file is a FAIL.