메뉴
Verify pre-registration integrity — no uncommitted edits to benchmarks/preregistration.md, and declared studies match produced result files. Use before paper building, release gates, or when resuming scientific work after a break.
Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.
SOC 직업 분류 기준
Run the pre-publish validation gate for a new release. Use before `make publish` or Zenodo deposit. Takes a version argument, runs make release-check, validates CHANGELOG [Unreleased] is non-empty, verifies version strings align, and previews the Zenodo metadata sync.
Run one of the project's comparative benchmark studies (realistic, ablation, appraisal, comparative). Use when executing Pre-reg S2/S3 or running appraisal quality. Validates env, runs the make target, summarizes results, and prompts for evidence update.
Run the LoCoMo external benchmark (Pre-reg Study S1) end-to-end. Use when executing or re-running the AFT vs naive_rag comparison on LoCoMo10. Enforces dry-run-first, env-var check, background execution, and post-run evidence update.
Update docs/research/09_current_evidence.md after a benchmark study completes. Use after any bench-locomo, bench-realistic, bench-ablation, or bench-appraisal run. Pulls numbers from results.md, enforces DISCOVERY vs CONFIRMATORY framing, cross-references the pre-reg.
Build the paper + arXiv submission bundle. Use before arXiv upload or when preparing a workshop submission. Runs latexmk, produces paper/arxiv-submission.tar.gz, then validates all result tables against committed benchmarks/*/results.md.
| name | prereg-guard |
| description | Verify pre-registration integrity — no uncommitted edits to benchmarks/preregistration.md, and declared studies match produced result files. Use before paper building, release gates, or when resuming scientific work after a break. |
| disable-model-invocation | true |
| allowed-tools | Bash(git diff*) Bash(git log*) Bash(git status*) Read |
Checks that benchmarks/preregistration.md has not been modified since its
last commit, and that result files exist for studies that have been run.
Any FAIL = stop before paper building or publishing.
The pre-registration file (benchmarks/preregistration.md) is the scientific
contract. It must never be edited after the first commit that introduced it.
Modifying it post-hoc would invalidate the confirmatory status of any study
whose hypothesis it declares.
git diff benchmarks/preregistration.md
git diff --cached benchmarks/preregistration.md
Expected: empty output. Any diff is a pre-reg violation.
git status --short benchmarks/preregistration.md
Expected: no output (file not listed).
Read benchmarks/preregistration.md to find declared studies. For each:
| Study | Result file to check |
|---|---|
| S1 (LoCoMo) | benchmarks/locomo/results.json, benchmarks/locomo/results.md |
| S2 (realistic v2) | benchmarks/realistic/results.v2.md |
| S3 (ablation v2) | benchmarks/ablation/results.v2.md |
If a result file is missing → WARN (study not yet run, not a violation). If a result file exists but is not committed → WARN (uncommitted results).
git log --oneline --format="%ci %s" benchmarks/preregistration.md | head -1
git log --oneline --format="%ci %s" benchmarks/locomo/results.json | head -1
Pre-reg commit timestamp must be strictly before each result commit timestamp for the study to qualify as CONFIRMATORY.
PASS ✓ preregistration.md unmodified (working tree clean)
PASS ✓ preregistration.md unmodified (staged area clean)
PASS ✓ S1 results present: benchmarks/locomo/results.json
PASS ✓ S1 results committed
PASS ✓ S1 pre-reg predates results → CONFIRMATORY claim valid
WARN ⚠ S2 results missing: benchmarks/realistic/results.v2.md (study not yet run)
WARN ⚠ S3 results missing: benchmarks/ablation/results.v2.md (study not yet run)
PASS on all checks → safe to proceed with paper building or release. Any FAIL → stop and resolve before continuing.