메뉴
Update docs/research/09_current_evidence.md after a benchmark study completes. Use after any bench-locomo, bench-realistic, bench-ablation, or bench-appraisal run. Pulls numbers from results.md, enforces DISCOVERY vs CONFIRMATORY framing, cross-references the pre-reg.
Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.
SOC 직업 분류 기준
Run the pre-publish validation gate for a new release. Use before `make publish` or Zenodo deposit. Takes a version argument, runs make release-check, validates CHANGELOG [Unreleased] is non-empty, verifies version strings align, and previews the Zenodo metadata sync.
Run one of the project's comparative benchmark studies (realistic, ablation, appraisal, comparative). Use when executing Pre-reg S2/S3 or running appraisal quality. Validates env, runs the make target, summarizes results, and prompts for evidence update.
Run the LoCoMo external benchmark (Pre-reg Study S1) end-to-end. Use when executing or re-running the AFT vs naive_rag comparison on LoCoMo10. Enforces dry-run-first, env-var check, background execution, and post-run evidence update.
Build the paper + arXiv submission bundle. Use before arXiv upload or when preparing a workshop submission. Runs latexmk, produces paper/arxiv-submission.tar.gz, then validates all result tables against committed benchmarks/*/results.md.
Verify pre-registration integrity — no uncommitted edits to benchmarks/preregistration.md, and declared studies match produced result files. Use before paper building, release gates, or when resuming scientific work after a break.
| name | evidence-update |
| description | Update docs/research/09_current_evidence.md after a benchmark study completes. Use after any bench-locomo, bench-realistic, bench-ablation, or bench-appraisal run. Pulls numbers from results.md, enforces DISCOVERY vs CONFIRMATORY framing, cross-references the pre-reg. |
| disable-model-invocation | true |
| arguments | ["study"] |
| allowed-tools | Read Edit |
Argument: $study — one of: locomo, realistic, ablation, appraisal.
Updates docs/research/09_current_evidence.md with the latest results from
the completed study, maintaining the DISCOVERY vs CONFIRMATORY framing that is
central to the paper's scientific credibility.
Do not remove existing limitations or caveats unless the new data specifically resolves them.
| Argument | Results file |
|---|---|
locomo | benchmarks/locomo/results.md |
realistic | benchmarks/realistic/results.md (or results.v2.md if v2 run) |
ablation | benchmarks/ablation/results.md (or results.v2.md if v2 run) |
appraisal | benchmarks/appraisal_quality/results.md |
Open the results file for $study. Extract:
git log --oneline --format="%ci" benchmarks/preregistration.md | head -1
git log --oneline --format="%ci" benchmarks/<study>/results.json | head -1
benchmarks/preregistration.md (H1/H2 for S1,
H3–H6 for S2, Ha–Hb for S3).docs/research/09_current_evidence.mdFind the section corresponding to $study. Update:
Keep existing cells for other studies untouched.
After editing, confirm the updated section contains: