菜单
Run one of the project's comparative benchmark studies (realistic, ablation, appraisal, comparative). Use when executing Pre-reg S2/S3 or running appraisal quality. Validates env, runs the make target, summarizes results, and prompts for evidence update.
用 Codex 或 Claude 帮你安装 复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。
基于 SOC 职业分类
Run the pre-publish validation gate for a new release. Use before `make publish` or Zenodo deposit. Takes a version argument, runs make release-check, validates CHANGELOG [Unreleased] is non-empty, verifies version strings align, and previews the Zenodo metadata sync.
Run the LoCoMo external benchmark (Pre-reg Study S1) end-to-end. Use when executing or re-running the AFT vs naive_rag comparison on LoCoMo10. Enforces dry-run-first, env-var check, background execution, and post-run evidence update.
Update docs/research/09_current_evidence.md after a benchmark study completes. Use after any bench-locomo, bench-realistic, bench-ablation, or bench-appraisal run. Pulls numbers from results.md, enforces DISCOVERY vs CONFIRMATORY framing, cross-references the pre-reg.
Build the paper + arXiv submission bundle. Use before arXiv upload or when preparing a workshop submission. Runs latexmk, produces paper/arxiv-submission.tar.gz, then validates all result tables against committed benchmarks/*/results.md.
Verify pre-registration integrity — no uncommitted edits to benchmarks/preregistration.md, and declared studies match produced result files. Use before paper building, release gates, or when resuming scientific work after a break.
| name | bench-study |
| description | Run one of the project's comparative benchmark studies (realistic, ablation, appraisal, comparative). Use when executing Pre-reg S2/S3 or running appraisal quality. Validates env, runs the make target, summarizes results, and prompts for evidence update. |
| disable-model-invocation | true |
| arguments | ["study"] |
| allowed-tools | Bash(make bench-*) Bash(make llm-config*) Read |
Argument: $study — one of: realistic, ablation, appraisal, comparative, comparative-sbert, appraisal-confound.
Stop immediately if the argument is not in the list above and ask the user to pick.
| Argument | Make target | Pre-reg | LLM required |
|---|---|---|---|
realistic | make bench-realistic | S2 | No |
ablation | make bench-ablation | S3 | No |
appraisal | make bench-appraisal | M1.3 | Yes (EMOTIONAL_MEMORY_LLM_API_KEY) |
comparative | make bench-comparative | — | No (hash embedder, quick) |
comparative-sbert | make bench-comparative-sbert | — | No (SBERT embedder, paper-canonical) |
appraisal-confound | make bench-appraisal-confound | Addendum A | No |
For appraisal:
make llm-config-strict
make bench-$study
For realistic, ablation, and comparative-sbert the Makefile already uses the SBERT embedder.
Read benchmarks/$study/results.md (or console output for appraisal).
Report:
docs/research/09_current_evidence.mdgit log --oneline benchmarks/preregistration.md | head -1
git log --oneline benchmarks/$study/results.json 2>/dev/null | head -1
Remind the user to run /evidence-update $study to update
docs/research/09_current_evidence.md with the new numbers and framing.