메뉴
Run experiments based on the plan from read-exp. Reads check_exp.json and check_code.json, creates environment, runs experiments, compares against paper numbers.
Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.
SOC 직업 분류 기준
Organize a paper PDF and its codebase into clean reading artifacts. Use this before any other read-* skills.
Verify each cited reference exists via web search.
Deep extraction of all verifiable claims from a research paper. Actively proposes implicit claims, splits compound statements, and flags cross-reference inconsistencies.
Verify paper claims with evidence_type "code" against the actual codebase. Use after read-txt to check if code matches what the paper says.
Review experiment design and produce a prioritized reproduction plan. Targets claims with evidence_type "experiment" from check_claim.json. Output feeds into run-exp.
Web-search validation of citation and novelty claims from check_claim.json. Checks whether cited facts are accurate and novelty claims ("first", "pioneering") hold against published literature.
| name | run-exp |
| description | Run experiments based on the plan from read-exp. Reads check_exp.json and check_code.json, creates environment, runs experiments, compares against paper numbers. |
| author | PaperDoctor Research |
| license | MIT |
| argument-hint | ["paper_dir"] |
| dependencies | ["read-exp","read-code"] |
| allowed-tools | Read, Grep, Glob, Bash |
This skill requires explicit user approval before execution. Do not start automatically after Phase 2 — present the reproduction plan summary and wait for the user to confirm.
{paper_dir}/reports/check_exp.json — experiment plan from read-exp (priority, feasibility, commands){paper_dir}/reports/check_code.json — code issues from read-code (fix before running)- [ ] Step 1: Create environment
- [ ] Step 2: Run experiments
conda create -n paperdr-{paper_stem} python=<version> -y
conda activate paperdr-{paper_stem}
# follow README for install
Download data/checkpoints as needed. Fix any warning/error items from check_code.json before running.
Run experiments from check_exp.json in priority order (high first). Prefer eval over training.
For each experiment, compare output against the paper's numbers. Update check_exp.json with:
{
"our_result": {"metric": 0.0},
"status": "pass",
"note": "..."
}
| Status | Meaning |
|---|---|
pass | Matches paper within ~1-2% |
warning | Partial match or needed fixes |
error | Significant difference or could not run |
error — note cause, do not retry endlessly