メニュー
Skill for venue-calibrated scoring, score weighting formulas, decision rules, anti-bias mechanisms, and score interpretation across different venue tiers.
Codex または Claude でインストール この Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。
SOC 職業分類に基づく
| name | scoring-calibration |
| description | Skill for venue-calibrated scoring, score weighting formulas, decision rules, anti-bias mechanisms, and score interpretation across different venue tiers. |
Use this skill when computing review scores, applying decision rules, or calibrating review standards to a specific venue.
Every review scores these 6 dimensions plus confidence:
| Dimension | Range | Description |
|---|---|---|
| Overall | 1-10 | Holistic assessment |
| Soundness | 1-10 | Technical correctness |
| Novelty | 1-10 | Originality of contribution |
| Clarity | 1-10 | Writing and presentation quality |
| Significance | 1-10 | Impact and importance |
| Reproducibility | 1-10 | Can results be reproduced? |
| Confidence | 1-5 | Reviewer's self-assessed expertise |
| Score | Meaning |
|---|---|
| 8-10 | Strong accept — top 10% of submissions |
| 6-7 | Weak accept — above threshold, some issues |
| 5 | Borderline — could go either way |
| 3-4 | Weak reject — below threshold, significant issues |
| 1-2 | Strong reject — fundamental flaws |
Acceptance threshold: Mean ≥ 7, no critical issues
| Score | Meaning |
|---|---|
| 7-10 | Strong accept |
| 5-6 | Accept with revisions |
| 4 | Borderline |
| 2-3 | Reject |
| 1 | Strong reject |
Acceptance threshold: Mean ≥ 6, critical issues addressed
| Score | Meaning |
|---|---|
| 6-10 | Accept |
| 4-5 | Accept with minor revisions |
| 3 | Borderline |
| 1-2 | Reject |
Acceptance threshold: Mean ≥ 5, no fatal flaws
The weighted final score combines dimensions with fixed weights:
final_score = (
0.30 × mean(soundness) +
0.20 × mean(novelty) +
0.20 × mean(significance) +
0.15 × mean(clarity) +
0.15 × mean(reproducibility)
)
These weights can be overridden in .review-config.yaml:
review:
score_weights:
soundness: 0.30
novelty: 0.20
significance: 0.20
clarity: 0.15
reproducibility: 0.15
| Condition | Decision |
|---|---|
| All reviewers ≥ 7, no critical weaknesses | Accept |
| All reviewers ≥ 6, only minor weaknesses | Accept with Minor Revision |
| Mean ≥ 5, no more than 1 reviewer below 5 | Major Revision |
| Mean < 5 or 2+ reviewers below 4 | Reject |
| Strong disagreement (spread ≥ 4 points) | Discussion round before decision |
The decision rules above use venue-specific thresholds:
| Rule Parameter | Top-Tier | Mid-Tier | Workshop |
|---|---|---|---|
| Accept threshold | ≥ 7 | ≥ 6 | ≥ 5 |
| Accept-minor threshold | ≥ 6 | ≥ 5 | ≥ 4 |
| Major revision threshold | ≥ 5 | ≥ 4 | ≥ 3 |
| Reject threshold | < 5 | < 4 | < 3 |
Track scores across revision rounds to detect convergence or stalling:
score_trajectory:
round_1:
alpha: 5
beta: 7
gamma: 6
mean: 6.0
weighted: 5.95
round_2:
alpha: 7
beta: 8
gamma: 7
mean: 7.3
weighted: 7.25
delta: +1.3
convergence_status: "improving"
If delta ≤ 0.3 for 2 consecutive rounds:
→ Flag DIMINISHING_RETURNS
→ Consider declaring EXHAUSTED
Skill providing the exact structured YAML review format that all reviewers must use. Enables automated parsing, scoring, action-item extraction, and convergence detection.
Skill for enforcing review blinding — ensuring reviewers cannot see each other's reports, managing worktree isolation, handling optional author anonymization, and controlling unblinding at synthesis.
Programmatic validation of figure data and generation code. Since LLMs cannot interpret rendered images, this skill ensures figure correctness by analysing the underlying data and plotting code — catching identical-value plots, missing labels, broken references, and other common defects.
Skill for determining whether the review process has converged (accept), needs another round (iterate), or has reached diminishing returns (exhausted).
Skill for managing the full peer review lifecycle: intake validation, reviewer dispatch, review collection, editorial synthesis, revision cycles, re-review, and convergence determination.
Skill for managing review state persistence — the .review/ directory structure, state.yaml lifecycle tracking, artifact hashing, round management, and final output generation.