review-template

name	review-template
description	Skill providing the exact structured YAML review format that all reviewers must use. Enables automated parsing, scoring, action-item extraction, and convergence detection.

Structured Review Template

Use this skill when producing or parsing a reviewer report. All reviewers MUST output their reviews in this exact YAML format. Structured output enables automated processing.

Full Review Template

review:
  # ── Metadata ──
  reviewer: "alpha" | "beta" | "gamma"
  round: <round number>
  timestamp: "<ISO 8601>"

  # ── Summary ──
  summary: |
    2-3 paragraph summary of the paper's contributions and approach.
    Written in reviewer's own words (tests comprehension).
    Must demonstrate understanding of the core contribution.

  # ── Scores ──
  scores:
    overall: <1-10>          # Holistic assessment (maps to venue scale)
    soundness: <1-10>        # Technical correctness
    novelty: <1-10>          # Originality of contribution
    clarity: <1-10>          # Writing and presentation quality
    significance: <1-10>     # Impact and importance
    reproducibility: <1-10>  # Can results be reproduced?
    confidence: <1-5>        # Reviewer's confidence in their assessment

  # ── Recommendation ──
  recommendation: "accept" | "accept_minor" | "major_revision" | "reject"

  # ── Strengths ──
  strengths:
    - id: S1
      text: "<specific strength>"
      significance: "high" | "medium" | "low"
      evidence: "<verbatim quote from the paper with section/page reference>"
    - id: S2
      text: "<specific strength>"
      significance: "high" | "medium" | "low"
      evidence: "<verbatim quote or exact file reference>"

  # ── Weaknesses ──
  weaknesses:
    - id: W1
      text: "<specific weakness>"
      severity: "critical" | "major" | "minor"
      category: "experiments" | "theory" | "writing" | "novelty" | "reproducibility"
      actionable: true | false
      suggested_fix: "<specific suggestion for fixing this weakness>"
      evidence: "<verbatim quote from the paper with exact section/page/line reference>"
    - id: W2
      text: "<specific weakness>"
      severity: "critical" | "major" | "minor"
      category: "<category>"
      actionable: true | false
      suggested_fix: "<suggestion>"
      evidence: "<verbatim quote or exact file reference>"

  # ── Questions for Authors ──
  questions:
    - id: Q1
      text: "<question that needs answering>"
      blocking: true | false     # Must be addressed for acceptance
      evidence: "<the specific text that prompted this question>"
    - id: Q2
      text: "<question>"
      blocking: true | false
      evidence: "<relevant quote or reference>"

  # ── Detailed Comments ──
  detailed_comments:
    - location: "<Section X.Y, Equation Z, Figure N, Table M>"
      type: "error" | "suggestion" | "clarification" | "praise"
      text: "<specific comment>"
      evidence: "<verbatim quote from the cited location>"
    - location: "<location>"
      type: "<type>"
      text: "<comment>"
      evidence: "<verbatim quote>"

  # ── Reproducibility Assessment ──
  reproducibility_check:
    code_provided: true | false
    instructions_clear: true | false
    dependencies_specified: true | false
    hyperparameters_documented: true | false
    random_seeds_reported: true | false
    data_available: true | false
    key_details_missing:
      - "<missing detail 1>"
      - "<missing detail 2>"

  # ── Figure and Table Assessment ──
  figure_assessment:
    total_figures: <count>
    total_tables: <count>
    figures_reviewed:
      - id: "Figure 1"
        description: "<what the figure shows>"
        quality: "good" | "needs_improvement" | "poor"
        issues:
          - "<specific issue, e.g., missing axis labels>"
        suggestions:
          - "<specific improvement suggestion>"
      - id: "Table 1"
        quality: "good" | "needs_improvement" | "poor"
        issues: []
        suggestions: []
    missing_visualizations:
      - "<suggested figure that would strengthen the paper>"

  # ── Constructive Suggestions ──
  # Beyond fixing weaknesses, suggest ideas that could strengthen the work.
  # These are positive, forward-looking proposals — not required for acceptance.
  constructive_suggestions:
    - id: CS1
      type: "experiment" | "analysis" | "framing" | "extension" | "visualization"
      text: "<specific constructive suggestion to improve the work>"
      expected_impact: "high" | "medium" | "low"
      effort: "low" | "medium" | "high"
      rationale: "<why this would strengthen the paper>"
    - id: CS2
      type: "<type>"
      text: "<suggestion>"
      expected_impact: "<impact>"
      effort: "<effort>"
      rationale: "<rationale>"

  # ── Ethics and Broader Impact ──
  ethics_review:
    concerns: true | false
    notes: "<description of any ethical concerns>"

Re-Review Template

Used in revision rounds when reviewing a revised submission:

re_review:
  reviewer: "alpha" | "beta" | "gamma"
  round: <round number>
  timestamp: "<ISO 8601>"

  original_score: <previous round's overall score>
  updated_score: <new overall score>
  recommendation: "accept" | "accept_minor" | "major_revision" | "reject"

  concerns_addressed:
    - id: W1
      status: "resolved" | "partially_resolved" | "unresolved"
      comment: "<assessment of how well the concern was addressed>"
    - id: W2
      status: "resolved" | "partially_resolved" | "unresolved"
      comment: "<assessment>"

  questions_answered:
    - id: Q1
      status: "answered" | "partially_answered" | "unanswered"
      comment: "<assessment of the answer>"

  new_concerns:
    - id: NW1
      text: "<new issue discovered in the revision>"
      severity: "critical" | "major" | "minor"
      category: "<category>"

  remaining_blockers:
    - "<anything still preventing acceptance>"

  narrative_alignment:
    direction: "aligned" | "partially_aligned" | "divergent"
    comment: |
      Assessment of how well the revision moves toward the editorial narrative's
      thesis — the paper's best possible version as articulated by the AE.
    roadmap_items_incorporated:
      - priority: 1
        status: "incorporated" | "partially" | "not_addressed"
        comment: "<how was this roadmap item handled?>"

  revision_quality: |
    Brief assessment of the revision quality and author responsiveness.

Meta-Review Template

Used by the Associate Editor to synthesize all reviews:

meta_review:
  round: <round number>
  timestamp: "<ISO 8601>"

  decision: "accept" | "minor_revision" | "major_revision" | "reject"

  consensus_strengths:
    - "<strength agreed by 2+ reviewers>"

  consensus_weaknesses:
    - "<weakness flagged by 2+ reviewers>"

  disagreements:
    - point: "<topic of disagreement>"
      reviewer_alpha: "<α's position and score>"
      reviewer_beta: "<β's position and score>"
      reviewer_gamma: "<γ's position and score>"
      resolution: "<how disagreement was resolved and why>"

  required_changes:
    - id: R1
      source: "W1-alpha"
      action: "<specific required action>"
      severity: "critical"
    - id: R2
      source: "W1-beta"
      action: "<specific required action>"
      severity: "major"

  optional_suggestions:
    - "<non-blocking improvement suggestion>"

  # ── Editorial Narrative ──
  # The AE's constructive thesis: what is the strongest version of this paper,
  # and what story should the revision pursue? This guides the revision agent
  # and gives reviewers a shared frame for re-review.
  editorial_narrative:
    thesis: |
      1-2 paragraph statement of the paper's best possible story — the version
      that maximizes impact while staying true to the evidence. Identifies
      the core contribution and the framing that best serves it.
    key_angle: "<the single most compelling aspect the paper should emphasize>"
    underexploited_strengths:
      - "<strength the paper has but doesn't leverage enough>"
    suggested_reframing: |
      Optional: if the paper would benefit from repositioning its contribution
      (e.g., from 'novel method' to 'surprising empirical finding'), describe here.
    improvement_roadmap:
      - priority: 1
        suggestion: "<highest-impact improvement from reviewer constructive_suggestions>"
        source: "CS1-alpha | CS2-beta | AE"
      - priority: 2
        suggestion: "<next improvement>"
        source: "<source>"

  score_summary:
    mean_overall: <float>
    min_overall: <int>
    max_overall: <int>
    weighted_final: <float>
    reviewer_agreement: "strong" | "moderate" | "weak"

Parsing Guidelines

Extracting Action Items

To extract all actionable items from a review:

1. Collect all weaknesses where actionable: true
2. Collect all questions where blocking: true
3. Collect all detailed_comments where type: "error"
4. Rank by severity: critical > major > minor

Score Aggregation

# Compute weighted mean across reviewers
def weighted_score(reviews, weights):
    return sum(
        weights[dim] * mean(r['scores'][dim] for r in reviews)
        for dim in weights
    )

DEFAULT_WEIGHTS = {
    'soundness': 0.30,
    'novelty': 0.20,
    'significance': 0.20,
    'clarity': 0.15,
    'reproducibility': 0.15
}

Convergence Detection

def check_convergence(re_reviews):
    all_accept = all(
        r['recommendation'] in ('accept', 'accept_minor')
        for r in re_reviews
    )
    no_blockers = all(
        len(r.get('remaining_blockers', [])) == 0
        for r in re_reviews
    )
    no_critical_new = all(
        not any(c['severity'] == 'critical' for c in r.get('new_concerns', []))
        for r in re_reviews
    )
    return all_accept and no_blockers and no_critical_new