| name | alterlab-peer-review |
| description | Writes structured, checklist-based manuscript and grant peer reviews — assesses methodology, statistical validity, reporting-standards compliance (CONSORT/STROBE/PRISMA), and gives constructive feedback. Use when writing a formal reviewer report, responding to a journal/grant review invitation, or revising a manuscript against reviewer criteria. For evaluating claims/evidence quality prefer alterlab-scientific-thinking; for a multi-reviewer mock-panel verdict use alterlab-paper-reviewer; for quantitative rubric scoring use alterlab-scholar-eval. Part of the AlterLab Academic Skills suite. |
| allowed-tools | Read Write Edit Bash |
| license | MIT |
| compatibility | No external tools, API keys, or services required — produces reviews from the Read/Write/Edit/Bash tools alone |
| metadata | {"skill-author":"AlterLab","version":"1.0.0"} |
Manuscript and Grant Peer Review
Overview
Peer review is a systematic process for evaluating scientific manuscripts and proposals. Assess methodology, statistics, design, reproducibility, ethics, and reporting standards, then deliver a structured, constructive reviewer report. Apply this skill for manuscript and grant review across disciplines.
When to Use This Skill
This skill should be used when:
- Conducting peer review of scientific manuscripts for journals
- Evaluating grant proposals and research applications
- Assessing methodology and experimental design rigor
- Reviewing statistical analyses and reporting standards
- Evaluating reproducibility and data availability
- Checking compliance with reporting guidelines (CONSORT, STROBE, PRISMA)
- Providing constructive feedback on scientific writing
Defer to a sibling skill when the task is:
- Grading evidence quality / spotting biases, confounders, or causal-inference flaws →
alterlab-scientific-thinking
- Producing a multi-reviewer mock-panel verdict (accept/reject decision from several simulated reviewers) →
alterlab-paper-reviewer
- Scoring or ranking work on a numeric weighted rubric →
alterlab-scholar-eval
Peer Review Workflow
Conduct peer review systematically through the following stages, adapting depth and focus based on the manuscript type and discipline.
Stage 1: Initial Assessment
Begin with a high-level evaluation to determine the manuscript's scope, novelty, and overall quality.
Key Questions:
- What is the central research question or hypothesis?
- What are the main findings and conclusions?
- Is the work scientifically sound and significant?
- Is the work appropriate for the intended venue?
- Are there any immediate major flaws that would preclude publication?
Output: Brief summary (2-3 sentences) capturing the manuscript's essence and initial impression.
Stage 2: Detailed Section-by-Section Review
Conduct a thorough evaluation of each manuscript section, documenting specific concerns and strengths.
Abstract and Title
- Accuracy: Does the abstract accurately reflect the study's content and conclusions?
- Clarity: Is the title specific, accurate, and informative?
- Completeness: Are key findings and methods summarized appropriately?
- Accessibility: Is the abstract comprehensible to a broad scientific audience?
Introduction
- Context: Is the background information adequate and current?
- Rationale: Is the research question clearly motivated and justified?
- Novelty: Is the work's originality and significance clearly articulated?
- Literature: Are relevant prior studies appropriately cited?
- Objectives: Are research aims/hypotheses clearly stated?
Methods
- Reproducibility: Can another researcher replicate the study from the description provided?
- Rigor: Are the methods appropriate for addressing the research questions?
- Detail: Are protocols, reagents, equipment, and parameters sufficiently described?
- Ethics: Are ethical approvals, consent, and data handling properly documented?
- Statistics: Are statistical methods appropriate, clearly described, and justified?
- Validation: Are controls, replicates, and validation approaches adequate?
Critical elements to verify:
- Sample sizes and power calculations
- Randomization and blinding procedures
- Inclusion/exclusion criteria
- Data collection protocols
- Computational methods and software versions
- Statistical tests and correction for multiple comparisons
Results
- Presentation: Are results presented logically and clearly?
- Figures/Tables: Are visualizations appropriate, clear, and properly labeled?
- Statistics: Are statistical results properly reported (effect sizes, confidence intervals, p-values)?
- Objectivity: Are results presented without over-interpretation?
- Completeness: Are all relevant results included, including negative results?
- Reproducibility: Are raw data or summary statistics provided?
Common issues to identify:
- Selective reporting of results
- Inappropriate statistical tests
- Missing error bars or measures of variability
- Over-fitting or circular analysis
- Batch effects or confounding variables
- Missing controls or validation experiments
Discussion
- Interpretation: Are conclusions supported by the data?
- Limitations: Are study limitations acknowledged and discussed?
- Context: Are findings placed appropriately within existing literature?
- Speculation: Is speculation clearly distinguished from data-supported conclusions?
- Significance: Are implications and importance clearly articulated?
- Future directions: Are next steps or unanswered questions discussed?
Red flags:
- Overstated conclusions
- Ignoring contradictory evidence
- Causal claims from correlational data
- Inadequate discussion of limitations
- Mechanistic claims without mechanistic evidence
References
- Completeness: Are key relevant papers cited?
- Currency: Are recent important studies included?
- Balance: Are contrary viewpoints appropriately cited?
- Accuracy: Are citations accurate and appropriate?
- Self-citation: Is there excessive or inappropriate self-citation?
Stage 3: Methodological and Statistical Rigor
Evaluate the technical quality and rigor of the research with particular attention to common pitfalls.
Statistical Assessment:
- Are statistical assumptions met (normality, independence, homoscedasticity)?
- Are effect sizes reported alongside p-values?
- Is multiple testing correction applied appropriately?
- Are confidence intervals provided?
- Is sample size justified with power analysis?
- Are parametric vs. non-parametric tests chosen appropriately?
- Are missing data handled properly?
- Are exploratory vs. confirmatory analyses distinguished?
Experimental Design:
- Are controls appropriate and adequate?
- Is replication sufficient (biological and technical)?
- Are potential confounders identified and controlled?
- Is randomization properly implemented?
- Are blinding procedures adequate?
- Is the experimental design optimal for the research question?
Computational/Bioinformatics:
- Are computational methods clearly described and justified?
- Are software versions and parameters documented?
- Is code made available for reproducibility?
- Are algorithms and models validated appropriately?
- Are assumptions of computational methods met?
- Is batch correction applied appropriately?
Stage 4: Reproducibility and Transparency
Assess whether the research meets modern standards for reproducibility and open science.
Data Availability:
- Are raw data deposited in appropriate repositories?
- Are accession numbers provided for public databases?
- Are data sharing restrictions justified (e.g., patient privacy)?
- Are data formats standard and accessible?
Code and Materials:
- Is analysis code made available (GitHub, Zenodo, etc.)?
- Are unique materials available or described sufficiently for recreation?
- Are protocols detailed in sufficient depth?
Reporting Standards:
- Does the manuscript follow discipline-specific reporting guidelines (CONSORT, PRISMA, ARRIVE, MIAME, MINSEQE, etc.)?
- See
references/reporting_standards.md for common guidelines
- Are all elements of the appropriate checklist addressed?
Stage 5: Figure and Data Presentation
Evaluate the quality, clarity, and integrity of data visualization.
Quality Checks:
- Are figures high resolution and clearly labeled?
- Are axes properly labeled with units?
- Are error bars defined (SD, SEM, CI)?
- Are statistical significance indicators explained?
- Are color schemes appropriate and accessible (colorblind-friendly)?
- Are scale bars included for images?
- Is data visualization appropriate for the data type?
Integrity Checks:
- Are there signs of image manipulation (duplications, splicing)?
- Are Western blots and gels appropriately presented?
- Are representative images truly representative?
- Are all conditions shown (no selective presentation)?
Clarity:
- Can figures stand alone with their legends?
- Is the message of each figure immediately clear?
- Are there redundant figures or panels?
- Would data be better presented as tables or figures?
Stage 6: Ethical Considerations
Verify that the research meets ethical standards and guidelines.
Human Subjects:
- Is IRB/ethics approval documented?
- Is informed consent described?
- Are vulnerable populations appropriately protected?
- Is patient privacy adequately protected?
- Are potential conflicts of interest disclosed?
Animal Research:
- Is IACUC or equivalent approval documented?
- Are procedures humane and justified?
- Are the 3Rs (replacement, reduction, refinement) considered?
- Are euthanasia methods appropriate?
Research Integrity:
- Are there concerns about data fabrication or falsification?
- Is authorship appropriate and justified?
- Are competing interests disclosed?
- Is funding source disclosed?
- Are there concerns about plagiarism or duplicate publication?
Stage 7: Writing Quality and Clarity
Assess the manuscript's clarity, organization, and accessibility.
Structure and Organization:
- Is the manuscript logically organized?
- Do sections flow coherently?
- Are transitions between ideas clear?
- Is the narrative compelling and clear?
Writing Quality:
- Is the language clear, precise, and concise?
- Are jargon and acronyms minimized and defined?
- Is grammar and spelling correct?
- Are sentences unnecessarily complex?
- Is the passive voice overused?
Accessibility:
- Can a non-specialist understand the main findings?
- Are technical terms explained?
- Is the significance clear to a broad audience?
Structuring Peer Review Reports
Organize feedback in a hierarchical structure that prioritizes issues and provides actionable guidance.
Summary Statement
Provide a concise overall assessment (1-2 paragraphs):
- Brief synopsis of the research
- Overall recommendation (accept, minor revisions, major revisions, reject)
- Key strengths (2-3 bullet points)
- Key weaknesses (2-3 bullet points)
- Bottom-line assessment of significance and soundness
Major Comments
List critical issues that significantly impact the manuscript's validity, interpretability, or significance. Number these sequentially for easy reference.
Major comments typically include:
- Fundamental methodological flaws
- Inappropriate statistical analyses
- Unsupported or overstated conclusions
- Missing critical controls or experiments
- Serious reproducibility concerns
- Major gaps in literature coverage
- Ethical concerns
For each major comment:
- Clearly state the issue
- Explain why it's problematic
- Suggest specific solutions or additional experiments
- Indicate if addressing it is essential for publication
Minor Comments
List less critical issues that would improve clarity, completeness, or presentation. Number these sequentially.
Minor comments typically include:
- Unclear figure labels or legends
- Missing methodological details
- Typographical or grammatical errors
- Suggestions for improved data presentation
- Minor statistical reporting issues
- Supplementary analyses that would strengthen conclusions
- Requests for clarification
For each minor comment:
- Identify the specific location (section, paragraph, figure)
- State the issue clearly
- Suggest how to address it
Specific Line-by-Line Comments (Optional)
For manuscripts requiring detailed feedback, provide section-specific or line-by-line comments:
- Reference specific page/line numbers or sections
- Note factual errors, unclear statements, or missing citations
- Suggest specific edits for clarity
Questions for Authors
List specific questions that need clarification:
- Methodological details that are unclear
- Seemingly contradictory results
- Missing information needed to evaluate the work
- Requests for additional data or analyses
Tone and Approach
Maintain a constructive, professional, and collegial tone throughout the review.
Best Practices:
- Be constructive: Frame criticism as opportunities for improvement
- Be specific: Provide concrete examples and actionable suggestions
- Be balanced: Acknowledge strengths as well as weaknesses
- Be respectful: Remember that authors have invested significant effort
- Be objective: Focus on the science, not the scientists
- Be thorough: Don't overlook issues, but prioritize appropriately
- Be clear: Avoid ambiguous or vague criticism
Avoid:
- Personal attacks or dismissive language
- Sarcasm or condescension
- Vague criticism without specific examples
- Requesting unnecessary experiments beyond the scope
- Demanding adherence to personal preferences vs. best practices
- Revealing your identity if reviewing is double-blind
Special Considerations by Manuscript Type
Original Research Articles
- Emphasize rigor, reproducibility, and novelty
- Assess significance and impact
- Verify that conclusions are data-driven
- Check for complete methods and appropriate controls
Reviews and Meta-Analyses
- Evaluate comprehensiveness of literature coverage
- Assess search strategy and inclusion/exclusion criteria
- Verify systematic approach and lack of bias
- Check for critical analysis vs. mere summarization
- For meta-analyses, evaluate statistical approach and heterogeneity
Methods Papers
- Emphasize validation and comparison to existing methods
- Assess reproducibility and availability of protocols/code
- Evaluate improvements over existing approaches
- Check for sufficient detail for implementation
Short Reports/Letters
- Adapt expectations for brevity
- Ensure core findings are still rigorous and significant
- Verify that format is appropriate for findings
Preprints
- Recognize that these have not undergone formal peer review
- May be less polished than journal submissions
- Still apply rigorous standards for scientific validity
- Consider providing constructive feedback to help authors improve before journal submission
Presentations and Slide Decks
⚠️ CRITICAL: For presentations (PowerPoint, Beamer, slide decks), NEVER read the PDF directly — it causes buffer overflows and misses visual formatting issues. ALWAYS convert to images first, using:
python skills/writing-tools/alterlab-scientific-slides/scripts/pdf_to_images.py presentation.pdf review/slide --dpi 150
Then inspect each review/slide-NNN.jpg sequentially and document issues by slide number.
The full image-based review workflow, presentation-specific evaluation criteria (visual
design, layout, content, structure, scientific content), the critical/major/minor issue
catalog, and the presentation review-report format are in
references/presentation_review.md.
Resources
This skill includes reference materials to support comprehensive peer review:
references/reporting_standards.md
Guidelines for major reporting standards across disciplines (CONSORT, PRISMA, ARRIVE, MIAME, STROBE, etc.) to evaluate completeness of methods and results reporting.
references/common_issues.md
Catalog of frequent methodological and statistical issues encountered in peer review, with guidance on identifying and addressing them.
references/presentation_review.md
Image-based review workflow and full evaluation criteria for scientific presentations and slide decks (used when the submission is a talk rather than a manuscript).
Final Checklist
Before finalizing the review, verify:
