// Systematically evaluate scholarly work using the ScholarEval framework, providing structured assessment across research quality dimensions including problem formulation, methodology, analysis, and writing with quantitative scoring and actionable feedback.
| name | scholar-evaluation |
| description | Systematically evaluate scholarly work using the ScholarEval framework, providing structured assessment across research quality dimensions including problem formulation, methodology, analysis, and writing with quantitative scoring and actionable feedback. |
| license | MIT license |
| metadata | {"skill-author":"K-Dense Inc."} |
Apply the ScholarEval framework to systematically evaluate scholarly and research work. This skill provides structured evaluation methodology based on peer-reviewed research assessment criteria, enabling comprehensive analysis of academic papers, research proposals, literature reviews, and scholarly writing across multiple quality dimensions.
Use this skill when:
When creating documents with this skill, always consider adding scientific diagrams and schematics to enhance visual communication.
If your document does not already contain schematics or diagrams:
For new documents: Scientific schematics should be generated by default to visually represent key concepts, workflows, architectures, or relationships described in the text.
How to generate schematics:
python scripts/generate_schematic.py "your diagram description" -o figures/output.png
The AI will automatically:
When to add schematics:
For detailed guidance on creating schematics, refer to the scientific-schematics skill documentation.
Begin by identifying the type of scholarly work being evaluated and the evaluation scope:
Work Types:
Evaluation Scope:
Ask the user to clarify if the scope is ambiguous.
Systematically evaluate the work across the ScholarEval dimensions. For each applicable dimension, assess quality, identify strengths and weaknesses, and provide scores where appropriate.
Refer to references/evaluation_framework.md for detailed criteria and rubrics for each dimension.
Core Evaluation Dimensions:
Problem Formulation & Research Questions
Literature Review
Methodology & Research Design
Data Collection & Sources
Analysis & Interpretation
Results & Findings
Scholarly Writing & Presentation
Citations & References
For each evaluated dimension, provide:
Qualitative Assessment:
Quantitative Scoring (Optional): Use a 5-point scale where applicable:
To calculate aggregate scores programmatically, use scripts/calculate_scores.py.
Provide an integrated evaluation summary:
Transform evaluation findings into constructive, actionable feedback:
Feedback Structure:
Feedback Format Options:
Adjust evaluation approach based on:
Stage of Development:
Purpose and Venue:
Discipline-Specific Norms:
Detailed evaluation criteria, rubrics, and quality indicators for each ScholarEval dimension. Load this reference when conducting evaluations to access specific assessment guidelines and scoring rubrics.
Search patterns for quick access:
Python script for calculating aggregate evaluation scores from dimension-level ratings. Supports weighted averaging, threshold analysis, and score visualization.
Usage:
python scripts/calculate_scores.py --scores <dimension_scores.json> --output <report.txt>
User Request: "Evaluate this research paper on machine learning for drug discovery"
Response Process:
references/evaluation_framework.md for detailed criteriaThis skill integrates seamlessly with the scientific writer workflow:
After Paper Generation:
SCHOLAR_EVALUATION.md alongside PEER_REVIEW.mdDuring Revision:
Publication Preparation:
This skill is based on the ScholarEval framework introduced in:
Moussa, H. N., Da Silva, P. Q., Adu-Ampratwum, D., East, A., Lu, Z., Puccetti, N., Xue, M., Sun, H., Majumder, B. P., & Kumar, S. (2025). ScholarEval: Research Idea Evaluation Grounded in Literature. arXiv preprint arXiv:2510.16234. https://arxiv.org/abs/2510.16234
Abstract: ScholarEval is a retrieval augmented evaluation framework that assesses research ideas based on two fundamental criteria: soundness (the empirical validity of proposed methods based on existing literature) and contribution (the degree of advancement made by the idea across different dimensions relative to prior research). The framework achieves significantly higher coverage of expert-annotated evaluation points and is consistently preferred over baseline systems in terms of evaluation actionability, depth, and evidence support.
Vibe Code Orchestrator (VCO) is a governed runtime entry that freezes requirements, plans XL-first execution, and enforces verification and phase cleanup.
Comprehensive citation management for academic research. Resolve bibliographic identifiers, extract accurate metadata, validate citations, deduplicate references, and generate properly formatted BibTeX entries. This skill should be used when you need to verify citation information, convert DOIs/PMIDs/arXiv IDs to BibTeX, or ensure reference accuracy in scientific writing.
Conduct comprehensive, systematic literature reviews using multiple academic databases (PubMed, arXiv, bioRxiv, Semantic Scholar, etc.). This skill should be used when conducting systematic literature reviews, meta-analyses, research synthesis, or comprehensive literature searches across biomedical, scientific, and technical domains. Creates professionally formatted markdown documents and PDFs with verified citations in multiple citation styles (APA, Nature, Vancouver, etc.).
Claude Skills meta-skill: extract domain material (docs/APIs/code/specs) into a reusable Skill (SKILL.md + references/scripts/assets), and refactor existing Skills for clarity, activation reliability, and quality gates.
Design experiments and quasi-experiments before analysis. Use when choosing study design, treatment/control structure, outcomes, assumptions, validation plans after scientific experiment failure, or which of DiD, ITS, synthetic control, or regression discontinuity fits the research question. For fitting models or estimating effects on existing data, use performing-causal-analysis instead.
Structured hypothesis formulation from observations. Use when you have experimental observations or data and need to formulate testable hypotheses with predictions, propose mechanisms, and design experiments to test them. Also owns explicit HypoGeniC-style or automated LLM-driven hypothesis generation/testing requests inside this single skill. For open-ended ideation use scientific-brainstorming.