소스 정보
- 저장소
- LigphiDonk/Oh-my--paper
- 최근 소스 활동
- 2026년 3월 21일 12:33
- 감지된 SKILL.md 언어
- 영어
- 스타
- 717
- 포크
- 52
설치 방법
기본적으로 소스를 먼저 확인하는 Prompt가 선택됩니다. 직접 명령으로 전환하거나 로컬 사본을 다운로드할 수도 있습니다.
소스 파일 검토
설치 여부를 결정하기 전에 SKILL.md와 SkillsMP에 표시된 보조 파일을 읽어 보세요.
메뉴
기본적으로 소스를 먼저 확인하는 Prompt가 선택됩니다. 직접 명령으로 전환하거나 로컬 사본을 다운로드할 수도 있습니다.
설치 여부를 결정하기 전에 SKILL.md와 SkillsMP에 표시된 보조 파일을 읽어 보세요.
Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.
직접 명령은 검토 Prompt를 거치지 않습니다. 실행하기 전에 소스를 확인하세요.
npx skills add https://github.com/LigphiDonk/Oh-my--paper --skill inno-idea-eval명령은 한 줄로 유지됩니다. 복사하기 전에 가로로 스크롤해 전체 내용을 확인하세요.
로컬 사본을 원하시나요? SkillsMP에서 현재 제공할 수 있는 파일을 다운로드하세요.
SOC 직업 분류 기준
SKILL.md 표시 중
| id | inno-idea-eval |
| name | inno-idea-eval |
| version | 1.0.0 |
| description | Multi-persona idea evaluation with quality gate. |
| stages | ["ideation"] |
| tools | ["read_file","search_project","write_file"] |
| summary | Multi-persona idea evaluation with quality gate. Evaluates ideas across 5 InnoEval dimensions (Clarity, Novelty, Validity, Feasibility, Significance) using 3 reviewer personas and a meta-review. Sits between inno-idea-generation and inno-c... |
| primaryIntent | ideation |
| intents | ["ideation","evaluation"] |
| capabilities | ["research-planning"] |
| domains | ["general"] |
| keywords | ["inno-idea-eval","idea evaluation","research-planning","inno","idea","eval","multi","persona","evaluation","quality","gate","evaluates"] |
| source | builtin |
| status | verified |
| upstream | {"repo":"dr-claw","path":"skills/inno-idea-eval","revision":"8322dc4ef575affaa374aa7922c0a0971c6db7d7"} |
| resourceFlags | {"hasReferences":true,"hasScripts":false,"hasTemplates":false,"hasAssets":false,"referenceCount":3,"scriptCount":0,"templateCount":0,"assetCount":0,"optionalScripts":false} |
Multi-persona idea evaluation with quality gate. Evaluates ideas across 5 InnoEval dimensions (Clarity, Novelty, Validity, Feasibility, Significance) using 3 reviewer personas and a meta-review. Sits between inno-idea-generation and inno-c...
Use this skill when the user request matches its research workflow scope. Prefer the bundled resources instead of recreating templates or reference material. Keep outputs traceable to project files, citations, scripts, or upstream evidence.
references/ only when the current task needs the extra detail.skills/inno-idea-eval/
├── SKILL.md ← this file
├── prompts/
│ ├── build_eval_query.md ← Per-persona evaluation query (all 5 dims)
│ ├── build_evidence_assembly.md ← How to compose evidence from pipeline artifacts
│ ├── build_meta_review_query.md ← Area-chair aggregation of 3 persona reviews
│ ├── build_novelty_queries.md ← Query extraction for novelty verification (Step 0.5a)
│ ├── build_novelty_analysis.md ← Similarity analysis for novelty verification (Step 0.5c)
│ └── build_refinement_feedback_query.md ← Structured feedback for refinement loop
└── references/
├── eval_agent_instructions.md ← Full eval agent system prompt + scoring rubrics
├── novelty_verification_config.md ← Novelty search config, threat levels, fast-fail protocol
└── reviewer_personas.md ← 3 persona definitions + evidence filter logic
How to use the resource files: Each prompt template in
prompts/documents the exact parameters, the full text template, and usage notes (when it is a new conversation vs. appended message, how to format evidence blocks, etc.). Thereferences/directory contains the Eval Agent's complete system instructions including its scoring rubrics, persona definitions, and evidence filter logic. Consult these files for the authoritative details; the steps below provide a summary.
Paths for Ideation/ideas and Ideation/references come from instance.json (instance.Ideation.ideas, instance.Ideation.references). They are absolute in Dr. Claw-created projects; use as-is. If relative, resolve with path.join(project_path, value).
| Parameter | Required | Description |
|---|---|---|
selected_idea | Yes | The idea to evaluate, read from Ideation/ideas/selected_idea.txt |
references | No* | Pre-formatted string listing all source papers (from inno-prepare-resources) |
prepare_res | No* | Full text response from the Prepare Agent (selected repositories and reasoning) |
download_res | No* | Result log from downloading arXiv paper sources |
data_module | No* | The imported metaprompt module (provides TASK field describing the ML task) |
context_variables | Yes | Shared context dictionary (must contain final_selected_idea_data) |
*Standalone mode: only selected_idea required; evaluation proceeds ungrounded with a noted limitation.
| Output | Description |
|---|---|
eval_report | Full markdown evaluation report (meta-review) |
eval_scores | Structured JSON: per-dimension, per-persona, aggregated |
eval_decision | One of: strong_accept / accept / borderline_accept / borderline_reject / reject |
eval_feedback | Strengths/weaknesses/suggestions (for refinement or downstream) |
context_variables["idea_evaluation_result"] | Complete structured result dict |
Each step produces two kinds of files:
.txt files (primary) -- the full markdown content of each review, written directly to Ideation/ideas/.json files (derived) -- structured metadata under Ideation/ideas/logs/, whose text fields must be copied verbatim from the corresponding .txt files (never summarized)Ideation/ideas/
├── novelty_grounding_report.txt ← Step 0.5: Active Novelty Verification report
├── eval_report.txt ← Step 4: full meta-review report (markdown)
├── eval_persona_1_review.txt ← Step 1: Senior ML Researcher review
├── eval_persona_2_review.txt ← Step 2: Domain Expert review
├── eval_persona_3_review.txt ← Step 3: Methods Specialist review
└── logs/
├── idea_eval_agent_novelty.json ← Step 0.5: Novelty search + analysis structured data
├── idea_eval_agent_persona_1.json ← Step 1: Persona 1 structured scores
├── idea_eval_agent_persona_2.json ← Step 2: Persona 2 structured scores
├── idea_eval_agent_persona_3.json ← Step 3: Persona 3 structured scores
└── idea_eval_agent_meta_review.json ← Step 4: Aggregated decision + full report
For every step, always write the .txt file first, then build the .json file by copying the .txt content into the appropriate field:
For the novelty verification step:
novelty_grounding_report.txt with the full novelty analysis reportreport_textlogs/idea_eval_agent_novelty.jsonFor each persona review:
eval_persona_{N}_review.txt with the agent's full reviewreview_textlogs/idea_eval_agent_persona_{N}.jsonFor the meta-review step:
eval_report.txt with the agent's full meta-review reportreport_textlogs/idea_eval_agent_meta_review.json.txt file naming| Step | File name | Content |
|---|---|---|
| Novelty verification | novelty_grounding_report.txt | Active Novelty Verification report |
| Persona 1 review | eval_persona_1_review.txt | Full markdown review from Senior ML Researcher |
| Persona 2 review | eval_persona_2_review.txt | Full markdown review from Domain Expert |
| Persona 3 review | eval_persona_3_review.txt | Full markdown review from Methods Specialist |
| Meta-review | eval_report.txt | Full markdown meta-review report |
.json file naming| Step | File name | Key fields |
|---|---|---|
| Novelty | idea_eval_agent_novelty.json | search_config, queries, novelty_threat_level, report_text |
| Persona 1 | idea_eval_agent_persona_1.json | persona, scores, review_text |
| Persona 2 | idea_eval_agent_persona_2.json | persona, scores, review_text |
| Persona 3 | idea_eval_agent_persona_3.json | persona, scores, review_text |
| Meta-review | idea_eval_agent_meta_review.json | aggregated_scores, decision, report |
.json file format (each persona)Each file contains context_variables only (no messages). The review_text field holds the full text copied from the corresponding .txt file:
{
"context_variables": {
"ideas_path": "<instance.Ideation.ideas>",
"references_path": "<instance.Ideation.references>",
"persona": "senior_ml_researcher | domain_expert | methods_specialist",
"scores": {
"clarity": { "score": 0, "reason": "...", "references": [] },
"novelty": { "score": 0, "reason": "...", "references": [] },
"validity": { "score": 0, "reason": "...", "references": [] },
"feasibility": { "score": 0, "reason": "...", "references": [] },
"significance": { "score": 0, "reason": "...", "references": [] }
},
"strengths": [],
"weaknesses": [],
"suggestions": [],
"recommendation": "Accept|Reject|...",
"review_text": "<FULL text from eval_persona_{N}_review.txt>"
}
}
.json file format (meta-review){
"context_variables": {
"ideas_path": "<instance.Ideation.ideas>",
"references_path": "<instance.Ideation.references>",
"aggregated_scores": {
"clarity": { "avg": 0, "scores": [0, 0, 0] },
"novelty": { "avg": 0, "scores": [0, 0, 0] },
"validity": { "avg": 0, "scores": [0, 0, 0] },
"feasibility": { "avg": 0, "scores": [0, 0, 0] },
"significance": { "avg": 0, "scores": [0, 0, 0] }
},
"overall_avg": 0,
"decision": "strong_accept|accept|borderline_accept|borderline_reject|reject",
"report_text": "<FULL text from eval_report.txt>",
"strengths": [],
"weaknesses": [],
"suggestions": [],
"idea_evaluation_result": { "...complete structured result..." }
}
}
.json file format (novelty verification){
"context_variables": {
"step": "novelty_verification",
"search_config": {
"num_queries": 4,
"sources": ["arxiv", "semantic_scholar", "openalex"],
"max_results_per_query": 10,
"year_from": "<current_year - 3>"
},
"queries": [
{ "type": "core_method", "query": "...", "rationale": "..." },
{ "type": "problem_domain", "query": "...", "rationale": "..." },
{ "type": "key_component", "query": "...", "rationale": "..." },
{ "type": "broad_approach", "query": "...", "rationale": "..." }
],
"idea_summary": "...",
"search_results": { "total_raw": 0, "total_unique": 0 },
"triage": [
{ "title": "...", "year": 0, "relevance": "high|medium|low|irrelevant", "is_inspiration_source": false, "assessment": "..." }
],
"detailed_analysis": [
{ "title": "...", "year": 0, "overlap": "...", "differences": "...", "threat_level": "..." }
],
"novelty_threat_level": "critical_overlap|high_overlap|moderate_overlap|low_overlap|novel",
"genuine_novel_contributions": ["..."],
"report_text": "<FULL text from novelty_grounding_report.txt>",
"fast_fail_triggered": false,
"user_decision": null
}
}
review_text and report_text must contain the complete markdown from the .txt file -- never a summary or abbreviation..json grows independently; the meta-review .json aggregates all three.Full template:
prompts/build_evidence_assembly.md
Read existing pipeline artifacts and compose 3 evidence blocks (one per persona knowledge level):
| Persona Knowledge | Evidence Included |
|---|---|
high (Senior ML) | All papers + LaTeX sources + all repos + full task context |
medium (Domain Expert) | Paper titles/abstracts + repo descriptions + task context |
medium (Methods Specialist) | Repo code + paper titles + implementation details |
Sources: Ideation/references/papers/, Experiment/code_references/, references string, prepare_res, data_module.TASK. No new search needed.
If running in standalone mode (no pipeline artifacts), note this limitation in each review and proceed with ungrounded evaluation.
Query template:
prompts/build_novelty_queries.mdAnalysis template:prompts/build_novelty_analysis.mdConfiguration:references/novelty_verification_config.md
Proactively search the literature to verify whether the idea (or key components) already exists. This step runs before persona reviews so all 3 reviewers have the prior art report as evidence.
Sub-steps:
0.5a — Extract search queries (LLM call using build_novelty_queries.md):
selected_idea + known source_papers (inspiration)0.5b — Execute searches (4 invocations of search_ai_papers.py):
python3 ~/.claude/skills/searching-ai-papers/scripts/search_ai_papers.py \
--query "<query>" --sources arxiv,semantic_scholar,openalex \
--max-results 10 --year-from <current_year-3> --format json
unverified)0.5c — Analyze similarity (LLM call using build_novelty_analysis.md):
selected_idea + deduplicated search results + source_papers + idea_summary + key_terms[INSPIRATION_SOURCE]0.5d — Fast-fail check:
critical_overlap on a non-inspiration paper AND CRITICAL_OVERLAP_FAST_FAIL is true:
0.5e — Inject report into evidence:
Save (txt first, then json):
Ideation/ideas/novelty_grounding_report.txtreport_text copied verbatim from the .txt fileIdeation/ideas/logs/idea_eval_agent_novelty.jsonFor refinement re-runs, save as novelty_grounding_report_v{N}.txt and idea_eval_agent_novelty_v{N}.json.
Full template:
prompts/build_eval_query.mdAgent system prompt:references/eval_agent_instructions.mdPersona definitions:references/reviewer_personas.md
For each persona (1=Senior ML Researcher, 2=Domain Expert, 3=Methods Specialist):
prompts/build_eval_query.md template with persona-specific evidence block from Step 0Scoring Calibration (from InnoEval):
Self-Discovery Check (Novelty only): If a found paper appears identical to the idea, assume it IS the idea's inspiration source -- don't penalize.
Save (txt first, then json) after each persona:
Ideation/ideas/eval_persona_{N}_review.txtIdeation/ideas/logs/idea_eval_agent_persona_{N}.jsonFull template:
prompts/build_meta_review_query.md
Aggregate all 3 reviews. The agent acts as Area Chair:
Decision Thresholds:
| Average Score | Decision | Action |
|---|---|---|
| >= 7.0 | strong_accept | Proceed to code survey |
| >= 6.0 | accept | Proceed to code survey |
| >= 5.0 | borderline_accept | Present report, ask user whether to proceed or refine |
| >= 4.0 | borderline_reject | Suggest refinement, ask user |
| < 4.0 | reject | Trigger refinement loop automatically |
Save (txt first, then json):
Ideation/ideas/eval_report.txtIdeation/ideas/logs/idea_eval_agent_meta_review.jsonstrong_accept or accept): Pipeline continues to inno-code-survey. selected_idea passes through unchanged.borderline_accept or borderline_reject): Present evaluation report to user. Ask whether to proceed, refine, or abandon.reject): Build structured feedback via prompts/build_refinement_feedback_query.md. Trigger refinement loop.Full template:
prompts/build_refinement_feedback_query.md
inno-idea-generation)Ideation/ideas/refined_idea_v{N}.txtselected_idea.txt and final_selected_idea_dataSet context_variables["idea_evaluation_result"] with complete structured data:
{
"decision": "strong_accept|accept|...",
"overall_avg": 0.0,
"aggregated_scores": { "..." },
"persona_reviews": [ "..." ],
"report": "<full report text>",
"novelty_verification": {
"threat_level": "critical_overlap|high_overlap|moderate_overlap|low_overlap|novel",
"genuine_novel_contributions": ["..."],
"search_coverage": { "total_raw": 0, "total_unique": 0, "sources": ["..."] },
"fast_fail_triggered": false,
"user_decision": null
},
"refinement_iterations": 0,
"grounded": true
}
If refinement occurred, also update:
Ideation/ideas/selected_idea.txt with the refined ideacontext_variables["final_selected_idea_data"] with updated text| Constant | Default | Description |
|---|---|---|
NUM_PERSONAS | 3 | Number of reviewer personas |
ACCEPT_THRESHOLD | 6.0 | Minimum avg score for automatic accept |
STRONG_ACCEPT_THRESHOLD | 7.0 | Minimum avg score for strong accept |
BORDERLINE_THRESHOLD | 5.0 | Minimum avg score before auto-reject |
REJECT_THRESHOLD | 4.0 | Below this triggers automatic refinement |
MAX_REFINEMENT_ITERATIONS | 2 | Maximum refinement attempts before user decision |
NUM_QUERIES | 4 | Search queries extracted from idea (Step 0.5) |
MAX_RESULTS_PER_QUERY | 10 | Results per query per source (Step 0.5) |
DEFAULT_SOURCES | arxiv,semantic_scholar,openalex | Search sources for novelty verification |
YEAR_WINDOW | 3 | Years back to search from current year |
CRITICAL_OVERLAP_FAST_FAIL | true | User checkpoint on critical overlap detection |
novelty_grounding_report.txt, then full text copied into logs/idea_eval_agent_novelty.jsoneval_persona_1_review.txt, then full text copied into logs/idea_eval_agent_persona_1.jsoneval_persona_2_review.txt, then full text copied into logs/idea_eval_agent_persona_2.jsoneval_persona_3_review.txt, then full text copied into logs/idea_eval_agent_persona_3.jsoneval_report.txt, then full text copied into logs/idea_eval_agent_meta_review.jsoncontext_variables["idea_evaluation_result"] set with complete structured data (including novelty_verification)selected_idea.txt updated, final_selected_idea_data updated.txt files written to Ideation/ideas/, all .json files written to Ideation/ideas/logs/