| name | sciatlas-idea-evaluate |
| description | Use only SciAtlas search-papers to take a novice user from zero setup to an evidence-grounded evaluation of a research idea's novelty, feasibility, soundness, and differentiation. Trigger when the user asks whether an idea is worth pursuing or needs literature-backed critique. |
SciAtlas Idea Evaluate
Use this skill to evaluate a research idea with evidence from search-papers only.
Operating Contract
- Do not call any SciAtlas downstream command. The only SciAtlas retrieval command allowed is
search-papers.
- Ask the user only for email, verification code, token, or one necessary clarification.
- Run setup and retrieval yourself where possible.
- Do not expose the full token.
Zero-Start Bootstrap
- Check whether the
sciatlas executable exists without invoking a SciAtlas command: use Get-Command sciatlas -ErrorAction SilentlyContinue on Windows PowerShell or command -v sciatlas on macOS/Linux. Install if missing with python -m pip install -e ./sciatlas or python -m pip install "git+https://github.com/zjunlp/SciAtlas.git#subdirectory=sciatlas".
- If no
SCIATLAS_API_KEY is available, open or provide http://scinet.openkg.cn/register.
- Walk the novice user through registration. Ask for the email verification code when required and the returned
sciatlas_xxx token when registration finishes.
- Configure the shell:
$env:SCIATLAS_API_BASE_URL = "http://scinet.openkg.cn"
$env:SCIATLAS_API_KEY = "<token>"
setx SCIATLAS_API_BASE_URL "http://scinet.openkg.cn"
setx SCIATLAS_API_KEY "<token>"
export SCIATLAS_API_BASE_URL="http://scinet.openkg.cn"
export SCIATLAS_API_KEY="<token>"
- Use only
search-papers for any token smoke test.
Evaluation Frame
Convert the idea into four claims:
- Novelty claim: what is supposed to be new.
- Feasibility claim: why it can be built with available methods/data.
- Soundness claim: why the mechanism should work.
- Differentiation claim: why it is not just a minor variant.
Search Plan
Run only search-papers. Use one direct search and, if needed, one stress-test search:
sciatlas search-papers --retrieval-mode hybrid --query "<research idea>" --keyword "high:<core mechanism>" --keyword "middle:<application domain>" --top-k 8 --top-keywords 0 --max-titles 0 --max-refs 0 --bias-keyword high --bias-related high --bias-citation low --bias-exploration low --ranking-profile precision --report-max-items 8
Stress-test novelty by searching for the closest alternative wording:
sciatlas search-papers --retrieval-mode hybrid --query "<closest competing formulation>" --keyword "high:<problem>" --keyword "middle:<neighbor method>" --top-k 8 --top-keywords 0 --max-titles 0 --max-refs 0 --bias-related high --ranking-profile precision --report-max-items 8
Reading Artifacts
Read report.md and response.json. Build an evaluation ledger:
- Evidence for novelty.
- Evidence against novelty.
- Evidence for feasibility: data, implementation pattern, benchmark, or system precedent.
- Evidence for soundness: theory, empirical result, mechanism, or ablation.
- Missing evidence.
Evaluation Method
Score each dimension as High, Medium, Low, or Unknown:
- Novelty: High only if close papers solve a different problem or use a meaningfully different mechanism.
- Feasibility: High only if retrieved work shows required components, data, or evaluation are realistic.
- Soundness: High only if evidence supports the core causal mechanism, not merely the topic.
- Differentiation: High only if the idea can be expressed as a clear contrast with closest prior work.
For each score:
- cite the supporting papers from the search result;
- explain the reasoning in plain language;
- state the biggest uncertainty;
- suggest one concrete improvement.
Deliverable
Return:
- Search command(s), with token omitted.
- Evaluation table: novelty, feasibility, soundness, differentiation.
- Closest-prior-art risk.
- Go / revise / no-go recommendation.
- Revised idea statement and next
search-papers query.
