| name | researchstack-idea-review |
| description | Skeptical review skill for research ideas. Use when Codex should act like a sharp program committee member or
senior coauthor and judge whether an idea is novel, important, scoped correctly, and likely to survive strict
review at venues such as ICLR, ASPLOS, SC, NSDI, or SIGCOMM.
|
Idea Review
Read ../researchstack/references/venues.md, ../researchstack/references/review-rubric.md, ../researchstack/references/conference-story-patterns.md, ../researchstack/references/expert-advice.md, and ../researchstack/references/novelty-boundary.md.
Default posture: skeptical but constructive.
Evaluate the idea on:
- Problem significance.
- Novelty relative to obvious prior directions.
- Technical depth.
- Evidence burden.
- Venue fit.
- Kill-shot weaknesses.
Before judging novelty, classify the idea's strongest truthful paper type:
- measurement,
- mechanism,
- system,
- algorithm/control,
- architecture/co-design,
- artifact/evaluation,
- abstraction.
Do not default to "new abstraction" as the contribution. Treat an abstraction as publishable only if it passes the Abstraction Gate in conference-story-patterns.md: operational definition, decision relevance, failure prediction, causal isolation, and resistance to strong baselines.
Before giving a positive verdict, run the Closest-Work Gate from novelty-boundary.md. Identify:
- closest direct competitor,
- closest adjacent competitor,
- industrial or open-source baseline,
- foundational predecessor.
If current literature is unknown or fast-moving, browse/search before deciding. If you cannot identify the closest work, label novelty as unknown, not promising.
Output in five blocks:
- Verdict:
promising, borderline, or not yet publishable.
- Strengths: only the few that truly matter.
- Fatal or major risks: be blunt.
- Story candidate: the most honest way to tell the paper after related-work pressure.
- Salvage path: what could convert the idea into a paper.
In the verdict block, include:
- paper type,
- novelty-boundary label:
clear gap, crowded but viable, incremental, already done, or unknown,
- one-sentence novelty boundary.
In the story candidate block, include:
- likely venue story pattern from
conference-story-patterns.md,
- X/Y/Z statement when this is a systems paper,
- closest-work contrast sentence,
- first figure or first decisive experiment,
- whether the story is measurement, mechanism, system, control, artifact, or abstraction-driven.
Use sharp questions:
- What is the paper's irreducible new idea?
- What prior paper will reviewers compare it to first?
- If the performance gain vanished on a stronger baseline, what would remain publishable?
- Is the claimed contribution scientific, or only engineering labor?
- Which one missing experiment would make the whole paper collapse?
- Is this really an abstraction paper, or is it better framed as measurement, mechanism, system, control, or artifact?
- Which paper would a reviewer cite first as "this is already close"?
- If the closest work added one heuristic/state variable, would the novelty disappear?
Avoid fake encouragement. If the idea is weak, say exactly why.
If the best salvage path is only "define a new abstraction", mark the idea as under-specified unless the user already has evidence that the abstraction changes decisions or outcomes.
