arxiv-novelty-review

name	arxiv-novelty-review
description	Judge whether a research paper, arXiv preprint, proposal, method, or idea has a real innovation point. Use when Codex needs to assess novelty, contribution strength, research taste, prior-work collisions, latest arXiv related work, literature-tree position, challenge-insight coverage, or whether an idea is already covered by recent papers.

arXiv Novelty Review

Overview

Use this skill to evaluate novelty with primary research evidence, especially arXiv papers. The output should be a defensible reviewer-style judgment, not a yes/no impression.

Always browse or otherwise verify current literature when judging novelty. arXiv and paper PDFs are primary sources; blogs, social posts, and summaries are only discovery aids.

Core Principles

1. Claim first: Convert the work into one or more precise novelty claims before searching.
2. Latest papers first: Prioritize recently submitted and recently updated arXiv papers. Many collisions come from the last few months, before citations exist.
3. Closest work over broad survey: Identify the nearest prior works by task, challenge, method, insight, and evidence.
4. Version-aware judgment: Record arXiv v1 date separately from latest updated version. Use v1 for priority and the latest version for current content.
5. No novelty without knowledge gain: A contribution should teach the field something new: task, setting, pipeline, representation, module, design choice, finding, or insight.
6. Well-established solution check: If the same failure case already has a mature solution in an equal or more general setting, novelty is weak unless the new work is simpler, broader, or reveals a new insight.

Workflow

1. Extract The Claim

Rewrite the paper or idea into a minimal claim:

In [task / setting / data / failure case],
the work addresses [technical challenge],
by proposing [pipeline / representation / module / objective / training strategy / experiment],
which yields [new capability / result / finding / insight].

If there are multiple claims, review each separately. Do not let a broad story hide a weak technical claim.

2. Build Search Terms

Create six keyword groups:

• Task: task name, input-output setting, benchmark setting.
• Failure case: limitation, hard case, failure mode, robustness gap.
• Technical challenge: underlying technical reason.
• Method: pipeline, representation, module, objective, training, inference, agent/tool mechanism.
• Dataset/benchmark: dataset names, metrics, protocols.
• Synonyms: older names, newer names, abbreviations, adjacent-domain terminology.

3. Search arXiv In Three Passes

Use arXiv Advanced Search, arXiv API, or web search restricted to arxiv.org. Use fielded searches where useful: title, abstract, author, category.

Pass A: Direct nearest neighbors

Search:

[task] [method]
[task] [representation]
[task] [module]
[task] [dataset]

Pass B: Problem nearest neighbors

Search:

[failure case] [task]
[technical challenge] [task]
[limitation] [task]

Pass C: Cross-domain technical neighbors

Search:

[technical challenge] [method]
[representation] [failure case]
[module/objective/training trick] [challenge]

For fast-moving areas such as LLMs, agents, diffusion/video generation, 3D generation, and RL:

• Carefully inspect the last 3-6 months.
• Sweep the last 12 months.
• If the area has a longer lineage, include 24 months plus seminal papers.

4. Make A Nearest-Work Table

Record every plausible collision:

| Work | arXiv ID | v1 date | latest version | Task overlap | Challenge overlap | Method overlap | Insight overlap | Evidence overlap | Notes |

Use high/medium/low overlap. Include at least:

• most recent similar papers;
• earliest similar paper;
• current SOTA or most cited baseline;
• same task with different method;
• same method/challenge in another domain.

5. Check Novelty Collision

For each nearest work, answer:

1. Task collision: Is the task/input-output/setting effectively the same?
2. Challenge collision: Is it solving the same failure case or technical challenge?
3. Method collision: Is the pipeline/module/objective/training strategy substantially the same?
4. Insight collision: Did prior work already state the same motivation, explanation, or mechanism?
5. Evidence collision: Did prior work already verify the same result on the same or stronger benchmark?

Decision hints:

• Same task + same challenge + same method + same insight = likely not novel.
• Same task/challenge but clearly different method = possible method novelty.
• Similar method but new task/data with new finding = weak to medium novelty, depending on importance.
• Simple A+B composition = weak unless the combination creates a new capability, mechanism, or generalizable insight.

6. Classify Contribution Type

Assign the strongest justified type:

A. Novel task / setting
B. Novel pipeline / representation
C. Novel module / objective / design choice
D. Novel non-obvious combination
E. New experimental finding / insight
F. Incremental improvement

If the contribution is only better metrics, ask whether the improvement is large, surprising, robust, and supported by strong baselines.

7. Apply Reviewer Risk Checks

Downgrade novelty if:

• the failure case is common and already has a mature solution;
• the technical idea is well-explored and the gain is predictable;
• the newest related arXiv papers already cover the claim;
• the work omits strong recent baselines;
• evidence is limited to one easy benchmark;
• ablations do not isolate the claimed module or insight;
• the method introduces stronger limitations than its benefits;
• the contribution is task-specific trickery without broader insight.

Remember: arXiv papers are not necessarily peer-reviewed. Treat them as priority and collision evidence, not automatic truth.

Output Format

Use this format for final novelty reviews:

Claim:
...

Latest Paper Coverage:
- Search date:
- Time window:
- Recent nearest papers:
- Possible latest collision:

Closest Prior Works:
| Work | arXiv ID | v1 date | overlap summary |

Novelty Collision:
- Task:
- Challenge:
- Method:
- Insight:
- Evidence:

Well-established Solution Check:
...

Contribution Type:
...

Evidence Strength:
- Baselines:
- Ablations:
- Datasets/metrics:
- Failure-case analysis:

Risk:
...

Final Judgment:
A / B / C / D / U

One-sentence Verdict:
...

Judgment scale:

• A strong novelty: new important task, pipeline, representation, or insight with no close arXiv collision.
• B valid novelty: close work exists, but the method/insight/evidence gap is substantive.
• C weak novelty: incremental or mostly recombination, but experimentally useful.
• D not novel enough: recent or prior work substantially covers the claim.
• U uncertain: search coverage, version history, or paper access is insufficient.