| name | opennovelty-scholarly-assessment |
| title | OpenNovelty: An LLM-powered Agentic System for Verifiable Scholarly Novelty Assessment |
| version | 0.0.2 |
| engine | skillxiv-v0.0.2-claude-opus-4.6 |
| license | MIT |
| url | https://arxiv.org/abs/2601.01576 |
| keywords | ["Novelty Assessment","Scholarly Evaluation","LLM Agents","Peer Review","Evidence Grounding"] |
| description | Build agentic systems for transparent, evidence-based novelty analysis of research submissions through four-phase pipelines: contribution extraction, prior work retrieval, hierarchical comparison, and structured reporting with explicit citations—enabling fair peer review at scale. |
Overview
OpenNovelty is an LLM-powered agentic system designed to assess research novelty in a transparent, verifiable manner. It addresses a critical challenge in peer review: evaluating submissions against rapidly evolving literature requires extensive domain knowledge and careful analysis.
Core Innovation: Unlike naive LLM-based approaches that generate unsupported assessments, OpenNovelty grounds all novelty judgments in retrieved real papers, ensuring verifiable, evidence-backed evaluation.
Four-Phase Assessment Pipeline
Phase 1: Contribution Extraction
Extract core task and specific contribution claims from research submissions.
Process:
- Identify the paper's central research task/problem
- Extract explicit contribution claims (technical, methodological, empirical)
- Generate retrieval queries capturing contribution scope
- Structure contributions for downstream analysis
Output: Structured specification of contributions ready for evidence gathering.
Phase 2: Prior Work Retrieval
Retrieve relevant prior work using semantic search and knowledge engines.
Implementation:
- Query generation from extracted contributions
- Semantic similarity search across paper databases
- Ranked retrieval of potentially relevant prior work
- Handles both well-known and obscure related work
Advantages:
- Discovers closely related papers authors may overlook
- Systematic coverage of related literature
- Evidence-based rather than recollection-based
Phase 3: Hierarchical Taxonomy Construction & Full-Text Comparison
Build a hierarchical taxonomy of prior work organized by contribution category, then perform detailed comparisons.
Taxonomy Structure:
- Level 1: Core task (same research problem)
- Level 2: Methodological approach (similar techniques)
- Level 3: Specific innovations (targeted improvements)
Comparison Process:
- Extract relevant details from each prior work paper
- Compare against submission's contributions point-by-point
- Identify overlaps, incremental vs. novel aspects
- Document evidence snippets from papers
Phase 4: Structured Novelty Report
Synthesize analyses into comprehensive report with explicit citations and evidence.
