| name | hfpclawer-academic-integrity |
| description | Feed a paper draft to automatically extract citations, verify each one against local + Semantic Scholar + OpenAlex, detect fabricated references, and generate a structured integrity report. Designed for researchers, reviewers, and literature-survey authors.
|
| category | research |
| author | Li Shen |
| version | 1.0.0 |
| metadata | {"hermes":{"tags":["academic-integrity","citation","verification","hallucination","research","audit"],"related_skills":["hfpclawer-paper-search","hfpclawer-citation-audit"],"requires":[{"hermes_tool":"web_extract"},{"mcp_server":"arxiv-search (for citation_graph)"}]}} |
hfpclawer Academic Integrity Audit
Take a paper draft (text, URL, or PDF) and verify every citation against
local FTS5 โ Semantic Scholar โ OpenAlex. Spot fabricated references,
misattributed authors, and wrong publication years.
Who this is for: Researchers who need to fact-check citations in
LLM-generated papers, peer review submissions, or their own drafts
before submission.
Overview
User: "Check this paper draft for citation integrity"
โ
โโโโโโโโโโโโโโดโโโโโโโโโโโโโ
โผ โผ
Extract citations Load paper (text/URL/MEDIA pdf)
โ โ
โโโโโโโโโโโโโโฌโโโโโโโโโโโโโ
โผ
โโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ For each citation: โ
โ L1: Local FTS5 (1ms) โ โ hfpclawer audit
โ L2: Semantic Scholar โ โ hfpclawer citation_audit_s2
โ L3: OpenAlex โ โ hfpclawer citation_audit_oa
โ L4: Citation Graph โ โ MCP arxiv-search citation_graph
โโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ
โผ
โโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ Integrity Report โ
โ [OK] VERIFIED 12 โ
โ [?] SUSPECTED 2 โ
โ [NF] NOT_FOUND 0 โ
โ [ERR] ERROR 1 โ
โ โ FABRICATED 1 โ โ title+author mismatch => hallucination
โโโโโโโโโโโโโโโโโโโโโโโโโโโ
Prerequisites
pip install hfpclawer>=0.5.0
Optional but recommended:
Quick Start
1. Verify a Draft Paper (most common)
Paste the paper text (including its references section) and ask:
> Run academic integrity audit on this draft.
[paste paper text with references]
# Output:
# โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
# โ Academic Integrity Audit Report โ
# โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
# โ [OK] VERIFIED "Fourier Neural Operator" (Li, 2020) โ
# โ [OK] VERIFIED "Physics-Informed Neural Networks" (2019) โ
# โ [?] SUSPECTED "Deep Learning for PDEs" โ โ no author/date
# โ [NF] NOT_FOUND "Novel Superconductor at 300K" (Smith 2025)โ โ possible hallucination
# โ [NF] NOT_FOUND "Quantum PDE Solver" (Chen 2024) โ โ possible hallucination
# โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
2. Audit a Paper from a URL (arXiv / blog / preprint)
> Check this paper for citation integrity:
> https://arxiv.org/abs/2010.08895
The skill will:
- web_extract the paper text
- Parse the references section
- Run each citation through L1โL2โL3โL4
- Return a formatted integrity report
3. Audit a Local PDF or Markdown File
Provide the file via Hermes Agent:
> Read and audit citations in MEDIA:/path/to/paper.pdf
The skill reads the text, extracts citations, and reports.
How It Works
Citation Extraction
The extractor finds these patterns:
"Title" (Author, Year) โ author-year-title formatAuthor (Year) "Title" โ title-quoted formatarXiv:XXXX.XXXXX โ arxiv IDs[1] Author. "Title". Journal. Year. โ numbered bibliography (basic)
Verification Cascade
| Level | Source | Speed | What It Confirms |
|---|
| L1 | Local FTS5 (arxiv_meta.db) | ~1ms | arXiv ID + title match |
| L2 | Semantic Scholar API | ~200ms | Title + authors + year |
| L3 | OpenAlex API | ~200ms | Title + authors + venue + DOI |
| L4 | Citation Graph (MCP) | ~500ms | Citing/cited-by validation |
Status Definitions
| Status | Icon | Meaning |
|---|
| VERIFIED | โ
| Found in โฅ1 source with title similarity โฅ0.70 |
| SUSPECTED | โ ๏ธ | Possible match (0.40-0.69) โ may be real but uncertain |
| NOT_FOUND | โ | No match in any source โ potential hallucination |
| FABRICATED | ๐จ | NOT_FOUND + no author match + unlikely title โ likely hallucination |
| ERROR | ๐ง | Source unavailable (DB not found, rate limited) |
Integrity Report Example
โโโ Academic Integrity Audit โโโ
Paper: "A Survey of Neural PDE Solvers"
Total citations scanned: 22
Time: 2026-05-20T14:30:00Z
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ
VERIFIED 15/22 (68%)
โ ๏ธ SUSPECTED 4/22 (18%)
โ NOT_FOUND 2/22 (9%) โ review these
๐จ FABRICATED 1/22 (5%) โ "Quantum Fourier Neural Net" (Li 2025)
๐ง ERROR 0/22 (0%)
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
Verification breakdown by source:
L1 Local FTS5: 11 verified
L2 Semantic Scholar: +6 verified (cascaded)
L3 OpenAlex: +2 verified (cascaded)
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
FABRICATED citations:
๐จ "Quantum Fourier Neural Net" (Li 2025)
No arXiv match, no S2 match, no OpenAlex match
Title sounds AI-generated (no such paper exists)
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
Recommendations:
- Remove 1 fabricated citation
- Check 4 suspected citations manually (2 may be preprints w/o arXiv ID)
- Add DOI/arXiv IDs to 3 citations for future verifiability
Common Pitfalls
-
arXiv-only papers may not appear in S2 or OpenAlex for 1-2 weeks after posting.
If a very recent paper shows NOT_FOUND, try L4 (citation graph) or check manually.
-
Title substring matching can produce false VERIFIED for short titles.
e.g. "Neural Operator" โ matches "Fourier Neural Operator" (accurate)
but "Deep Learning" โ too generic, will NOT_FOUND or false-match.
-
S2 rate limits without API key: ~1 req/s, ~100 req/day. Set S2_API_KEY.
-
Local FTS5 requires arxiv-metadata-service clone.
Without it, the cascade starts at L2 (slower but works).
-
Citation extractor is regex-based, not ML-based. Some citation styles
(especially Chicago notes-bibliography) may not be parsed correctly.
For best results, provide the references section in plain author-title-year format.
Verification Checklist
