| name | citation-verification |
| description | Use when verifying that citations in a manuscript are real, correctly attributed, and actually support the claims they're attached to. Covers existence/metadata verification against Crossref, OpenAlex, and Semantic Scholar; detecting hallucinated and chimeric references (real title, wrong authors); retraction checking; and claim-support auditing (does the cited paper say what the manuscript says it says). |
Citation verification
Two distinct problems, often conflated:
| Problem | Question | Failure mode |
|---|
| Existence/metadata | Is this reference real, and are its fields right? | Hallucinated or chimeric entries, wrong year/venue/pages |
| Claim-support | Does the source say what the manuscript claims? | Real paper, misrepresented finding |
Different tools for each: existence is checked against bibliographic APIs; claim-support requires reading the actual source. A bibliography can pass existence checks completely and still misrepresent half its sources.
The verification trio
Three free APIs, all curl-able without keys:
| API | Endpoint | Notes |
|---|
| Crossref | api.crossref.org/works/<DOI> or ?query.bibliographic=<title+authors> for fuzzy | Authoritative for DOI metadata |
| OpenAlex | api.openalex.org/works/doi:<DOI> or ?filter=title.search:<title> | No auth, the full scholarly graph, has is_retracted |
| Semantic Scholar | api.semanticscholar.org/graph/v1/paper/DOI:<DOI> | Unauthenticated access shares a global pool and throttles under load; go sequential, sleep between calls |
Be polite: set a User-Agent with contact info, query sequentially with short sleeps.
The decision rule
For each reference, query all three and compare title, authors, year, venue:
- 2+ sources confirm → verified.
- Exactly 1 confirms → suspicious. One index can be wrong or stale; say so, don't bless it.
- 0 confirm → not found. Likely hallucinated.
- Title matches but authors don't (or vice versa) → chimeric. The most dangerous class: a real title welded to wrong authors looks plausible to every casual checker, including the original author skimming their own bibliography.
Red-flag heuristics
Cheap checks before any API call:
- Malformed DOI (doesn't match
10\.\d{4,}/\S+).
- Publication year in the future, or before the venue existed.
- Suspiciously generic survey title ("A Survey of Deep Learning Methods for X"); LLMs invent these constantly.
- Page ranges that don't parse (end < start, or absurd spans).
- The double-real trap: the DOI resolves fine, but to a different paper than the entry describes. Always compare resolved metadata against the entry, never just check that the DOI resolves.
Retraction checking
- Crossref: a retraction notice carries an
update-to field pointing at the retracted DOI. Query api.crossref.org/works?filter=updates:<DOI> to find notices targeting a given work.
- OpenAlex: the work record has a plain
is_retracted boolean. Check it for every reference; it's free.
A retracted citation is CRITICAL regardless of how it's used, unless the manuscript cites it as retracted.
The 5-layer audit
Full audits proceed in order; each layer assumes the previous one passed.
| Layer | Checks |
|---|
| L1 Authenticity | Reference exists (the decision rule above) |
| L2 Bibliographic accuracy | Every field matches canonical metadata: authors, year, venue, volume, pages, DOI |
| L3 Text↔list consistency | Every in-text citation has a list entry; every entry is cited; keys resolve |
| L4 Appropriateness | The source supports the claim it's attached to, and isn't cited for something it argues against (the worst L4 failure, and common) |
| L5 Formatting/version | Consistent style; arXiv preprint cited where a published version exists; stale "in press" entries |
L1–L3 are mechanical and scriptable. L4 requires reading sources. L5 is cleanup.
Hard rules
Non-negotiable:
- Never emit a BibTeX entry from memory. Fetch the canonical entry via DOI content negotiation:
curl -LH "Accept: application/x-bibtex" https://doi.org/<DOI>. Memory-generated BibTeX is exactly how chimeric references are born.
- An unverifiable claim gets
[CITATION NEEDED], never a guessed reference. A visible gap is fixable; a plausible fake is poison.
- Claims a source doesn't support get dropped or narrowed, not softened. "May suggest" in front of a misattributed finding is still a misattributed finding.
- Every claim-support verdict must show a verbatim source quote that would substring-match the source text. No quote, no verdict: a paraphrase presented as evidence is itself a fabrication.
Related: [[citation-formatting]].
