ref-check

name

ref-check

description

Verify the authenticity, accuracy, and formatting of bibliographic references. Use this skill whenever the user wants to check, validate, or verify references (also called citations, bibliography, works cited, 参考文献). This includes: checking if references are real, verifying DOIs, detecting AI-fabricated citations, checking formatting consistency, finding duplicates, or producing a verification report. Trigger even if the user just says "check my references", "核查参考文献", "verify citations", or provides a document and asks about reference quality. Supports all reference types: journal articles, conference papers, preprints, books, reports, blog posts, webpages, datasets, and more.

Reference Checker

Verify the authenticity, accuracy, and formatting of bibliographic references, then produce a Word (.docx) verification report.

Workflow

Step 1: Extract and Understand

Read the user's input (Word doc, PDF, plain text, or pasted content) and extract all references. For each reference, identify as many fields as possible:

• Authors, title, source (journal / website / publisher), year
• Volume, issue, pages (if applicable)
• DOI or URL (if present)
• Reference type: journal article, preprint, book, report, blog, webpage, conference paper, dataset, thesis, etc.

After extraction, present a brief summary to the user showing what you found — total count, breakdown by type, and any references that look unusual at first glance (e.g., missing fields, unusual formatting).

Step 2: Socratic Dialogue

After extracting references, STOP. Do not proceed to verification. Present what you found and have a conversation with the user first.

This is a hard stop — you must send a message to the user, wait for their reply, and only continue after they respond. The reason: verification choices (citation style, target venue) fundamentally change what gets checked. Guessing wastes time and produces a report the user may not need.

What to present: Show the user your extraction results — total count, types found, and any references that look unusual at first glance. Then ask about:

1. Report language: Chinese or English?
2. Target venue: Which journal or conference are they submitting to? This determines the citation style (APA, GB/T 7714, Vancouver, Nature, IEEE, etc.) and whether formatting compliance matters.
3. Citation style: If no target venue, ask which format to check — or whether to skip formatting checks.
4. Optional extras: Journal indexing info (SCI/SSCI/EI, impact factor)? Default: no.
5. Any known concerns: Are there specific references they're already worried about?

Adapt your phrasing to the conversation's language and tone. If the user already provided some of this information, acknowledge it and only ask about the gaps. Be conversational, not formulaic.

After the user replies, confirm the plan before proceeding: "I'll verify all N references, check formatting against [style], and generate a [language] report. Sound good?"

Only after confirmation, move to Step 3.

Step 3: Verify Each Reference

Core principle: CrossRef API first, web search for secondary verification, only conclude after cross-checking.

3a. CrossRef API Query

Use scripts/crossref_lookup.py to query each reference:

python <skill-path>/scripts/crossref_lookup.py --title "Paper Title" --author "First Author Last Name"

If the reference includes a DOI, also validate it:

python <skill-path>/scripts/crossref_lookup.py --doi "10.xxxx/xxxxx"

Use --batch mode for efficiency when checking multiple references.

3b. Interpret CrossRef Results

After getting CrossRef results, classify into three cases:

Case A — Strong match (match_score ≥ 80, key fields consistent): Title, authors, journal, and year all align with the citation. The reference is verified, but still cross-check minor details (volume, pages) against the CrossRef data. If there are small discrepancies (e.g., one-year difference between online and print publication), mark as "needs attention" with an explanation.

Case B — Partial match (match_score 40–80, or some fields differ): CrossRef found something related but not conclusive. Proceed to Step 3c for web search verification. Do not draw conclusions from API results alone.

Case C — No match (match_score < 40, or no results): CrossRef found nothing. This does NOT mean the reference is fake — many legitimate works are absent from CrossRef (Chinese journals, blogs, reports, webpages, very recent publications). Proceed to Step 3c.

3c. Web Search Secondary Verification

Perform web search verification in these situations:

• CrossRef returned Case B or C
• The reference type is not suited for CrossRef (blogs, reports, webpages, Chinese-language works)
• URL accessibility needs to be confirmed

Strategies by type:

• English academic works: Search the full title in quotes
• Chinese-language works (中文文献): Chinese journals are mostly absent from CrossRef. Search the Chinese title, look for results from CNKI, Wanfang, or VIP databases
• Blogs, reports, webpages: Use WebFetch to access the URL. Confirm the content exists and matches the citation
• Preprints (arXiv, SSRN, bioRxiv): Search the preprint platform directly

3d. Final Verdict

Based on combined CrossRef + web search results, assign each reference a status:

• ✅ Verified: At least one authoritative source fully confirms the reference's existence and key metadata
• ⚠️ Needs Attention: The reference exists but has detail issues (slight year/page discrepancy, formatting problems, partially broken URL, etc.)
• ❌ Problem Found: After both API and web search verification, strong evidence of serious issues
• ℹ️ Unable to Verify: Neither confirming nor disconfirming evidence found (internal reports, personal communications, etc.)

3e. AI Fabrication Detection

Only flag a reference as "suspected AI fabrication" when MULTIPLE signals co-occur:

• Title sounds plausible but no trace anywhere (both API and web search return nothing)
• Authors are real people but have never co-published in this field
• The journal exists but has no record of this article
• DOI format is valid but does not resolve
• Internal metadata contradictions (e.g., volume/year mismatch for the journal)

A single missing signal is not sufficient — real papers can be new, niche, or behind paywalls. The pattern of multiple red flags is what matters.

Step 4: Additional Checks

• Duplicate detection: Flag entries citing the same work (even if formatted differently), including preprint–published version pairs
• Format compliance: If a citation style was specified, check each reference against that style's rules. Note specific deviations (missing punctuation, wrong author name format, incorrect italicization, etc.)
• Year plausibility: Flag references dated beyond the current year — they may be forthcoming or fabricated
• URL accessibility: For references with URLs, check whether the links are still live

Step 5: Generate Word Report

Generate the report using python-docx, dynamically adapting to the actual verification results. Read references/report-spec.md for the exact structure, styling, and formatting requirements. Follow that spec strictly to ensure consistency across reports.

Process:

1. Read references/report-spec.md to understand the report template
2. Write a Python script using python-docx that generates the .docx based on the verification results
3. Run the script to produce the final report

The level of detail for each reference can vary — some may need extensive findings while others are straightforward. But the overall structure and visual style must follow the spec.

Step 6: Present Results

Save the report and tell the user where to find it. Provide a brief verbal summary: how many passed, how many have issues, and what the most critical findings are.

Important Notes

• Be conservative with conclusions. Only mark a reference as "problem found" after both CrossRef and web search fail to confirm it AND there is positive evidence of an issue. "Not found" ≠ "fabricated."
• CrossRef has limited coverage. It skews toward English-language and Western publishers. For Chinese, Japanese, Korean, and other non-English references, web search is often the primary verification method.
• "Unable to verify" ≠ "problem." Internal reports, personal communications, and unpublished manuscripts are inherently hard to verify. Label them honestly rather than forcing a judgment.
• Respect API rate limits. CrossRef has a polite-use policy; the script includes appropriate delays. Be mindful during batch queries.
• Current date matters. Year plausibility checks depend on knowing today's date. Future-dated references need scrutiny but aren't automatically wrong (forthcoming works exist).