citation-management

name	citation-management
description	Complete citation lifecycle management for academic writing: collect citations from DOI/OpenAlex/arXiv, verify against Google Scholar to prevent AI-faked references, convert between BibTeX/APA/MLA/Chicago/Vancouver/IEEE formats, and retrieve open-access full-text via Unpaywall
tags	["Research","Academic","Citation","Reference","Bibliography","BibTeX","APA","MLA","Chicago","Vancouver","IEEE","Open Access","Unpaywall","Verification"]
version	1.0.0
metadata	{"openclaw":{"emoji":"🔖","category":"writing","subcategory":"citation","keywords":["citation","BibTeX","APA","reference management","bibliography","formatting","citation verification","open access","Unpaywall","DOI"],"source":"wentor-research-plugins"}}

Citation Management

Complete academic citation lifecycle: collect citations from DOIs and APIs, verify they actually exist (anti-AI-hallucination), convert between formats, and retrieve open-access full-text — all with scriptable Python/Bash tooling.

Overview

Citation management is a persistent friction point in academic writing. Researchers collect references from multiple sources (databases, PDFs, colleagues, web pages), store them in different formats, and must output them in the specific style required by each target journal. Errors in citations — misspelled author names, incorrect years, broken DOIs, inconsistent formatting — are among the most common reasons for desk rejection and reviewer criticism.

In the age of AI-assisted writing, an additional critical problem has emerged: AI models fabricate citations at a rate of approximately 40%. Every citation generated by an LLM must be proactively verified before inclusion.

This skill provides a four-phase citation lifecycle that goes beyond what GUI reference managers offer. It can retrieve complete metadata from a DOI in seconds, verify paper existence on Google Scholar, convert between any citation format, detect and merge duplicate entries, validate entries against CrossRef and OpenAlex databases, generate properly formatted bibliographies for any major citation style, and find open-access full-text versions of paywalled papers via Unpaywall.

The approach is text-based and scriptable, making it ideal for integration with LaTeX workflows, Markdown writing pipelines, and automated document generation. All citation data is stored in standard BibTeX format as the canonical source, with on-demand conversion to other formats for specific manuscript requirements.

---

Phase 1: Collect Citations

Retrieve complete citation metadata from identifiers (DOI, arXiv ID, OpenAlex Work ID) and output clean BibTeX.

From DOI (CrossRef)

import requests

def get_bibtex_from_doi(doi):
    """Retrieve BibTeX entry from a DOI via CrossRef."""
    url = f"https://doi.org/{doi}"
    headers = {"Accept": "application/x-bibtex"}
    response = requests.get(url, headers=headers, allow_redirects=True)
    if response.status_code == 200:
        return response.text
    return None

# Example
bibtex = get_bibtex_from_doi("10.1038/s41586-021-03819-2")
print(bibtex)
# @article{Jumper_2021,
#   title={Highly accurate protein structure prediction with AlphaFold},
#   author={Jumper, John and Evans, Richard and ...},
#   journal={Nature},
#   volume={596},
#   pages={583--589},
#   year={2021},
#   publisher={Springer}
# }

From OpenAlex

def get_citation_from_openalex(work_id):
    """Retrieve citation data from OpenAlex API."""
    url = f"https://api.openalex.org/works/{work_id}"
    headers = {"User-Agent": "ResearchPlugins/1.0 (https://wentor.ai)"}
    response = requests.get(url, headers=headers)
    if response.status_code == 200:
        data = response.json()
        return format_as_bibtex(data)
    return None

def format_as_bibtex(oa_data):
    """Convert OpenAlex data to BibTeX."""
    authorships = oa_data.get("authorships", [])
    author_str = " and ".join(a["author"]["display_name"] for a in authorships)
    first_author = authorships[0]["author"]["display_name"].split()[-1] if authorships else "Unknown"
    year = str(oa_data.get("publication_year", ""))
    key = f"{first_author}_{year}"

    venue = oa_data.get("primary_location", {}) or {}
    journal = (venue.get("source") or {}).get("display_name", "")

    return f"""@article{{{key},
  title={{{oa_data.get('title', '')}}},
  author={{{author_str}}},
  year={{{year}}},
  journal={{{journal}}},
  doi={{{oa_data.get('doi', '')}}}
}}"""

From arXiv ID

def get_bibtex_from_arxiv(arxiv_id):
    """Retrieve BibTeX from arXiv."""
    import feedparser
    url = f"http://export.arxiv.org/api/query?id_list={arxiv_id}"
    feed = feedparser.parse(url)
    if feed.entries:
        entry = feed.entries[0]
        authors = " and ".join(a["name"] for a in entry.authors)
        first_author = entry.authors[0]["name"].split()[-1]
        year = entry.published[:4]
        return f"""@article{{{first_author}_{year},
  title={{{entry.title.replace(chr(10), ' ')}}},
  author={{{authors}}},
  year={{{year}}},
  journal={{arXiv preprint arXiv:{arxiv_id}}},
  url={{https://arxiv.org/abs/{arxiv_id}}}
}}"""
    return None

---

Phase 2: Verify Citations

AI-generated citations have a ~40% hallucination rate. Every citation must be proactively verified during the writing process — not as a post-processing step.

Verification Principles

Core Principle: Proactively verify every citation during the writing process using WebSearch and Google Scholar.

1. Proactive Verification (Verify During Writing)

Verify immediately when adding a citation, rather than checking after the manuscript is complete.

• Search for the paper via WebSearch each time a citation is needed
• Confirm the paper exists on Google Scholar
• Add to bibliography only after verification passes
• If citing a specific claim, confirm the claim actually appears in the paper

2. Google Scholar Verification

Google Scholar provides the most comprehensive academic literature coverage, citation counts as credibility indicators, and direct BibTeX export — all free without API keys.

Verification steps:

1. WebSearch query: "site:scholar.google.com [paper title] [first author]"
2. Confirm the paper appears in results
3. Check citation count (abnormally low counts may indicate issues)
4. Click "Cite" on Google Scholar
5. Select BibTeX format and copy the entry

Information that must match:

• Title (minor differences allowed, e.g., capitalization)
• Authors (at least first author must match)
• Year (±1 year difference allowed, considering preprints)
• Publication venue (conference/journal name)

3. Claim Verification

When citing a specific empirical claim:

• Use WebSearch to find the paper PDF
• Search within the PDF for relevant keywords
• Confirm the claim is actually supported
• Record the section/page where the claim appears

Verification Workflow

Need a citation during writing
    ↓
WebSearch to find the paper
    ↓
Google Scholar to verify existence
    ↓
Confirm paper details (title, authors, year, venue)
    ↓
Get BibTeX from Google Scholar or CrossRef
    ↓
(If citing a specific claim) Verify the claim in the paper
    ↓
Add to bibliography

Handling Verification Failures

Paper not found on Google Scholar:

1. Check spelling — title or author name may be wrong
2. Try different keyword combinations
3. Try alternative sources — arXiv, DOI direct lookup
4. Mark as [CITATION NEEDED] if still unfound
5. Notify the user — clearly state the citation cannot be verified

Information mismatch:

1. Confirm you found the correct paper
2. Check preprint vs. published version differences
3. Update information to the most accurate version
4. Record discrepancies with a note

Preventing Fake Citations

Wrong Approach	Correct Approach
Generating BibTeX from memory	Search every citation with WebSearch
Skipping Google Scholar verification	Confirm on Google Scholar
Assuming a paper exists	Copy BibTeX from Google Scholar
Not marking unverifiable citations	Clearly mark [CITATION NEEDED]

CrossRef Validation (Programmatic)

def validate_citation(doi):
    """Validate a citation against CrossRef metadata."""
    url = f"https://api.crossref.org/works/{doi}"
    response = requests.get(url)
    if response.status_code != 200:
        return {"valid": False, "error": "DOI not found in CrossRef"}

    data = response.json()["message"]
    return {
        "valid": True,
        "title": data.get("title", [None])[0],
        "authors": [f"{a.get('family', '')}, {a.get('given', '')}"
                    for a in data.get("author", [])],
        "year": data.get("published-print", {}).get("date-parts", [[None]])[0][0],
        "journal": data.get("container-title", [None])[0],
        "type": data.get("type", "unknown")
    }

Common Citation Errors

Error	Detection	Fix
Missing DOI	Check doi field is empty	Query CrossRef by title
Wrong year	Compare against CrossRef	Use CrossRef year
Author name variants	Fuzzy match against ORCID	Standardize to ORCID name
Duplicate entries	Title similarity > 85%	Merge into single entry
Broken URL	HTTP HEAD request returns 4xx/5xx	Update or remove URL
Incomplete entry	Missing required fields for style	Retrieve from DOI
Fake/AI-hallucinated citation	Paper does not exist in Google Scholar	Remove or find real replacement

---

Phase 3: Format Management

Convert between citation formats programmatically. BibTeX is the canonical storage format; convert to target style for manuscript submission.

BibTeX to APA 7th

def bibtex_to_apa7(entry):
    """Convert a parsed BibTeX entry to APA 7th edition format."""
    authors = format_apa_authors(entry["author"])
    year = entry.get("year", "n.d.")
    title = entry["title"]
    journal = entry.get("journal", "")
    volume = entry.get("volume", "")
    issue = entry.get("number", "")
    pages = entry.get("pages", "")
    doi = entry.get("doi", "")

    # Article format
    citation = f"{authors} ({year}). {title}. "
    if journal:
        citation += f"*{journal}*"
        if volume:
            citation += f", *{volume}*"
        if issue:
            citation += f"({issue})"
        if pages:
            citation += f", {pages}"
        citation += "."
    if doi:
        citation += f" https://doi.org/{doi}"

    return citation

def format_apa_authors(author_string):
    """Format author names in APA style: Last, F. M."""
    authors = [a.strip() for a in author_string.split(" and ")]
    formatted = []
    for author in authors:
        parts = author.split(", ") if ", " in author else author.rsplit(" ", 1)[::-1]
        if len(parts) >= 2:
            last = parts[0]
            firsts = parts[1].split()
            initials = " ".join(f"{f[0]}." for f in firsts)
            formatted.append(f"{last}, {initials}")
        else:
            formatted.append(parts[0])

    if len(formatted) == 1:
        return formatted[0]
    elif len(formatted) == 2:
        return f"{formatted[0]}, & {formatted[1]}"
    elif len(formatted) <= 20:
        return ", ".join(formatted[:-1]) + f", & {formatted[-1]}"
    else:
        return ", ".join(formatted[:19]) + f", ... {formatted[-1]}"

Format Examples

The same reference rendered in every supported style:

BibTeX (canonical):

@article{Jumper_2021,
  title={Highly accurate protein structure prediction with AlphaFold},
  author={Jumper, John and Evans, Richard and Pritzel, Alexander},
  journal={Nature},
  volume={596},
  pages={583--589},
  year={2021},
  doi={10.1038/s41586-021-03819-2}
}

APA 7th: Jumper, J., Evans, R., & Pritzel, A. (2021). Highly accurate protein structure prediction with AlphaFold. Nature, 596, 583-589. https://doi.org/10.1038/s41586-021-03819-2

MLA 9th: Jumper, John, Richard Evans, and Alexander Pritzel. "Highly Accurate Protein Structure Prediction with AlphaFold." Nature, vol. 596, 2021, pp. 583-89.

Chicago (Author-Date): Jumper, John, Richard Evans, and Alexander Pritzel. 2021. "Highly Accurate Protein Structure Prediction with AlphaFold." Nature 596: 583-89.

Vancouver: Jumper J, Evans R, Pritzel A. Highly accurate protein structure prediction with AlphaFold. Nature. 2021;596:583-9.

IEEE: J. Jumper, R. Evans, and A. Pritzel, "Highly accurate protein structure prediction with AlphaFold," Nature, vol. 596, pp. 583-589, 2021.

Deduplication

from difflib import SequenceMatcher

def find_duplicates(bib_entries, threshold=0.85):
    """Find duplicate BibTeX entries by title similarity."""
    duplicates = []
    titles = [(key, normalize_title(entry["title"]))
              for key, entry in bib_entries.items()]

    for i in range(len(titles)):
        for j in range(i + 1, len(titles)):
            similarity = SequenceMatcher(
                None, titles[i][1], titles[j][1]
            ).ratio()
            if similarity >= threshold:
                duplicates.append({
                    "entry_a": titles[i][0],
                    "entry_b": titles[j][0],
                    "similarity": similarity
                })
    return duplicates

def normalize_title(title):
    """Normalize title for comparison."""
    import re
    title = title.lower()
    title = re.sub(r'[{}\\]', '', title)  # Remove LaTeX formatting
    title = re.sub(r'[^a-z0-9\s]', '', title)  # Remove punctuation
    title = ' '.join(title.split())  # Normalize whitespace
    return title

def merge_duplicates(entry_a, entry_b):
    """Merge two duplicate entries, preferring the more complete one."""
    merged = {}
    all_fields = set(list(entry_a.keys()) + list(entry_b.keys()))
    for field in all_fields:
        val_a = entry_a.get(field, "")
        val_b = entry_b.get(field, "")
        # Prefer the longer (more complete) value
        merged[field] = val_a if len(str(val_a)) >= len(str(val_b)) else val_b
    return merged

---

Phase 4: Find Full Text (Open Access)

Many paywalled papers have legally available free versions (preprints, author manuscripts, institutional repositories). Unpaywall finds them with a simple REST API.

When to Use

• DOI resolution hits a paywall
• Paper not available in PubMed Central
• Publisher site requires subscription
• Need full text for a highly relevant paper

Use BEFORE giving up on full-text access.

Unpaywall API

Simple REST API — no authentication required for reasonable usage.

Parameters:

• DOI — The paper's DOI (URL-encoded if needed)
• email — User's email (required, for courtesy/contact)

IMPORTANT: Ask the user for their email at the start of the research session. Do NOT use placeholder emails like [e-mail] or [e-mail].

Example request:

curl "https://api.unpaywall.org/v2/10.1038/nature12373?email=[e-mail]"

Response format:

{
  "doi": "10.1038/nature12373",
  "title": "Paper Title",
  "is_oa": true,
  "best_oa_location": {
    "url": "https://europepmc.org/articles/pmc3858213",
    "url_for_pdf": "https://europepmc.org/articles/pmc3858213?pdf=render",
    "version": "publishedVersion",
    "license": "cc-by",
    "host_type": "repository"
  },
  "oa_locations": [
    {"url": "https://europepmc.org/articles/pmc3858213", "version": "publishedVersion"},
    {"url": "https://arxiv.org/abs/1234.5678", "version": "submittedVersion"}
  ]
}

Key response fields:

Field	Type	Meaning
is_oa	boolean	Open access version available
best_oa_location	object	Unpaywall's recommended best OA source (prioritizes published versions over preprints)
oa_locations	array	All known open access locations, ordered by version quality

version types (priority order):

1. publishedVersion — Final published version (best)
2. acceptedVersion — Author's accepted manuscript (good)
3. submittedVersion — Preprint before peer review (useful)

Implementation Pattern

1. Check Unpaywall After Paywall Hit

# Try DOI first
curl -L "https://doi.org/10.1234/example.2023"

# If paywall detected (403, subscription required, etc):
curl "https://api.unpaywall.org/v2/10.1234/example.2023?email=[e-mail]"

2. Extract Best URL

# Parse JSON response
response=$(curl -s "https://api.unpaywall.org/v2/DOI?email=EMAIL")

# Check if OA available
is_oa=$(echo $response | jq -r '.is_oa')

if [ "$is_oa" = "true" ]; then
  # Get best PDF URL
  pdf_url=$(echo $response | jq -r '.best_oa_location.url_for_pdf // .best_oa_location.url')

  # Download
  curl -L -o "papers/paper.pdf" "$pdf_url"
fi

3. Report to User

When OA found:

⚠️ Paper behind paywall at publisher
✓ Found open access version via Unpaywall!
   Source: Europe PMC (published version)
   PDF: https://europepmc.org/articles/pmc3858213?pdf=render
   → Downloading...

When no OA found:

⚠️ Paper behind paywall at publisher
✗ No open access version found via Unpaywall
   Options:
   - Request via institutional access
   - Contact authors for preprint
   - Continue with abstract only

4. Prioritize by Version

If multiple locations are available, choose in order:

1. publishedVersion from publisher or PMC
2. acceptedVersion from institutional repository
3. submittedVersion from preprint server (arXiv, bioRxiv)

Python Helper

import requests
import time

def find_open_access(doi, email):
    """
    Find open access version via Unpaywall.
    Returns: (pdf_url, version, source) or (None, None, None)
    """
    url = f"https://api.unpaywall.org/v2/{doi}"
    params = {"email": email}

    try:
        response = requests.get(url, params=params, timeout=10)
        response.raise_for_status()
        data = response.json()

        if not data.get('is_oa'):
            return None, None, None

        best_loc = data.get('best_oa_location')
        if not best_loc:
            return None, None, None

        pdf_url = best_loc.get('url_for_pdf') or best_loc.get('url')
        version = best_loc.get('version', 'unknown')
        source = best_loc.get('host_type', 'unknown')

        return pdf_url, version, source

    except Exception as e:
        print(f"Error checking Unpaywall for {doi}: {e}")
        return None, None, None

# Usage
doi = "10.1038/nature12373"
pdf_url, version, source = find_open_access(doi, "[e-mail]")

if pdf_url:
    print(f"Found {version} at {source}")
    print(f"PDF: {pdf_url}")
    # Download PDF
    response = requests.get(pdf_url)
    with open(f'papers/{doi.replace("/", "_")}.pdf', 'wb') as f:
        f.write(response.content)
else:
    print("No open access version found")

time.sleep(0.1)  # Rate limiting

Full-Text Fetch Workflow

Integrate Unpaywall into a multi-stage retrieval pipeline:

Stage: Fetch Full Text

Try in order:
A. PubMed Central (free full text)
B. DOI resolution → If paywall, try Unpaywall
C. Unpaywall direct lookup
D. Preprints (bioRxiv, arXiv)

# 1. Try PMC
pmc_result=$(curl "https://eutils.ncbi.nlm.nih.gov/...")
if has_pmc_fulltext; then
  fetch_pmc
  exit 0
fi

# 2. Try DOI
doi_result=$(curl -L "https://doi.org/$doi")
if is_paywall; then
  # 3. Try Unpaywall
  unpaywall_result=$(curl "https://api.unpaywall.org/v2/$doi?email=$EMAIL")
  if has_oa; then
    fetch_unpaywall_pdf
    exit 0
  fi
fi

# 4. No full text available
report_no_fulltext

Common OA Sources Found

Repositories: Europe PMC / PubMed Central, institutional repositories (university sites), PubMed Central international mirrors

Preprint servers: bioRxiv (biology), medRxiv (medicine), arXiv (physics, CS, math), ChemRxiv (chemistry)

Publisher sites: Open access journals, hybrid journals (OA articles in subscription journals), delayed open access (embargo expired)

Rate Limiting

Free tier (with email):

• 100,000 requests per day
• No hard rate limit, but be respectful
• Include email in requests (required)

Best practices:

• Add 100ms delay between requests
• Cache responses (don't re-check the same DOI)
• Only check for papers you actually need

Error Handling

DOI not found:

{"error": "true", "message": "DOI not found"}

→ Check DOI format, try alternative identifiers

Network errors:

• Retry with exponential backoff (max 3 attempts)
• Report to user if all fail

Malformed response:

• Check for is_oa field
• Fallback to oa_locations array if best_oa_location missing

Common Mistakes

• Using placeholder email ([e-mail]) → ask user for their real email
• Not including email — required parameter, requests will fail
• Checking every paper — only check when needed
• Ignoring version type — published version better than preprint
• Single source only — check oa_locations array for alternatives
• No rate limiting — add delays even though no hard limit

Success Criteria

Successful when:

• Paywalled paper's OA version found and downloaded
• Version type recorded (published/accepted/submitted)
• User informed about source and version
• Fallback options provided if no OA available

---

Integration with Writing Tools

LaTeX

% In document preamble
\usepackage[backend=biber,style=apa]{biblatex}
\addbibresource{references.bib}

% In text
\textcite{Jumper_2021} showed that...
As demonstrated by previous work \parencite{Jumper_2021}...

% At end of document
\printbibliography

Pandoc Markdown

Previous work [@Jumper_2021] showed that...

## References

pandoc paper.md --citeproc --bibliography=references.bib \
  --csl=apa.csl -o paper.pdf

---

Quick Reference

Phase	Task	Tool/Method
1. Collect	DOI → BibTeX	CrossRef API (Accept: application/x-bibtex)
1. Collect	OpenAlex → BibTeX	api.openalex.org/works/{id}
1. Collect	arXiv → BibTeX	export.arxiv.org/api/query?id_list={id}
2. Verify	Paper exists?	Google Scholar via WebSearch
2. Verify	Programmatic validation	CrossRef API api.crossref.org/works/{doi}
3. Format	BibTeX → APA/MLA/Chicago/Vancouver/IEEE	Python conversion functions
3. Format	Deduplication	Title similarity > 85% merge
4. Full text	OA version available?	api.unpaywall.org/v2/{doi}?email={email}
4. Full text	Download PDF	Parse best_oa_location.url_for_pdf

References

• CrossRef API: https://api.crossref.org
• OpenAlex API: https://api.openalex.org
• Unpaywall API: https://api.unpaywall.org
• APA 7th Edition Manual: https://apastyle.apa.org/products/publication-manual-7th-edition
• BibTeX documentation: http://www.bibtex.org
• CSL styles repository: https://github.com/citation-style-language/styles