check-paper-library

name	check-paper-library
description	Comprehensive paper library maintenance — cross-references papers.md against summaries, techniques, and PDFs. Detects orphans, missing registrations, and pulls missing papers.

Check Paper Library Integrity

Mental Model

The paper library has four sources of truth that must stay synchronized:

1. references/papers.md — The master index (authoritative list of all papers)
2. docs/upstream/papers/{slug}.md — Paper summaries (one per paper)
3. resources/techniques/{slug}.py — Technique implementations (when applicable)
4. docs/upstream/pdfs/{arxiv_id}-{slug}.pdf — Downloaded PDFs

When any one of these is out of sync, future agents waste time rediscovering what's already known. This skill ensures the library is a reliable source of truth.

Key paths (verified against repo):

• Summaries: docs/upstream/papers/*.md
• PDFs: docs/upstream/pdfs/*.pdf (NOT docs/upstream/papers/)
• Techniques: resources/techniques/*.py
• Index: references/papers.md

---

What This Skill Checks

1. Cross-Reference Validation

Every paper in papers.md should have corresponding files where expected:

Paper Type	papers.md	Summary	Technique	PDF
Architectural/Algorithmic	row	{slug}.md	{slug}.py	{arxiv_id}-{slug}.pdf
Evaluation Benchmark	row	{slug}.md	—	{arxiv_id}-{slug}.pdf
Theoretical/Alignment	row	{slug}.md	maybe	{arxiv_id}-{slug}.pdf
Pretrained Backbone	row	{slug}.md	—	{arxiv_id}-{slug}.pdf

2. Orphan Detection

Orphaned summaries: docs/upstream/papers/{slug}.md exists but the slug is not registered in papers.md.

Orphaned techniques: resources/techniques/{slug}.py exists but the slug is not registered in papers.md. Note: resources/techniques/ also contains __init__.py, perceiver.py, and attn_res.py — these are infrastructure files, not paper-linked techniques. Exclude them from orphan checks.

Orphaned PDFs: docs/upstream/pdfs/{arxiv_id}-{slug}.pdf exists but the slug is not registered in papers.md.

3. Missing Registration Detection

Registered but missing summary: papers.md lists a slug but docs/upstream/papers/{slug}.md does not exist.

Registered but missing technique: papers.md lists a technique module path (e.g., resources/techniques/slug.py) but the file does not exist. Some papers are observational or evaluation-only — these legitimately have no technique module. Check the "Notes" column in papers.md before flagging.

Registered but missing PDF: papers.md lists an arXiv ID but no PDF exists at docs/upstream/pdfs/{arxiv_id}-{slug}.pdf. Papers without arXiv IDs (e.g., wales1997basinhopping which uses a DOI) legitimately have no PDF — note this in the summary.

4. Summary Header Validation

Each summary file (docs/upstream/papers/{slug}.md) must contain an **arXiv**: line with a link, e.g.:

**arXiv**: [2103.03206](https://arxiv.org/abs/2103.03206)

This header is critical because tools/pull_pdfs.py uses it as a fallback to resolve arXiv IDs for papers not yet in papers.md. If missing, the paper cannot have its PDF pulled.

5. PDF Pull Status

Every registered paper with an arXiv ID should have its PDF downloaded.

Check:

• PDF exists in docs/upstream/pdfs/{arxiv_id}-{slug}.pdf
• If missing, attempt to pull using the actual CLI

Pull commands (verified against tools/pull_pdfs.py):

# Pull all missing PDFs for every paper in the library
python tools/pull_pdfs.py --all

# Pull specific papers by slug
python tools/pull_pdfs.py jaegle2022perceiverio ge2023icae

# List what would be downloaded (dry run)
python tools/pull_pdfs.py --list

# Force re-download even if file exists
python tools/pull_pdfs.py --force --all

Auto-fix: Pull missing PDFs using python tools/pull_pdfs.py --all.

6. Superseded Paper Detection

A paper may be superseded by a newer version. This is informational only — do not auto-remove.

How to detect:

• Check paper summaries for "extends", "improves", "replaces" language
• Look for papers that cite and improve upon existing techniques
• Check if the same author has a newer version

Known supersessions (report but don't auto-fix):

• dohare2023plasticity may be superseded by dohare2024plasticity (same arXiv ID: 2306.13812 — the 2024 version is the Nature publication of the same work)

7. Duplicate Detection

Duplicate slugs: Same slug used for different papers (should never happen).

Duplicate arXiv IDs: Same arXiv ID listed for different slugs. The dohare2023plasticity / dohare2024plasticity case is the known legitimate duplicate (preprint + publication of same work).

---

Anti-Patterns to Fix

Pattern	Where to Look	Fix
Orphaned summary	docs/upstream/papers/*.md not in papers.md	Add row to papers.md
Orphaned technique	resources/techniques/*.py not in papers.md	Add row to papers.md (or verify it's infrastructure)
Orphaned PDF	docs/upstream/pdfs/*.pdf not in papers.md	Register in papers.md
Missing summary	papers.md lists slug, no docs/upstream/papers/{slug}.md	Write the summary
Missing technique	papers.md lists technique path, file missing	Implement or verify paper is observational
Missing PDF	papers.md lists arXiv ID, no docs/upstream/pdfs/{arxiv_id}-{slug}.pdf	Run python tools/pull_pdfs.py {slug}
Missing arXiv header	Summary file lacks **arXiv**: line	Add the header (needed for pull_pdfs.py fallback)

---

Actionable Steps

Step 1: Parse references/papers.md

Extract from each table row:

• Slug: content in backticks in the first column (| \slug` |`)
• arXiv ID: number in square brackets linking to arxiv.org ([2307.06945](...))
• Technique module: path in backticks in the last column (| \resources/techniques/slug.py` |`)
• Notes column: may contain "Observational — no novel algorithm" or similar (means no technique expected)

The table format is:

| `slug` | Title | [arxiv_id](url) | `resources/techniques/slug.py` or description or — |

Step 2: Scan docs/upstream/papers/

List all *.md files. The stem of each file is the slug. Cross-reference against papers.md slugs.

Step 3: Scan resources/techniques/

List all *.py files excluding __init__.py. Also exclude perceiver.py and attn_res.py (infrastructure, not paper-linked). Extract slugs from filenames. Cross-reference against papers.md technique column.

Step 4: Scan docs/upstream/pdfs/

List all *.pdf files. Parse each filename as {arxiv_id}-{slug}.pdf. Cross-reference slugs against papers.md.

Step 5: Cross-Reference

• papers.md slugs vs docs/upstream/papers/ directory
• papers.md technique paths vs resources/techniques/ directory
• resources/techniques/ slugs vs papers.md registrations
• docs/upstream/pdfs/ slugs vs papers.md registrations
• Summary files: check for **arXiv**: header presence

Step 6: Fix Issues

• Orphaned files: Add missing rows to papers.md
• Missing PDFs: Run python tools/pull_pdfs.py --all (or specific slugs)
• Missing summaries: Write them (follow existing format in docs/upstream/papers/)
• Missing arXiv headers: Add **arXiv**: [id](url) line to summary
• Issues requiring manual intervention: Report them

---

Verification

After fixing issues, verify counts:

# Count registrations (subtract 2 for header/separator lines)
grep -c "^| \`" references/papers.md

# Count summary files
ls docs/upstream/papers/*.md | wc -l

# Count technique files (exclude infrastructure)
ls resources/techniques/*.py | grep -v __init__ | grep -v perceiver.py | grep -v attn_res.py | wc -l

# Count PDFs
ls docs/upstream/pdfs/*.pdf | wc -l

Counts should be approximately:

• Summaries ≈ registrations (summaries may be slightly higher due to unregistered papers)
• Techniques ≤ registrations (not all papers have implementations)
• PDFs ≈ registrations with arXiv IDs (some papers use DOIs or lack PDFs)

---

Report Format

STATUS: [no_work | fixed]

SUMMARY:
  papers.md registrations:     N
  summary files:               N
  technique modules:           N
  PDFs:                        N

CHANGES:
- references/papers.md: Added registration for orphan2024
- docs/upstream/pdfs/2201.12345-orphan2024.pdf: Pulled PDF

ORPHANS:
- docs/upstream/papers/orphan2024.md (not in papers.md)

MISSING:
- papers.md lists foo2023bar but no docs/upstream/papers/foo2023bar.md

DETAILS:
{Detailed explanation of each change}