add-paper

name

add-paper

description

Add a single new paper or repository to docs/initialDocs/revisedBibliography.md and, when possible, materialise its `literature/papers/<slug>/` folder (PDF + shallow-cloned repo + notes.md + citation.bib) following docs/initialDocs/bibliographySpec.md. The folder is created only when at least one artefact (PDF or repo) was downloaded AND year + first author are known; otherwise the row is added with an empty Literature dir cell. Use when the user pastes a URL, BibTeX, or arXiv ID and asks to "add this paper", "save this reference", "track this in the bibliography", etc.

This skill captures the end-to-end flow for adding ONE new entry to the project's bibliography. For bulk processing of many rows at once, delegate to the paper-organizer agent instead.

Authoritative paths

• Layout / slug / notes.md template: docs/initialDocs/bibliographySpec.md (canonical — read it whenever you need the verbatim rules).
• Main inventory: docs/initialDocs/revisedBibliography.md.
• Derived (auto-generated, refreshed at the end of this flow): docs/initialDocs/derivedBibliography.md.
• Open proposals (untouched here): docs/initialDocs/paperProposals.md.

Layout & slug summary

Each new entry that successfully downloads ends up as a self-contained folder under literature/papers/<slug>/ with at minimum paper.pdf, citation.bib, and notes.md (see bibliographySpec.md for the full tree, including optional supplementary.pdf and the code/original/ / code/experiments/ subtree).

Slug format: YYYY_firstauthor_keyword/.

• Lowercase, no accents, no spaces. Only _ and - as separators.
• _ separates year / first-author last-name / keyword.
• - separates words inside a multi-word keyword.
• Keyword priority: official acronym → model name → main technical term.
• Do NOT use the full title as the slug; do NOT rename a folder once it exists.

Good: 2020_carion_detr/, 2023_liu_grounding-dino/. Bad: 2020_End-to-End_Object_Detection_with_Transformers/, paper_final/.

The BibTeX key inside citation.bib follows <lastname><year><keyword> (e.g. carion2020detr, kirillov2023sam).

bib_init_paper.py writes notes.md for you on a successful commit using the template defined in bibliographySpec.md. Unknown fields stay as TODO (private fact we will fill later) or Not found (public fact that does not exist). Never invent values.

What to gather from the user

Before editing anything, make sure you have:

1. Title as it should appear in the table (full, English).
2. Main link — preferably arXiv abs/, the official OA PDF, or the canonical journal page.
3. GitHub URL if there is official code (or "—" if none).
4. Section letter of docs/initialDocs/revisedBibliography.md where the row belongs (A–S). Use the section's existing rows to decide.
5. Status icon to start with: ☐ (open access, not yet fetched), 🔒 (paywall), 🟡 (URL or title unverified), 🛠️ (resource/repo).
6. Organization / venue string in the same style as neighbouring rows (e.g. MDPI Sensors, arXiv, IEEE Xplore, Nature Scientific Reports).

If anything is missing, ask the user before writing.

Steps

1. Ensure the column header exists (idempotent, safe to run any time): python3 scripts/bib_update.py ensure-dir-column.

   Caveat: bib_update.py currently hardcodes docs/bibliography.md as its target. If that path no longer exists, point it at the new location (or alias / patch the script) before relying on it.

2. Insert the row at the bottom of the correct section of docs/initialDocs/revisedBibliography.md using the existing column order, leaving the Literature dir cell empty: | <status> | <title> | <organization> | <main_link> | <github or —> | |

3. Try to materialise the paper directory by piping the new row through bib_init_paper.py:

   python3 scripts/bib_parse.py docs/initialDocs/revisedBibliography.md \
     | jq '[.[] | select(.title == "<exact title>")][0]' \
     | python3 scripts/bib_init_paper.py
   

   The script:

   • Fetches publisher metadata (year, first author).
   • Stages downloads in /tmp/bib_staging_*.
   • Decides to commit to literature/papers/<slug>/ only when (PDF or repo for 🛠️) succeeded AND year + first author are real.
   • On commit, writes notes.md and citation.bib from the template.
   • Prints JSON with committed, slug, pdf, clone, metadata, reason.

4. Sync docs/initialDocs/revisedBibliography.md:

   • If the JSON has committed: true:

     python3 scripts/bib_update.py set-cell 'title:<exact title>' \
       literature_dir '<slug>/'
     python3 scripts/bib_update.py set-status '<slug>' '✅'
     

   • If committed: false: leave the row exactly as inserted (empty Literature dir cell, original status icon). The result.reason field tells you why — usually paywall, ResearchGate block, MDPI Akamai 403, or missing publisher metadata.

5. Refresh the derived bibliography (only when committed: true). A new PDF means new citations; the derived list must reflect them and re-deduplicate against the now-tracked entry:

   python3 scripts/bib_extract_refs.py --pass1   # extracts IDs from the new paper
   python3 scripts/bib_extract_refs.py --pass2   # resolves DOIs/arXiv via Crossref + arXiv
   python3 scripts/bib_derived.py                # rewrites docs/initialDocs/derivedBibliography.md
   

   Skip this step if committed: false (no new PDF, nothing to harvest).

6. Final verification: tail -n 5 docs/initialDocs/revisedBibliography.md — confirm the new row's status and Literature dir cell match what you expect. Glance at docs/initialDocs/derivedBibliography.md if step 5 ran: the new paper's citations should now be reflected in the topic buckets, and any reference that we just promoted should have been dropped.

Things to NOT do

• Don't create literature/papers/<slug>/ directly. Always go through bib_init_paper.py so the staging-then-commit invariant holds.
• Don't fill the Literature dir cell unless bib_init_paper.py returned committed: true.
• Don't rename existing folders under literature/papers/. Use bib_cleanup.py for re-slugging.
• Don't invent author names, years, or BibTeX keys. The metadata fetcher fills what it can; everything else stays TODO.
• Don't try to download whole datasets even when the row points to one (MVTec, COCO, ImageNet, KolektorSDD2, Real-IAD). The bibliography row is enough; no folder is needed.
• Don't skip the derived-bibliography refresh after a successful commit — it is the only mechanism that keeps docs/initialDocs/derivedBibliography.md in sync with the corpus.
• Don't edit docs/initialDocs/derivedBibliography.md by hand — it is regenerated from the cache + per-paper extraction artefacts.