Write publication-ready ML/AI/Systems papers for NeurIPS, ICML, ICLR, ACL, AAAI, COLM, OSDI, NSDI, ASPLOS, SOSP. Use when drafting papers from research repos, structuring arguments, verifying citations, or preparing camera-ready submissions. Includes LaTeX templates, reviewer guidelines, and citation verification workflows.
Instrucciones de origen · Vista previa de solo lectura
name
20-ml-paper-writing
description
Write publication-ready ML/AI/Systems papers for NeurIPS, ICML, ICLR, ACL, AAAI, COLM, OSDI, NSDI, ASPLOS, SOSP. Use when drafting papers from research repos, structuring arguments, verifying citations, or preparing camera-ready submissions. Includes LaTeX templates, reviewer guidelines, and citation verification workflows.
Expert-level guidance for writing publication-ready papers targeting NeurIPS, ICML, ICLR, ACL, AAAI, COLM (ML/AI venues) and (Systems venues). This skill combines writing philosophy from top researchers (Nanda, Farquhar, Karpathy, Lipton, Steinhardt) with practical tools: LaTeX templates, citation verification APIs, and conference checklists.
OSDI, NSDI, ASPLOS, SOSP
Core Philosophy: Collaborative Writing
Paper writing is collaborative, but Claude should be proactive in delivering drafts.
The typical workflow starts with a research repository containing code, results, and experimental artifacts. Claude's role is to:
Understand the project by exploring the repo, results, and existing documentation
Deliver a complete first draft when confident about the contribution
Search literature using web search and APIs to find relevant citations
Refine through feedback cycles when the scientist provides input
Ask for clarification only when genuinely uncertain about key decisions
Key Principle: Be proactive. If the repo and results are clear, deliver a full draft. Don't block waiting for feedback on every section—scientists are busy. Produce something concrete they can react to, then iterate based on their response.
⚠️ CRITICAL: Never Hallucinate Citations
This is the most important rule in academic writing with AI assistance.
The Problem
AI-generated citations have a ~40% error rate. Hallucinated references—papers that don't exist, wrong authors, incorrect years, fabricated DOIs—are a serious form of academic misconduct that can result in desk rejection or retraction.
The Rule
NEVER generate BibTeX entries from memory. ALWAYS fetch programmatically.
Action
✅ Correct
❌ Wrong
Adding a citation
Search API → verify → fetch BibTeX
Write BibTeX from memory
Uncertain about a paper
Mark as [CITATION NEEDED]
Guess the reference
Can't find exact paper
Note: "placeholder - verify"
Invent similar-sounding paper
When You Can't Verify a Citation
If you cannot programmatically verify a citation, you MUST:
% EXPLICIT PLACEHOLDER - requires human verification
\cite{PLACEHOLDER_author2024_verify_this} % TODO: Verify this citation exists
Always tell the scientist: "I've marked [X] citations as placeholders that need verification. I could not confirm these papers exist."
Recommended: Install Exa MCP for Paper Search
For the best paper search experience, install Exa MCP which provides real-time academic search:
Claude Code:
claude mcp add exa -- npx -y mcp-remote "https://mcp.exa.ai/mcp"
These are high-signal starting points for Related Work—the scientist has already deemed them relevant.
Step 3: Clarify the Contribution
Before writing, explicitly confirm with the scientist:
"Based on my understanding of the repo, the main contribution appears to be [X].
The key results show [Y]. Is this the framing you want for the paper,
or should we emphasize different aspects?"
Never assume the narrative—always verify with the human.
Which specific results to show (make a choice, flag it)
Citation completeness (draft with what you find, note gaps)
The Narrative Principle
The single most critical insight: Your paper is not a collection of experiments—it's a story with one clear contribution supported by evidence.
Every successful ML paper centers on what Neel Nanda calls "the narrative": a short, rigorous, evidence-based technical story with a takeaway readers care about.
Three Pillars (must be crystal clear by end of introduction):
If you cannot state your contribution in one sentence, you don't yet have a paper.
Paper Structure Workflow
Workflow 1: Writing a Complete Paper (Iterative)
Copy this checklist and track progress. Each step involves drafting → feedback → revision:
Paper Writing Progress:
- [ ] Step 1: Define the one-sentence contribution (with scientist)
- [ ] Step 2: Draft Figure 1 → get feedback → revise
- [ ] Step 3: Draft abstract → get feedback → revise
- [ ] Step 4: Draft introduction → get feedback → revise
- [ ] Step 5: Draft methods → get feedback → revise
- [ ] Step 6: Draft experiments → get feedback → revise
- [ ] Step 7: Draft related work → get feedback → revise
- [ ] Step 8: Draft limitations → get feedback → revise
- [ ] Step 9: Complete paper checklist (required)
- [ ] Step 10: Final review cycle and submission
Step 1: Define the One-Sentence Contribution
This step requires explicit confirmation from the scientist.
Before writing anything, articulate and verify:
What is the single thing your paper contributes?
What was not obvious or present before your work?
"I propose framing the contribution as: '[one sentence]'. Does this capture
what you see as the main takeaway? Should we adjust the emphasis?"
Step 2: Draft Figure 1
Figure 1 deserves special attention—many readers skip directly to it.
Convey core idea, approach, or most compelling result
Use vector graphics (PDF/EPS for plots)
Write captions that stand alone without main text
Ensure readability in black-and-white (8% of men have color vision deficiency)
Step 3: Write Abstract (5-Sentence Formula)
From Sebastian Farquhar (DeepMind):
1. What you achieved: "We introduce...", "We prove...", "We demonstrate..."
2. Why this is hard and important
3. How you do it (with specialist keywords for discoverability)
4. What evidence you have
5. Your most remarkable number/result
Delete generic openings like "Large language models have achieved remarkable success..."
Step 4: Write Introduction (1-1.5 pages max)
Must include:
2-4 bullet contribution list (max 1-2 lines each in two-column format)
Clear problem statement
Brief approach overview
Methods should start by page 2-3 maximum
Step 5: Methods Section
Enable reimplementation:
Conceptual outline or pseudocode
All hyperparameters listed
Architectural details sufficient for reproduction
Present final design decisions; ablations go in experiments
Step 6: Experiments Section
For each experiment, explicitly state:
What claim it supports
How it connects to main contribution
Experimental setting (details in appendix)
What to observe: "the blue line shows X, which demonstrates Y"
Requirements:
Error bars with methodology (standard deviation vs standard error)
Hyperparameter search ranges
Compute infrastructure (GPU type, total hours)
Seed-setting methods
Step 7: Related Work
Organize methodologically, not paper-by-paper:
Good: "One line of work uses Floogledoodle's assumption [refs] whereas we use Doobersnoddle's assumption because..."
Bad: "Snap et al. introduced X while Crackle et al. introduced Y."
NeurIPS, ICML, and ICLR all require paper checklists.
Writing Philosophy for Top ML Conferences
This section distills the most important writing principles from leading ML researchers. These aren't optional style suggestions—they're what separates accepted papers from rejected ones.
"A paper is a short, rigorous, evidence-based technical story with a takeaway readers care about." — Neel Nanda
The Sources Behind This Guidance
This skill synthesizes writing philosophy from researchers who have published extensively at top venues:
Source
Key Contribution
Link
Neel Nanda (Google DeepMind)
The Narrative Principle, What/Why/So What framework
Why? Most reviewers form judgments before reaching your methods. Readers encounter your paper as: title → abstract → introduction → figures → maybe the rest.
These principles are based on how readers actually process prose. Violating them forces readers to spend cognitive effort on structure rather than content.
Principle
Rule
Example
Subject-verb proximity
Keep subject and verb close
❌ "The model, which was trained on..., achieves" → ✅ "The model achieves... after training on..."
Stress position
Place emphasis at sentence ends
❌ "Accuracy improves by 15% when using attention" → ✅ "When using attention, accuracy improves by 15%"
Topic position
Put context first, new info after
✅ "Given these constraints, we propose..."
Old before new
Familiar info → unfamiliar info
Link backward, then introduce new
One unit, one function
Each paragraph makes one point
Split multi-point paragraphs
Action in verb
Use verbs, not nominalizations
❌ "We performed an analysis" → ✅ "We analyzed"
Context before new
Set stage before presenting
Explain before showing equation
Micro-Level Tips (Ethan Perez)
These small changes accumulate into significantly clearer prose:
Minimize pronouns: ❌ "This shows..." → ✅ "This result shows..."
Verbs early: Position verbs near sentence start
Unfold apostrophes: ❌ "X's Y" → ✅ "The Y of X" (when awkward)
Consistent terminology: Different terms for same concept creates confusion. Pick one and stick with it.
State assumptions formally: Before theorems, list all assumptions explicitly
Intuition + rigor: Provide intuitive explanations alongside formal proofs
What Reviewers Actually Read
Understanding reviewer behavior helps prioritize your effort:
Paper Section
% Reviewers Who Read
Implication
Abstract
100%
Must be perfect
Introduction
90%+ (skimmed)
Front-load contribution
Figures
Examined before methods
Figure 1 is critical
Methods
Only if interested
Don't bury the lede
Appendix
Rarely
Put only supplementary details
Bottom line: If your abstract and intro don't hook reviewers, they may never read your brilliant methods section.
Conference Requirements Quick Reference
ML/AI Conferences
Conference
Page Limit
Extra for Camera-Ready
Key Requirement
NeurIPS 2025
9 pages
+0
Mandatory checklist, lay summary for accepted
ICML 2026
8 pages
+1
Broader Impact Statement required
ICLR 2026
9 pages
+1
LLM disclosure required, reciprocal reviewing
ACL 2025
8 pages (long)
varies
Limitations section mandatory
AAAI 2026
7 pages
+1
Strict style file adherence
COLM 2025
9 pages
+1
Focus on language models
Systems Conferences
Conference
Page Limit
Extra for Camera-Ready
Key Requirement
Template
OSDI 2026
12 pages
+2 (14 pages)
Research + Operational Systems tracks
USENIX
NSDI 2027
12 pages
varies
Prescreening via Introduction; 3 tracks
USENIX
ASPLOS 2027
12 pages (ACM)
varies
Rapid review on first 2 pages; dual cycles
ACM SIGPLAN
SOSP 2026
12 pages
varies
Optional artifact evaluation; author response
ACM SIGPLAN
Universal Requirements:
Double-blind review (anonymize submissions)
References don't count toward page limit
Appendices unlimited but reviewers not required to read
LaTeX required for all venues
Systems venues: USENIX uses custom .sty; ACM uses acmart.cls
Using LaTeX Templates Properly
Workflow 4: Starting a New Paper from Template
Always copy the entire template directory first, then write within it.
Template Setup Checklist:
- [ ] Step 1: Copy entire template directory to new project
- [ ] Step 2: Verify template compiles as-is (before any changes)
- [ ] Step 3: Read the template's example content to understand structure
- [ ] Step 4: Replace example content section by section
- [ ] Step 5: Keep template comments/examples as reference until done
- [ ] Step 6: Clean up template artifacts only at the end
Step 1: Copy the Full Template
# Create your paper directory with the complete templatecp -r templates/neurips2025/ ~/papers/my-new-paper/
cd ~/papers/my-new-paper/
# Verify structure is completels -la
# Should see: main.tex, neurips.sty, Makefile, etc.
⚠️ IMPORTANT: Copy the ENTIRE directory, not just main.tex. Templates include:
Style files (.sty) - required for compilation
Bibliography styles (.bst) - required for references
Example content - useful as reference
Makefiles - for easy compilation
Step 2: Verify Template Compiles First
Before making ANY changes, compile the template as-is:
# Using latexmk (recommended)
latexmk -pdf main.tex
# Or manual compilation
pdflatex main.tex
bibtex main
pdflatex main.tex
pdflatex main.tex
If the unmodified template doesn't compile, fix that first. Common issues:
Missing TeX packages → install via tlmgr install <package>
Wrong TeX distribution → use TeX Live (recommended)
Step 3: Keep Template Content as Reference
Don't immediately delete all example content. Instead:
% KEEP template examples commented out as you write
% This shows you the expected format
% Template example (keep for reference):
% \begin{figure}[t]
% \centering
% \includegraphics[width=0.8\linewidth]{example-image}
% \caption{Template shows caption style}
% \end{figure}
% Your actual figure:
\begin{figure}[t]
\centering
\includegraphics[width=0.8\linewidth]{your-figure.pdf}
\caption{Your caption following the same style.}
\end{figure}
Step 4: Replace Content Section by Section
Work through the paper systematically:
Replacement Order:
1. Title and authors (anonymize for submission)
2. Abstract
3. Introduction
4. Methods
5. Experiments
6. Related Work
7. Conclusion
8. References (your .bib file)
9. Appendix
For each section:
Read the template's example content
Note any special formatting or macros used
Replace with your content following the same patterns
Compile frequently to catch errors early
Step 5: Use Template Macros
Templates often define useful macros. Check the preamble for:
% Common template macros to use:
\newcommand{\method}{YourMethodName} % Consistent method naming
\newcommand{\eg}{e.g.,\xspace} % Proper abbreviations
\newcommand{\ie}{i.e.,\xspace}
\newcommand{\etal}{\textit{et al.}\xspace}
Step 6: Clean Up Only at the End
Only remove template artifacts when paper is nearly complete:
% BEFORE SUBMISSION - remove these:
% - Commented-out template examples
% - Unused packages
% - Template's example figures/tables
% - Lorem ipsum or placeholder text
% KEEP these:
% - All style files (.sty)
% - Bibliography style (.bst)
% - Required packages from template
% - Any custom macros you're using
Template Pitfalls to Avoid
Pitfall
Problem
Solution
Copying only main.tex
Missing .sty, won't compile
Copy entire directory
Modifying .sty files
Breaks conference formatting
Never edit style files
Adding random packages
Conflicts, breaks template
Only add if necessary
Deleting template content too early
Lose formatting reference
Keep as comments until done
Not compiling frequently
Errors accumulate
Compile after each section
Quick Template Reference
ML/AI Conferences
Conference
Main File
Key Style File
Notes
NeurIPS 2025
main.tex
neurips.sty
Has Makefile
ICML 2026
example_paper.tex
icml2026.sty
Includes algorithm packages
ICLR 2026
iclr2026_conference.tex
iclr2026_conference.sty
Has math_commands.tex
ACL
acl_latex.tex
acl.sty
Strict formatting
AAAI 2026
aaai2026-unified-template.tex
aaai2026.sty
Very strict compliance
COLM 2025
colm2025_conference.tex
colm2025_conference.sty
Similar to ICLR
Systems Conferences
Conference
Main File
Key Style File
Notes
OSDI 2026
main.tex
usenix-2020-09.sty
USENIX format, 12pp, two-column, 10pt
NSDI 2027
main.tex
usenix-2020-09.sty
Same USENIX format as OSDI
ASPLOS 2027
main.tex
acmart.cls (sigplan)
ACM SIGPLAN format, 12pp
SOSP 2026
main.tex
acmart.cls (sigplan)
ACM SIGPLAN, same as ASPLOS
Conference Resubmission & Format Conversion
When a paper is rejected or withdrawn from one venue and resubmitted to another, format conversion is required. This is a common workflow in ML research.
Paper checklist (appendix), lay summary if accepted
Systems Venues
Target Venue
Key Required Additions
OSDI 2026
Choose Research or Operational Systems track; anonymize system name
NSDI 2027
Strong Introduction (prescreening); choose track
ASPLOS 2027
Self-contained first 2 pages (rapid review); resubmission note
SOSP 2026
ACM SIGPLAN format; optional Artifact Evaluation
Step 5: Update References
% Remove self-citations that reveal identity (for blind review)
% Update any "under review" citations to published versions
% Add new relevant work published since last submission
Step 6: Addressing Previous Reviews
When resubmitting after rejection:
Do address reviewer concerns in the new version
Do add experiments/clarifications reviewers requested
Don't include a "changes from previous submission" section (blind review)
Don't reference the previous submission or reviews
Common Conversion Pitfalls:
❌ Copying \usepackage commands (causes conflicts)
❌ Keeping old conference header/footer commands
❌ Forgetting to update \bibliography{} path
❌ Missing conference-specific required sections
❌ Exceeding page limit after format change
Citation Workflow (Hallucination Prevention)
⚠️ CRITICAL: AI-generated citations have ~40% error rate. Never write BibTeX from memory.
The Golden Rule
IF you cannot programmatically fetch a citation:
→ Mark it as [CITATION NEEDED] or [PLACEHOLDER - VERIFY]
→ Tell the scientist explicitly
→ NEVER invent a plausible-sounding reference
Workflow 2: Adding Citations
Citation Verification (MANDATORY for every citation):
- [ ] Step 1: Search using Exa MCP or Semantic Scholar API
- [ ] Step 2: Verify paper exists in 2+ sources (Semantic Scholar + arXiv/CrossRef)
- [ ] Step 3: Retrieve BibTeX via DOI (programmatically, not from memory)
- [ ] Step 4: Verify the claim you're citing actually appears in the paper
- [ ] Step 5: Add verified BibTeX to bibliography
- [ ] Step 6: If ANY step fails → mark as placeholder, inform scientist
Step 0: Use Exa MCP for Initial Search (Recommended)
If Exa MCP is installed, use it to find relevant papers:
Search: "RLHF language model alignment 2023"
Search: "sparse autoencoders interpretability"
Search: "attention mechanism transformers Vaswani"
Then verify each result with Semantic Scholar and fetch BibTeX via DOI.
Step 1: Search Semantic Scholar
from semanticscholar import SemanticScholar
sch = SemanticScholar()
results = sch.search_paper("attention mechanism transformers", limit=5)
for paper in results:
print(f"{paper.title} - {paper.paperId}")
print(f" DOI: {paper.externalIds.get('DOI', 'N/A')}")
Step 2: Verify Existence
Confirm paper appears in at least two sources (Semantic Scholar + CrossRef/arXiv).
Step 3: Retrieve BibTeX via DOI
import requests
defdoi_to_bibtex(doi: str) -> str:
"""Get verified BibTeX from DOI via CrossRef."""
response = requests.get(
f"https://doi.org/{doi}",
headers={"Accept": "application/x-bibtex"}
)
response.raise_for_status()
return response.text
# Example
bibtex = doi_to_bibtex("10.48550/arXiv.1706.03762")
print(bibtex)
Step 4: Verify Claims
Before citing for a specific claim, access the paper and confirm the attributed claim actually appears.
Step 5: Handle Failures Explicitly
If you cannot verify a citation at ANY step:
% Option 1: Explicit placeholder
\cite{PLACEHOLDER_smith2023_verify} % TODO: Could not verify - scientist must confirm
% Option 2: Note in text
... as shown in prior work [CITATION NEEDED - could not verify Smith et al. 2023].
Always inform the scientist:
"I could not verify the following citations and have marked them as placeholders:
Smith et al. 2023 on reward hacking - could not find in Semantic Scholar
Jones 2022 on scaling laws - found similar paper but different authors
Please verify these before submission."
Summary: Citation Rules
Situation
Action
Found paper, got DOI, fetched BibTeX
✅ Use the citation
Found paper, no DOI
✅ Use arXiv BibTeX or manual entry from paper
Paper exists but can't fetch BibTeX
⚠️ Mark placeholder, inform scientist
Uncertain if paper exists
❌ Mark [CITATION NEEDED], inform scientist
"I think there's a paper about X"
❌ NEVER cite - search first or mark placeholder
🚨 NEVER generate BibTeX from memory—always fetch programmatically. 🚨
Common Issues and Solutions
Issue: Abstract too generic
Delete first sentence if it could be prepended to any ML paper. Start with your specific contribution.
Issue: Introduction exceeds 1.5 pages
Split background into Related Work. Front-load contribution bullets. Methods should start by page 2-3.
Issue: Experiments lack explicit claims
Add sentence before each experiment: "This experiment tests whether [specific claim]..."
Issue: Reviewers find paper hard to follow
Add explicit signposting: "In this section, we show X"
Use consistent terminology throughout
Include figure captions that stand alone
Issue: Missing statistical significance
Always include:
Error bars (specify: std dev or std error)
Number of runs
Statistical tests if comparing methods
Reviewer Evaluation Criteria
Reviewers assess papers on four dimensions:
Criterion
What Reviewers Look For
Quality
Technical soundness, well-supported claims
Clarity
Clear writing, reproducible by experts
Significance
Community impact, advances understanding
Originality
New insights (doesn't require new method)
Scoring (NeurIPS 6-point scale):
6: Strong Accept - Groundbreaking, flawless
5: Accept - Technically solid, high impact
4: Borderline Accept - Solid, limited evaluation
3: Borderline Reject - Solid but weaknesses outweigh
2: Reject - Technical flaws
1: Strong Reject - Known results or ethics issues
Tables and Figures
Tables
Use booktabs LaTeX package for professional tables:
