ocr-and-documents

name	ocr-and-documents
description	Extract text from PDFs and scanned documents. Use web_extract for remote URLs, pymupdf for local text-based PDFs, marker-pdf for OCR/scanned docs. For DOCX use python-docx, for PPTX see the powerpoint skill.
version	2.4.0
author	Hermes Agent
license	MIT
metadata	{"hermes":{"tags":["PDF","Documents","Research","Arxiv","Text-Extraction","OCR"],"related_skills":["powerpoint"]}}

PDF & Document Extraction

For DOCX: use python-docx (parses actual document structure, far better than OCR). For PPTX: see the powerpoint skill (uses python-pptx with full slide/notes support). This skill covers PDFs and scanned documents.

Step 1: Remote URL Available?

If the document has a URL, always try web_extract first:

web_extract(urls=["https://arxiv.org/pdf/2402.03300"])
web_extract(urls=["https://example.com/report.pdf"])

This handles PDF-to-markdown conversion via Firecrawl with no local dependencies.

Only use local extraction when: the file is local, web_extract fails, or you need batch processing.

Step 2: Choose Local Extractor

Feature	pymupdf (~25MB)	marker-pdf (~3-5GB)
Text-based PDF	✅	✅
Scanned PDF (OCR)	❌	✅ (90+ languages)
Tables	✅ (basic)	✅ (high accuracy)
Equations / LaTeX	❌	✅
Code blocks	❌	✅
Forms	❌	✅
Headers/footers removal	❌	✅
Reading order detection	❌	✅
Images extraction	✅ (embedded)	✅ (with context)
Images → text (OCR)	❌	✅
EPUB	✅	✅
Markdown output	✅ (via pymupdf4llm)	✅ (native, higher quality)
Install size	~25MB	~3-5GB (PyTorch + models)
Speed	Instant	~1-14s/page (CPU), ~0.2s/page (GPU)

Decision: Use pymupdf unless you need OCR, equations, forms, or complex layout analysis.

If the user needs marker capabilities but the system lacks ~5GB free disk:

"This document needs OCR/advanced extraction (marker-pdf), which requires ~5GB for PyTorch and models. Your system has [X]GB free. Options: free up space, provide a URL so I can use web_extract, or I can try pymupdf which works for text-based PDFs but not scanned documents or equations."

---

pymupdf (lightweight)

pip install pymupdf pymupdf4llm

IMPORTANT: Always use the helper script below. DO NOT write custom extraction scripts — the helper script covers all common use cases including text, images, metadata, page rendering, and combined extraction.

Via helper script:

python scripts/extract_pymupdf.py document.pdf              # Plain text
python scripts/extract_pymupdf.py document.pdf --markdown    # Markdown
python scripts/extract_pymupdf.py document.pdf --tables      # Tables
python scripts/extract_pymupdf.py document.pdf --images out/ # Extract embedded images
python scripts/extract_pymupdf.py document.pdf --metadata    # Title, author, pages
python scripts/extract_pymupdf.py document.pdf --pages 0-4   # Specific pages
python scripts/extract_pymupdf.py document.pdf --render out/ # Render pages as 2x PNG (for vision)
python scripts/extract_pymupdf.py document.pdf --all out/    # → out/full_text.txt + out/metadata.json + out/renders/
python scripts/extract_pymupdf.py document.pdf --all out/ --no-cache  # Force re-parse (skip cache)

--all is cache-aware: if out/ already contains results from the same PDF (matched by path + size + mtime), it skips parsing and returns the cached paths immediately. Pass --no-cache to force re-parse.

Fallback (if helper script unavailable) — text only:

python3 -c "
import pymupdf
doc = pymupdf.open('document.pdf')
for page in doc:
    print(page.get_text())
"

---

marker-pdf (high-quality OCR)

# Check disk space first
python scripts/extract_marker.py --check

pip install marker-pdf

Via helper script:

python scripts/extract_marker.py document.pdf                # Markdown
python scripts/extract_marker.py document.pdf --json         # JSON with metadata
python scripts/extract_marker.py document.pdf --output_dir out/  # Save images
python scripts/extract_marker.py scanned.pdf                 # Scanned PDF (OCR)
python scripts/extract_marker.py document.pdf --use_llm      # LLM-boosted accuracy

CLI (installed with marker-pdf):

marker_single document.pdf --output_dir ./output
marker /path/to/folder --workers 4    # Batch

---

Arxiv Papers

# Abstract only (fast)
web_extract(urls=["https://arxiv.org/abs/2402.03300"])

# Full paper
web_extract(urls=["https://arxiv.org/pdf/2402.03300"])

# Search
web_search(query="arxiv GRPO reinforcement learning 2026")

Split, Merge & Search

pymupdf handles these natively — use execute_code or inline Python:

# Split: extract pages 1-5 to a new PDF
import pymupdf
doc = pymupdf.open("report.pdf")
new = pymupdf.open()
for i in range(5):
    new.insert_pdf(doc, from_page=i, to_page=i)
new.save("pages_1-5.pdf")

# Merge multiple PDFs
import pymupdf
result = pymupdf.open()
for path in ["a.pdf", "b.pdf", "c.pdf"]:
    result.insert_pdf(pymupdf.open(path))
result.save("merged.pdf")

# Search for text across all pages
import pymupdf
doc = pymupdf.open("report.pdf")
for i, page in enumerate(doc):
    results = page.search_for("revenue")
    if results:
        print(f"Page {i+1}: {len(results)} match(es)")
        print(page.get_text("text"))

No extra dependencies needed — pymupdf covers split, merge, search, and text extraction in one package.

---

Pitfalls

• DO NOT write custom extraction scripts: The helper script scripts/extract_pymupdf.py already handles text, images, metadata, page rendering, and combined extraction. Writing a custom script wastes tokens, duplicates logic (often incorrectly), and produces less maintainable output. If you need something the helper script doesn't cover, extend it rather than replacing it.
• doc.extract_image() produces black/incomplete images: Never call this directly — use the helper script's --images flag which uses the correct Pixmap approach (CMYK→RGB conversion + SMask merging).
• pymupdf4llm version incompatibility: pymupdf4llm requires a matching PyMuPDF version. If you get ImportError: cannot import name 'mupdf' from 'pymupdf', the versions are mismatched (e.g., PyMuPDF 1.23.8 with pymupdf4llm 1.27.x). Fall back to the direct fitz API (page.get_text() for text, Pixmap for images) — no extra dependency needed.
• On NixOS: pymupdf (fitz) fails with ImportError: libstdc++.so.6 unless LD_LIBRARY_PATH includes the gcc-12 lib path. Always set: LD_LIBRARY_PATH=/nix/store/...gcc-12.2.0-lib/lib:$LD_LIBRARY_PATH. See memory for the exact path.
• Always activate venv first: source venv/bin/activate before running Python commands — system Python may not have pymupdf installed.
• Import name: Older PyMuPDF versions use import fitz, newer ones use import pymupdf. If import pymupdf fails, try import fitz.