ocr

name	ocr
description	Use when OCR-ing screenshot images of text (books, papers, essays) to markdown with LaTeX math. Triggers on "ocr this", "transcribe these images", "ocr /path/to/images", image-to-text conversion of scanned pages. NOT for PDF extraction (use pdf-to-markdown instead).

OCR Images to Markdown

Overview

OCR a directory of screenshot images into clean markdown with LaTeX math blocks, using Claude or OpenAI Vision APIs. Designed for academic text with mathematical expressions, logical notation, and structured formatting.

Scripts

Both scripts live in the project's scripts/ directory (the claude-scripts submodule):

• scripts/ocr-images.py — sends images to Vision API, outputs per-page .md files + combined output
• scripts/clean-ocr-latex.py — post-processing: converts stray LaTeX formatting (\textit, \textbf, \begin{center}, etc.) to markdown equivalents

Workflow

1. OCR the images

# OpenAI (default — fewer content filter issues)
python3 scripts/ocr-images.py /path/to/images/ -o /path/to/output/

# Claude provider
python3 scripts/ocr-images.py /path/to/images/ -o /path/to/output/ --provider claude

# Dry run — list images without calling API
python3 scripts/ocr-images.py /path/to/images/ --dry-run

# Custom model or concurrency
python3 scripts/ocr-images.py /path/to/images/ -o /path/to/output/ --model gpt-4o --jobs 4

Output structure when -o points to a directory:

• One .md file per image (named by image stem)
• combined.md with all pages joined by --- separators and <!-- page: filename --> markers

When -o points to a .md file, all output goes to that single file. Without -o, output goes to stdout.

2. Clean up LaTeX formatting (if needed)

GPT-4o sometimes uses LaTeX formatting commands where markdown should be used. Run the cleanup script on the output:

# Single file
python3 scripts/clean-ocr-latex.py /path/to/output/combined.md

# Entire directory tree
python3 scripts/clean-ocr-latex.py /path/to/output/

# Dry run
python3 scripts/clean-ocr-latex.py /path/to/output/ --dry-run

Conversions: \textit{} to *...*, \textbf{} to **...**, \textsc{} to plain text, \emph{} to *...*, \texttt{} to backticks, \begin{quote} to blockquote, \begin{enumerate} to numbered list, \begin{itemize} to bullet list, \section{} to ##. Math environments are preserved.

Environment

API keys are read from environment variables or a .env file in the project root:

OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...

Requirements

pip install anthropic openai

Provider Selection

Provider	Default model	Notes
openai	gpt-4o	Fewer content filter issues
claude	claude-sonnet-4-6	Better accuracy on complex math notation

Common Issues

Issue	Fix
Content filter rejection (Claude)	Switch to --provider openai
LaTeX formatting in output	Run clean-ocr-latex.py on the output
Rate limiting	Reduce --jobs (default 3). Script retries automatically
Code fences wrapping output	Already handled — script strips them automatically