extracting-mistral-ocr

name	extracting-mistral-ocr
description	Extracts text, tables, and images from PDFs (including scanned PDFs) using the Mistral OCR API. Use when user asks to OCR a PDF/image, extract text from a PDF, parse a scanned document, convert a PDF to Markdown, or extract structured fields from a document.
compatibility	Requires network access and a MISTRAL_API_KEY environment variable. Expects Python 3.9+ and the mistralai package.
allowed-tools	Read,Write,Bash(python:*)
metadata	{"author":"generated-by-chatgpt","version":"0.1.0","api":"mistral","default-model":"mistral-ocr-latest"}

Mistral OCR PDF extraction

Quick start (default)

Run the bundled script to OCR a local PDF and write Markdown + JSON outputs:

python {baseDir}/scripts/mistral_ocr_extract.py --input path/to/file.pdf --out out/ocr

Output directory layout:

• combined.md (all pages concatenated)
• pages/page-000.md (per-page markdown)
• raw_response.json (full OCR response)
• images/ (decoded embedded images, if requested)
• tables/ (separate tables, if requested)

Workflow

1. Pick input mode

   • Local PDF (most common): upload via Files API, then OCR via file_id.
   • Public URL: OCR directly via document_url.

2. Choose output fidelity (defaults are safe for RAG)

   • Keep table_format=inline unless the user explicitly wants tables split out.
   • Set --include-image-base64 when the user needs figures/diagrams extracted.
   • Use --extract-header/--extract-footer if header/footer noise hurts downstream search.

3. Run OCR

   • Use scripts/mistral_ocr_extract.py to produce a deterministic on-disk artefact set.

4. (Optional) Structured extraction from the whole document

   • If the user wants fields (invoice totals, contract parties, etc.), provide an annotation prompt.
   • The OCR API can return a document-level document_annotation in addition to page markdown.

   Example:

   python {baseDir}/scripts/mistral_ocr_extract.py \
     --input invoice.pdf \
     --out out/invoice \
     --annotation-prompt "Extract supplier_name, invoice_number, invoice_date (ISO-8601), currency, total_amount. Return JSON." \
     --annotation-format json_object
   

Decision rules

• If the PDF is local and not publicly accessible, upload it (the script does this automatically).
• If the PDF URL is private or requires authentication, do not pass it as document_url; upload instead.
• If output quality is critical, prefer table_format=html for downstream parsing over brittle regex.

Common failure modes

• Missing MISTRAL_API_KEY: set it in the environment before running.
• URL OCR fails: the URL likely is not publicly accessible; upload the file.
• Large files: upload supports large files, but very large PDFs may need page selection (--pages) or batch processing.

References

• API + parameters: references/mistral_ocr_api.md
• Output mapping rules (placeholders to extracted images/tables): references/output_mapping.md
• Example annotation prompts for common document types: references/annotation_prompts.md