| name | table-extraction |
| description | Extract tables from PDFs, HTML pages, and images into structured formats. Triggers on: extracting tabular data from PDF reports or scanned documents, converting document tables to CSV/JSON/DataFrames, parsing tables from HTML without API access, handling merged cells or multi-row headers, OCR-based table extraction from images. Does NOT trigger on: creating new tables in documents, generating Excel files from scratch, extracting plain text from PDFs, or editing existing PDFs. |
| license | Apache-2.0 |
| compatibility | {"clients":["openai-codex","gemini-cli","opencode"]} |
| metadata | {"owner":"codex","domain":"table-extraction","maturity":"draft","risk":"low","tags":["table-extraction","camelot","tabula","row-detection","header-inference"]} |
Purpose
Extract tables from PDFs, HTML pages, and images into structured data formats. Detect row/column boundaries, infer headers, handle merged cells and multi-line rows, and output clean CSV, JSON, or pandas DataFrames using camelot, tabula-py, pdfplumber, or custom parsers — building reliable table extraction pipelines for data processing workflows.
When to use
- Extracting tabular data from PDF reports, financial statements, or government filings.
- Pulling tables from HTML pages where the data is not available via API.
- Building a pipeline that converts document tables into database-ready structured formats.
- Processing scanned documents with OCR to extract table content from images.
- Handling complex tables with merged cells, multi-row headers, or nested structures.
- Converting document tables to CSV, JSON, or pandas DataFrames for analysis.
When NOT to use
- Creating tables in new documents — prefer
document-writing or docx-generation.
- Generating Excel files from scratch — prefer
xlsx-generation.
- Extracting non-tabular text from PDFs — prefer
pdf-extraction.
- Editing or annotating existing PDFs — prefer
pdf-editor.
- Querying data already available via a REST API — use API clients directly.
Procedure
- Identify the source format. Determine whether the input is a native PDF (text-based), a scanned PDF (image-based requiring OCR), an HTML page, or a standalone image. Test by attempting text extraction: if
pdfplumber returns meaningful text, it is text-based; if only images are detected, it requires OCR.
- Select the extraction tool. Choose based on source type and table structure:
- Text-based PDFs with visible borders: use camelot
flavor='lattice'.
- Text-based PDFs without visible borders: use camelot
flavor='stream' or pdfplumber.
- Scanned PDFs or images: apply OCR first (Tesseract via pytesseract or
pdf2image + OCR), then extract tables from the recognized text.
- HTML pages: use
pandas.read_html() as the first attempt; fall back to BeautifulSoup only if pandas fails on complex nested structures.
- Java-available environments: consider tabula-py as an alternative to camelot for simple bordered tables.
- Configure extraction parameters. Adjust based on document characteristics:
- For camelot lattice with thin ruling lines:
line_scale=15, joint_tol=2, edge_tol=5.
- For camelot lattice with thick ruling lines:
line_scale=40, joint_tol=5, edge_tol=10.
- For camelot stream with tightly packed tables:
row_tol=5, column_tol=5.
- For camelot stream with loosely spaced tables:
row_tol=10, column_tol=10.
- For pdfplumber with partial-page tables: define explicit
page.crop(bounding_box) to isolate the table region before extraction.
- Extract raw table data. Run the extraction and capture the raw DataFrame output. For multi-page documents, iterate pages and extract tables from each:
camelot.read_pdf(path, pages='1-end', flavor='lattice').
- Detect and validate headers. Check if the first row contains header values or data. Heuristics: headers are typically bold, contain no numeric-only values, and have unique values. If headers span multiple rows, merge them into a single header row with concatenated names.
- Handle merged cells. Identify cells spanning multiple rows or columns. For vertically merged cells, forward-fill the value down. For horizontally merged cells, assign the value to the first column and mark others as derived. Log each merge operation for traceability.
Decision rules
- Use camelot lattice mode as the default for PDFs with visible table borders — it is the most reliable.
- Switch to camelot stream mode only when lattice produces zero tables (no ruling lines detected).
- Use pdfplumber when tables have complex layouts that camelot mishandles (nested tables, tables with footnotes embedded in cells).
- Use pandas
read_html() for HTML sources — it handles most standard HTML table markup correctly.
- Apply OCR (Tesseract) only for scanned/image-based documents — it is slow and error-prone on text-based PDFs.
- When extraction confidence is below 80%, flag the output for manual review rather than silently delivering bad data.
Output contract
- Structured data in the requested format (CSV, JSON, or DataFrame) with consistent column types.
- Extraction metadata: source file path, page number, table index, row count, column count, and confidence score.
- A cleaning log listing transformations applied (merged cells filled, whitespace stripped, formats normalized).
- Flagged rows where extraction confidence is low or column counts are inconsistent.
References
Next steps
pdf-extraction — for extracting non-tabular content (text, images, metadata) from PDFs.document-to-structured-data — for broader document-to-data conversion beyond tables.xlsx-generation — for producing Excel files from extracted table data.
Anti-patterns
- Running extraction without inspecting the PDF first — always check if it is text-based or scanned before choosing a tool.
- Using OCR on text-based PDFs — wastes time and introduces errors that do not exist in the source.
- Ignoring merged cells — produces misaligned data where values shift to wrong columns.
- Hardcoding column indices instead of matching by header name — breaks when the source table format changes.
- Treating all extracted numbers as strings — prevents downstream numeric operations and aggregations.
Failure handling
- Zero tables detected by camelot lattice: Automatically retry with
flavor='stream' and tolerance adjustments (row_tol=10, column_tol=10). If still zero tables, inspect the PDF visually to confirm tables exist, then try pdfplumber with explicit bounding boxes.
- OCR produces garbled or low-confidence text: Check source resolution. If below 300 DPI, report: "Re-scan at 300+ DPI and reprocess." If at 300 DPI already, report: "OCR quality insufficient; consider manual review or specialized OCR tuning."
- Column count mismatch across rows: Output the raw extraction with a warning header listing the specific row indices where column counts differ. Include both expected and actual column counts for each mismatched row.
- Missing required library: Output the exact install command and halt immediately:
pip install camelot-py[cv] for camelot with computer vision supportpip install pdfplumber for pdfplumberpip install pytesseract pdf2image for OCR workflowspip install tabula-py for tabula-py (requires Java runtime)
- Password-protected PDF: Report the specific error (
PdfReadError: File has not been decrypted) and request the password or an unlocked version before proceeding.
- Merged cells causing data misalignment: Log each fill operation with source cell coordinates and destination cells. Report: "Merged cell at [row, col] filled to [start_row:end_row, col]."
- Header detection uncertainty: When header confidence is below 80%, output a warning: "Header detection uncertain. Review rows 1-{N} and confirm header boundaries."
- Multi-page extraction partial failure: If some pages fail extraction while others succeed, output successful tables with a failure log listing page numbers and specific error messages for failed pages.
