Chunk a document into provenance-carrying JSONL ready for embedding via the Nutrient Data Extraction API (`/extraction/parse`, spatial output). Every chunk carries element type, page index, bounding box, confidence, and reading order, traceable to a specific page region. Use for grounded or auditable RAG, cited retrieval, confidence-aware ingestion, or any pipeline that must prove which document region a retrieved answer came from. Triggers include grounded RAG, auditable RAG, RAG with provenance, chunk with bounding box, cite page, provenance-carrying chunks, confidence-aware chunking, or embedding pipeline. Not for cheap Markdown โ use document-extraction-api with `--output-format markdown` for that. Not for known target fields โ if the user wants specific fields ("invoice number and total", "map to my schema", "return these fields cited"), use document-extraction-api's `extract` (one cited call), not whole-document chunking.
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
A direct command skips the review prompt. Inspect the source before running it.
Chunk a document into provenance-carrying JSONL ready for embedding via the Nutrient Data Extraction API (`/extraction/parse`, spatial output). Every chunk carries element type, page index, bounding box, confidence, and reading order, traceable to a specific page region. Use for grounded or auditable RAG, cited retrieval, confidence-aware ingestion, or any pipeline that must prove which document region a retrieved answer came from. Triggers include grounded RAG, auditable RAG, RAG with provenance, chunk with bounding box, cite page, provenance-carrying chunks, confidence-aware chunking, or embedding pipeline. Not for cheap Markdown โ use document-extraction-api with `--output-format markdown` for that. Not for known target fields โ if the user wants specific fields ("invoice number and total", "map to my schema", "return these fields cited"), use document-extraction-api's `extract` (one cited call), not whole-document chunking.
license
MIT
metadata
{"author":"nutrient-sdk","version":"1.0","homepage":"https://www.nutrient.io/api/data-extraction-api/","repository":"https://github.com/PSPDFKit-labs/nutrient-skills","compatibility":"Requires Python 3.10+, uv, and internet. Works with Claude Code, Codex CLI, Gemini CLI, OpenCode, Cursor, Windsurf, GitHub Copilot, Amp, or any Agent Skills-compatible product.","short-description":"Chunk documents into provenance-carrying JSONL for grounded, auditable RAG"}
Grounded RAG Ingestion
Turn a document into a stream of provenance-carrying chunks โ one JSONL line per chunk,
each tagged with its element type, page index, bounding box, confidence score, and
reading-order index, all traceable back to a specific region of the source page.
The differentiator is provenance, a capability frontier-LLM Markdown extraction structurally
cannot supply: a downstream system can highlight the exact region of the source document a
retrieval result came from. That is the compliance-grade requirement for RAG in finance,
healthcare, and legal workflows where "the model said so" is not an acceptable citation. This
skill does not relitigate raw Markdown quality (see the pdf-to-markdown skill for that) โ it
adds the provenance layer on top of Nutrient extraction.
This is a doer skill: a bundled uv-runnable Python script (chunk.py) drives the pipeline
end to end and emits JSONL to stdout or a file. It is embedding-agnostic โ it stops at the
chunk boundary. No vector DB client, no embedding provider import.
When to use
Grounded / cited RAG: retrieval results include page_index and bbox so the UI can draw
a highlight box on the source document.
Confidence-aware ingestion: gate on confidence >= threshold to suppress low-confidence
chunks before they reach the index.
Reading-order-aware chunking: preserve reading_order so sliding-window chunking respects
the document's logical sequence.
Auditable pipelines: regulated workflows that must prove which page region a retrieved
answer came from โ the bbox + page_index tuple is the audit trail.
Not for cheap whole-document Markdown โ that stays in document-extraction-api with
--output-format markdown. Not for known target fields โ if the request names specific
fields ("the invoice number and total", "extract these fields", "map to my schema", "return these fields cited"), use document-extraction-api's extract (one call returns your schema's fields,
each cited) instead of chunking the whole document for open-ended retrieval. Reach for this
skill when retrieval is open-ended (semantic search over the whole document), not when the
target fields are already known. Not for vector-store upsert or embedding generation โ this
skill stops at the chunk JSONL.
Dependency: document-extraction-api
This skill calls document-extraction-api under the hood. Read that skill first for mode
selection, key setup, and credit-cost guidance. chunk.py reuses its create_client() helper
and client.parse(..., output_format="spatial") call โ it does not re-implement the API call.
The extraction skill must be present in the same plugin at
../document-extraction-api/scripts/. If it lives elsewhere, set PARSE_SCRIPT_PATH to its
scripts/ directory; chunk.py fails fast with a clear message if it cannot find it.
Setup
DWS Extract is a separate product from DWS Processor and has its own API key.
cd <directory containing this SKILL.md> && uv run scripts/chunk.py --help
If your tenant has migrated to global DWS API keys, a single key set as either
NUTRIENT_EXTRACT_API_KEY or NUTRIENT_API_KEY works for both products.
Mode selection
Mode selection for this skill follows the same rules as document-extraction-api (mode โ
cost โ when to use). Pass --mode through to the parse call; the default is structure.
Mode
Cost (cr/pg)
When
text
1
Cheapest; incompatible with spatial โ not usable here (this skill requires spatial)
structure
1.5
Default. OCR + spatial typed elements with bounds and tables
understand
9
AI layout: required for keyValueRegion and formula elements
agentic
18
VLM: adds altDescription on pictures
Mode-gating โ important.keyValueRegion and formula elements are only populated by
understand mode or higher. Under the default structure mode the key-value and formula
chunk paths produce nothing. chunk.py warns on stderr when KV/formula content would be
expected but the mode is too low. To exercise key-value chunking, pass --mode understand.
element.page.pageIndex (null when the element carries no page โ never fabricated 0)
reading_order
int|null
element.readingOrder (null sorts last)
bbox
object|null
{x, y, width, height} (normalized at runtime โ see OQ-1; null when bounds are missing, never {0,0,0,0})
confidence
float|null
element-level confidence (0โ1); null when unknown
text
string
element text per type-dispatch rules
chunking_warning
string?
present only on table span-expansion fallback chunks
chunk_id is deterministic: {doc_id}__p{page_index}_r{reading_order}_e{element_index}{disc}.
_e{element_index} (the element's position in the (page_index, reading_order) sort)
guarantees inter-element uniqueness even when reading_order is null or shared by two
elements.
{disc} is the intra-element discriminator for elements that emit multiple chunks:
_tr{row} (table-row), _kv{pair_index} (key-value pairs), _w{window_index}
(reading-order windows); empty for single-chunk elements.
doc_id is --doc-id if supplied, else a short content hash of the input bytes โ never the
basename, so two same-named files in different directories get disjoint IDs and upsert stays
idempotent and tenant-safe.
Full spec: references/provenance-chunk-schema.md.
Chunking strategies
--strategy element (default): one chunk per typed spatial element โ the purest expression
of the grounding value prop. A table becomes one whole-table TSV chunk.
--strategy table-row: each table row becomes its own chunk (_tr{row}) โ better precision on
wide financial tables.
--strategy reading-order-window: sliding window over reading-order-sorted text elements
(--window-size in tokens, _w{window_index}) โ better retrieval coherence for prose, at some
provenance precision (a window may span two elements' bboxes).
Cost gate
chunk.py runs its own preflight before the network call: it counts the input PDF's pages
locally and estimates pages ร mode-cost. If the estimate exceeds 200 credits (or the API's
remainingCredits when known), it refuses to proceed unless --yes is passed or interactive
confirmation is given. (The extraction skill's 200-credit rule is an agent instruction, not
enforceable code โ this gate is real code here.) For non-PDF inputs where local page count is
unavailable, the gate requires --yes for understand/agentic runs.
Illustrative: passing chunks to an embedder
The boundary is the chunk schema. See examples/embed-chunks-illustrative.py for a minimal,
illustrative-only example of loading the JSONL and passing text to an embedder while
keeping chunk_id + bbox + page_index provenance attached to each embedding record. Swap the
provider import for your stack โ no embedding library is a dependency of this skill.
Anti-patterns
Do not use this skill when raw Markdown is sufficient โ use document-extraction-api with
--output-format markdown.
Do not bundle vector-DB or embedding-provider logic into chunk.py โ the boundary is the
chunk JSONL.
Do not expect key-value or formula chunks under the default structure mode โ they require
--mode understand.
Do not derive doc_id from the filename basename โ it breaks cross-document collision
safety (R15).
Security Hardening Addendum
Never store NUTRIENT_EXTRACT_API_KEY in committed files. Use process env injection at runtime
(shell/export, secrets manager, or host env).
The output JSONL carries extracted document text (potentially finance/health/legal content) and
is written with 0600 permissions. Treat it as sensitive; do not commit it.
source_doc carries the input basename, which may encode PII (e.g. patient-jsmith.pdf). For
sensitive corpora, supply a sanitized --doc-id and be deliberate about what reaches the index.
The script prints credit usage (the numeric cost only) to stderr and never logs the API key.
Rules
Always require spatial output โ Markdown output loses provenance and is rejected.
Preserve the printed credit-usage summary so the operator can observe per-call cost.
Reuse document-extraction-api's create_client() and client.parse(); do not re-implement
the API call or mode-selection logic.
