structured-text-pipeline

name	structured-text-pipeline
description	Design deterministic-first structured text extraction pipelines for repeated document formats, forms, quizzes, receipts, tables, logs, or semi-structured exports. Use when deciding whether to parse with rules, schemas, regex, LLM fallback, confidence scoring, or hybrid validation.

Structured Text Pipeline

Use this skill to turn messy repeated text into reliable structured data without defaulting to all-LLM extraction.

Operating Rule

Start with deterministic parsing when the format repeats. Add an LLM only for low-confidence cases, ambiguous fragments, or formats that are genuinely free-form.

Decision Path

1. Sample at least 10 representative chunks when available.
2. Identify anchors: numbering, headings, labels, delimiters, table columns, choice markers, timestamps, totals, or section boundaries.
3. Classify the source:
   • stable: most records share the same pattern. Use deterministic parsing first.
   • semi-stable: common skeleton with noisy exceptions. Use parser plus confidence scoring.
   • free-form: no reliable skeleton. Use schema-guided LLM extraction directly.
4. Define a target schema before parsing.
5. Build a parser that emits both data and quality signals.
6. Route only flagged records to review or LLM fallback.
7. Record parser metrics: total records, parsed records, rejected records, fallback count, and known failure modes.

Implementation Shape

Use this architecture for stable or semi-stable text:

source text -> normalizer -> deterministic parser -> confidence scorer
            -> accepted records
            -> flagged records -> validator or LLM fallback -> merged output

Keep these pieces separate:

• normalize: fix encoding, whitespace, page markers, bullets, and OCR artifacts.
• parse: extract fields into the target schema.
• score: flag missing fields, suspicious lengths, duplicate IDs, impossible values, or low field coverage.
• fallback: repair only flagged records.
• audit: report metrics and examples of failures.

Confidence Signals

Score records with explicit reasons, not a single unexplained number.

Useful signals:

• required field is missing;
• field value fails an enum, date, currency, or ID pattern;
• answer count or column count is outside expected range;
• record text is unusually short or long;
• duplicate key appears;
• parser consumed too little of the source chunk;
• separators or labels drifted from known examples.

LLM Fallback Rules

Use LLM fallback only when it adds value:

• Provide the exact target schema.
• Send the smallest relevant chunk, not the whole document.
• Ask for JSON only when downstream code needs JSON.
• Validate the LLM result with the same schema checks.
• Preserve the original source span for traceability.
• Never hide fallback count from the final report.

Testing

Create fixtures for:

• clean happy path;
• missing field;
• extra whitespace or OCR noise;
• malformed record;
• duplicate record ID;
• one known real-world edge case.

Acceptance: deterministic parsing handles the stable majority, low-confidence records are explicit, and fallback use is measurable.

Avoid

• Sending every repeated record to an LLM by default.
• Writing one giant parser that normalizes, extracts, validates, and calls LLMs in one function.
• Returning data without confidence or rejection metadata.
• Treating one sample document as the whole format.