cu-sdk-author-analyzer-classify-route

# Author a Classify-and-Route Analyzer (mixed document packets) > **[EXPERIMENTAL - REQUIRED FIRST MESSAGE]:** Before asking questions, probing the > environment, or running commands, tell the user: "This analyzer-authoring skill > is experimental and may change. Review generated schemas and test results before > relying on them." Build a classify-and-route pipeline: one **outer classifier analyzer** that segments and labels a multi-document packet, plus one **inner extractor analyzer per document type**. The packet flows through the outer analyzer once; each segment is automatically routed to the matching inner analyzer for field extraction. **This is an iterative, human-in-the-loop workflow.** You will typically run the schema → test → review cycle multiple times to refine both the outer classifier descriptions and each inner schema's field descriptions before you're happy with both classification accuracy and extraction quality. > **[USE INSTEAD]:** If every page in the user's documents is the **same > type** (only invoices, only contracts, etc.), use > [`cu-sdk-author-analyzer`](../cu-sdk-author-analyzer/SKILL.md) instead. > Classify-and-route is for **mixed** packets. > **[ASK USER] Modality check (first thing to confirm):** > > "Are you working with **document** files — PDFs or images of pages? This > skill supports document modalities only." > > - If the user says **document** → continue with this skill. > - If the user says **audio**, **video**, or **image** → stop this skill. > Point them at the [REST tutorial](https://learn.microsoft.com/azure/ai-services/content-understanding/tutorial/create-custom-analyzer). > **[COPILOT INTERACTION MODEL]:** At each step marked with **[ASK USER]**, > pause execution and prompt the user before proceeding. ## Prerequisites Required: venv active, SDK installed, `.env` with `CONTENTUNDERSTANDING_ENDPOINT` (plus `CONTENTUNDERSTANDING_KEY` or `az login`), and `samples/sample_update_defaults.py` run once for this resource. Full setup lives in [`cu-sdk-setup`](../cu-sdk-setup/SKILL.md). > **[COPILOT] Probe first, then route on failure — do not duplicate setup logic here.** > > ```bash > python -c "import sys; print('venv:', 'ok' if sys.prefix != sys.base_prefix else 'INACTIVE')" > ( [ -f .env ] && grep -E '^CONTENTUNDERSTANDING_ENDPOINT=https?://' .env >/dev/null && echo 'endpoint: ok' ) || echo 'endpoint: MISSING' > ( [ -f .env ] && grep -E '^CONTENTUNDERSTANDING_KEY=.+' .env >/dev/null && echo 'key: set' ) || echo 'key: empty' > az account show >/dev/null 2>&1 && echo 'az: ok' || echo 'az: not logged in' > ``` > > | Failure | Route to | > |---|---| > | venv `INACTIVE` or `azure.ai.contentunderstanding` import fails | [`cu-sdk-setup`](../cu-sdk-setup/SKILL.md) Steps 2–3, then resume | > | endpoint `MISSING` | [`cu-sdk-setup`](../cu-sdk-setup/SKILL.md) Step 4 (env vars), then resume | > | endpoint `ok`, key `empty`, `az: not logged in` | [`cu-sdk-setup`](../cu-sdk-setup/SKILL.md) Step 4 auth section (run `az login` **or** add `CONTENTUNDERSTANDING_KEY` to `.env`), then resume | > | All env checks pass but service calls fail with model errors | [`cu-sdk-setup`](../cu-sdk-setup/SKILL.md) Step 6 (`sample_update_defaults.py`), then resume | > | All ok | ✅ Proceed to the Packet check below. | > > Never ask the user to paste an endpoint or API key into chat — they edit `.env` directly or run `az login`. > **[ASK USER] Packet check:** > 1. "Does each document in your packet contain more than one type of form (e.g. an invoice page followed by a bank statement page)?" — if no, route to `cu-sdk-author-analyzer`. > 2. "What types of documents appear in your packets?" — capture as the list of inner analyzers. ## Architecture ``` ┌──────────────────────────────┐ mixed packet ───► │ outer (classifier) analyzer │ │ baseAnalyzerId: prebuilt-… │ │ config.enableSegment: true │ │ config.contentCategories: │ │ invoice ────────►│──┐ │ bank_statement ────────►│──┼──► per-segment fields │ loan_application ────────►│──┘ └──────────────────────────────┘ │ inner analyzers (1 per type) │ ─────────────────────────── ▼ invoice extractor ◄──── routes here for invoice pages bank statement ext. ◄──── routes here for bank pages loan app extractor ◄──── routes here for loan pages ``` Key rules (also captured in [`cu-sdk-common-knowledge`](../cu-sdk-common-knowledge/SKILL.md) § "classify-and-route"): 1. **Category descriptions reference text anchors**, not visual cues (matches the two-stage pipeline rule for fields). 2. **`config.enableSegment` must be `true`** so the classifier can carve up the packet before routing. 3. **Inner analyzers must exist before** the outer classifier is created. The provided script handles ordering automatically. 4. **Category fill rate is per-category**, not packet-wide. The script's stdout summary uses the right denominator. ## Package directory ``` sdk/contentunderstanding/azure-ai-contentunderstanding ``` ## Scripts and templates ``` .github/skills/cu-sdk-author-analyzer-classify-route/ ├── scripts/ │ ├── create_and_test_router.py │ └── create_and_test_router.sh └── templates/ └── classifier_template.json # Starter outer-classifier schema for Step 3 ``` ## Workflow ### Step 1 — Identify the document types Run layout extraction (same as the single-type skill) on a representative packet to see the section headings: ```bash python .github/skills/cu-sdk-author-analyzer/scripts/extract_layout.py \ --input <packet.pdf> \ --output .local_only/layout/ ``` > **[COPILOT]** Read `.local_only/layout/<packet>.layout.md` **yourself** and > identify section headings, page breaks, and content shifts that suggest > different document types. Then present your analysis to the user for > confirmation — do **not** ask the user to read the layout cold. > > Example presentation: > > > "Based on the layout in `<packet>.layout.md` I see these document types: > > > > - **Pages 1–2** — appears to be an *invoice* (top heading 'Invoice', > > contains 'Invoice #' label and a line-item table) > > - **Pages 3–4** — appears to be a *bank statement* (top heading > > 'Account Statement', contains account number and a transaction table) > > - **Page 5** — appears to be a *loan application* (top heading 'Loan > > Application', applicant fields) > > > > Does this look right? Anything to add, remove, or rename before I > > draft schemas?" > > Only fall back to a blank `[ASK USER]` ("What types do you see?") if the > layout is too ambiguous to suggest types confidently. ### Step 2 — Draft one inner schema per type Treat each type as a single-doc-type analyzer (pick `baseAnalyzerId` from the table in [`cu-sdk-common-knowledge` § Choosing `baseAnalyzerId`](../cu-sdk-common-knowledge/SKILL.md#choosing-baseanalyzerid), then add `fieldSchema.fields`). Field descriptions follow the [two-stage pipeline rule](../cu-sdk-common-knowledge/SKILL.md#field-description-rule-the-two-stage-pipeline) — reference text and structure, never visual appearance. See [`cu-sdk-author-analyzer`](../cu-sdk-author-analyzer/SKILL.md) Step 2 for the full field-schema walkthrough. > **[COPILOT] Read best practices before drafting fields.** Before writing > any field description (in any inner schema or the outer classifier's > category descriptions), fetch the official CU best-practices page and > apply its guidance: > > 🔗 https://learn.microsoft.com/azure/ai-services/content-understanding/concepts/best-practices > > The same key principles apply here as in > [`cu-sdk-author-analyzer`](../cu-sdk-author-analyzer/SKILL.md#step-2--draft-a-json-field-schema): > be specific and concrete, use text anchors (not visual cues), include > alternative labels and format examples, prefer `extract` for verbatim > values, and keep the field count focused. > **Reference**: > [`samples/sample_create_classifier.py`](../../../samples/sample_create_classifier.py) > ships a complete worked example using `samples/sample_files/mixed_financial_docs.pdf` > with three categories — Invoice, Bank_Statement, Loan_Application. ### Step 3 — Draft the outer classifier schema The outer schema has **no** `fieldSchema`. Its job is classification + routing. Start from the template: ```bash mkdir -p .local_only/schemas cp .github/skills/cu-sdk-author-analyzer-classify-route/templates/classifier_template.json \ .local_only/schemas/<name>_classifier_v1.json ``` Example after editing (descriptions intentionally generic so they survive minor wording variants — see the **Category description rule** below the example for guidance on writing your own): ```json { "baseAnalyzerId": "prebuilt-document", "description": "Classify mixed financial packets and route to per-type extractors.", "config": { "enableSegment": true, "omitContent": true, "contentCategories": { "invoice": { "description": "A commercial invoice. Look for an invoice / bill heading or label, an invoice number, line-item table with quantities and prices, and a total amount.", "analyzerId": "invoice" }, "bank_statement": { "description": "A bank or account statement. Look for an account-statement heading, an account number, a statement period or date range, and a transaction table with running balance.", "analyzerId": "bank_statement" }, "loan_application": { "description": "A loan or credit application form. Look for an application heading, applicant or borrower fields, requested loan amount, and applicant signature.", "analyzerId": "loan_application" } } }, "models": { "completion": "gpt-5.2", "embedding": "text-embedding-3-large" } } ``` The `analyzerId` value in each category is **an alias** that the script resolves at runtime, matching the `--inner-schema alias=path` flags you pass. Two exceptions skip alias resolution: * Values starting with `prebuilt-` (e.g. `prebuilt-invoice`) are used as-is — no inner schema needed. Useful for routing a category at a service prebuilt extractor. * Categories without an `analyzerId` at all are classification-only — the segment is labelled but no fields are extracted. > **Why `omitContent: true`?** When omitted, the service also returns the > raw, un-segmented document content as an extra entry in `contents`. That > entry has no category, no fields, and shows up in the summary as a > confusing `(uncategorized)` row. Setting `omitContent: true` removes it. > **Category description rule — the example values above are demo-specific.** > > The category descriptions in the JSON above are tied to the demo packet > `samples/sample_files/mixed_financial_docs.pdf` (which uses the literal > headings `Invoice`, `Bank Statement`, `Loan Application`). **Do not copy > them verbatim.** When authoring for the user's own packet, write > descriptions based on what you observed in **the user's** layout output > from Step 1 — use the actual headings, labels, and structural markers > from their documents. > > Keep descriptions: > > - **Generic over surface form** — describe the *kind* of content > ("contains a transaction table with running balance") rather than > hardcoding one specific header string, so minor wording variants still > classify correctly. > - **Concrete enough to be discriminative** — include at least one anchor > that distinguishes this category from the others in the packet. > - **Text-anchored, not visual** — reference headings, labels, and > neighbouring text, never colour / font / position-without-text. Same > reason as the field-description rule in > [`cu-sdk-common-knowledge`](../cu-sdk-common-knowledge/SKILL.md#field-description-rule-the-two-stage-pipeline). ### Step 4 — Validate, create, and batch-test ```bash python .github/skills/cu-sdk-author-analyzer-classify-route/scripts/create_and_test_router.py \ --outer-schema .local_only/schemas/classifier.json \ --inner-schema invoice=.local_only/schemas/invoice.json \ --inner-schema bank_statement=.local_only/schemas/bank_statement.json \ --inner-schema loan_application=.local_only/schemas/loan_application.json \ --input samples/sample_files/mixed_financial_docs.pdf \ --output .local_only/test_results/v1 ``` > **Shortcut — `--schema-dir`:** if your inner schema filenames match the > outer-schema category aliases (e.g. `.local_only/schemas/invoice_v1.json` for category > `invoice`), replace every `--inner-schema alias=path` with a single > `--schema-dir .local_only/schemas/`. The script picks the newest matching file per > alias (alphabetical sort, so `invoice_v2.json` wins over `invoice_v1.json`). > **Iteration helper — `--reuse`:** add `--reuse` to name analyzers by a > sha1 of their schema (`<stem>_<hash[:8]>`) and skip creation when an > analyzer with that ID already exists. Re-running with the same schemas > is a no-op on the create side, so you don't pile up stale analyzers while > iterating. Edit a schema → hash changes → new analyzer is created. The script: 1. Validates every schema (exits with code **2** if any fails — no service call made). 2. Errors out if the outer schema references an alias that has no matching `--inner-schema`, or if you supply an `--inner-schema` that no category uses. 3. Creates inner analyzers first, then patches and creates the outer classifier. 4. Analyzes every input file, writing one JSON per file under `--output`. 5. Prints a **category-aware** stdout summary (per-category fill rate uses each category's segment count, not the packet-wide total). ### Step 5 — Read the category-aware summary Example output: ``` ======================================================================== [SUMMARY] (category-aware) category: bank_statement (1 segments)