cu-sdk-author-analyzer

# Author a Custom Analyzer (single document type) > **[EXPERIMENTAL - REQUIRED FIRST MESSAGE]:** Before asking questions, probing the > environment, or running commands, tell the user: "This analyzer-authoring skill > is experimental and may change. Review generated schemas and test results before > relying on them." Author a custom Content Understanding analyzer for one document type end-to-end: extract layout, draft a field schema, validate locally, create the analyzer, batch-test it on sample files, and read a quality summary. **This is an iterative, human-in-the-loop workflow.** You will typically run the schema → test → review cycle multiple times to refine field descriptions before you're happy with the extraction quality. The workflow uses the typed `ContentUnderstandingClient` shipped in this package — the same client `samples/sample_create_analyzer.py` and `samples/sample_analyze_binary.py` use. > **[COPILOT INTERACTION MODEL]:** This skill is designed to be interactive. > At each step marked with **[ASK USER]**, pause execution and prompt the user > for input or confirmation before proceeding. Do NOT silently skip these > prompts. Use the `ask_questions` tool when available. > **[USE INSTEAD]:** If the user's packet contains **multiple different > document types** (for example, an invoice, a bank statement, and a loan > application in one PDF), route them to the > [`cu-sdk-author-analyzer-classify-route`](../cu-sdk-author-analyzer-classify-route/SKILL.md) > skill instead. This skill assumes one document type per analyzer. > **[ASK USER] Modality check (first thing to confirm):** > > "Are you working with **document** files — PDFs or images of pages? This > skill supports document modalities only." > > - If the user says **document** → continue with this skill. > - If the user says **audio**, **video**, or **image** → stop this skill and > point them at the matching SDK sample > ([`samples/sample_create_analyzer.py`](../../../samples/sample_create_analyzer.py) > with `prebuilt-audio` / `prebuilt-video` / `prebuilt-imageAnalyzer`) or the > [REST tutorial](https://learn.microsoft.com/azure/ai-services/content-understanding/tutorial/create-custom-analyzer). ## Prerequisites Required: venv active, SDK installed, `.env` with `CONTENTUNDERSTANDING_ENDPOINT` (plus `CONTENTUNDERSTANDING_KEY` or `az login`), and `samples/sample_update_defaults.py` run once for this resource. Full setup lives in [`cu-sdk-setup`](../cu-sdk-setup/SKILL.md). > **[COPILOT] Probe first, then route on failure — do not duplicate setup logic here.** > > ```bash > python -c "import sys; print('venv:', 'ok' if sys.prefix != sys.base_prefix else 'INACTIVE')" > ( [ -f .env ] && grep -E '^CONTENTUNDERSTANDING_ENDPOINT=https?://' .env >/dev/null && echo 'endpoint: ok' ) || echo 'endpoint: MISSING' > ( [ -f .env ] && grep -E '^CONTENTUNDERSTANDING_KEY=.+' .env >/dev/null && echo 'key: set' ) || echo 'key: empty' > az account show >/dev/null 2>&1 && echo 'az: ok' || echo 'az: not logged in' > ``` > > | Failure | Route to | > |---|---| > | venv `INACTIVE` or `azure.ai.contentunderstanding` import fails | [`cu-sdk-setup`](../cu-sdk-setup/SKILL.md) Steps 2–3, then resume | > | endpoint `MISSING` | [`cu-sdk-setup`](../cu-sdk-setup/SKILL.md) Step 4 (env vars), then resume | > | endpoint `ok`, key `empty`, `az: not logged in` | [`cu-sdk-setup`](../cu-sdk-setup/SKILL.md) Step 4 auth section (run `az login` **or** add `CONTENTUNDERSTANDING_KEY` to `.env`), then resume | > | All env checks pass but service calls fail with model errors | [`cu-sdk-setup`](../cu-sdk-setup/SKILL.md) Step 6 (`sample_update_defaults.py`), then resume | > | All ok | ✅ Proceed to Step 1. | > > Never ask the user to paste an endpoint or API key into chat — they edit `.env` directly or run `az login`. > **[ASK USER]** "How many representative documents do you have, and where are they?" — fewer than 3 is fine, but more is better for testing coverage. ## Package directory ``` sdk/contentunderstanding/azure-ai-contentunderstanding ``` ## Scripts and templates ``` .github/skills/cu-sdk-author-analyzer/ ├── scripts/ │ ├── extract_layout.py # Stage 1 │ ├── extract_layout.sh │ ├── create_and_test.py # Stage 2 │ └── create_and_test.sh └── templates/ └── schema_template.json # Starter schema for Step 2 ``` See [`scripts/README.md`](scripts/README.md) for a one-page reference. ## Workflow ### Step 1 — Extract layout for representative documents The model behind Content Understanding sees the **text and structure** the service extracts from your file, not the original pixels. Reviewing the layout output is the fastest way to know what labels and headings you can anchor your field descriptions to. > **[ASK USER]** "Point me at one of your sample documents (or a folder of > them). I'll run layout extraction so we can see what the model will be > looking at." Run: ```bash python .github/skills/cu-sdk-author-analyzer/scripts/extract_layout.py \ --input <path-to-folder-or-file> \ --output .local_only/layout/ ``` This produces one `<doc>.layout.md` and one `<doc>.layout.json` per input. Open the `.layout.md` file in VS Code and look for the **text anchors** you want to extract from — labels (`"Invoice #:"`), section headings (`"Bill To"`), table headers, etc. > **Reference**: this is the same call pattern as > [`samples/sample_analyze_binary.py`](../../../samples/sample_analyze_binary.py) > using `prebuilt-documentSearch`. ### Step 2 — Draft a JSON field schema Start from the template instead of writing from scratch: ```bash mkdir -p .local_only/schemas cp .github/skills/cu-sdk-author-analyzer/templates/schema_template.json \ .local_only/schemas/<name>_v1.json ``` Then edit `.local_only/schemas/<name>_v1.json`: set `baseAnalyzerId`, replace every `REPLACE:` placeholder, and add/remove fields. The schema is a JSON object with two required top-level keys: - `baseAnalyzerId` — which prebuilt analyzer your custom analyzer extends. Use the table below. - `fieldSchema.fields` — the named fields you want to extract. > **[COPILOT] Read best practices before drafting fields.** Before writing > any field description, fetch the official CU best-practices page and apply > its guidance: > > 🔗 https://learn.microsoft.com/azure/ai-services/content-understanding/concepts/best-practices > > Key principles that affect schema quality: > > - Be specific and concrete in field descriptions — vague descriptions > produce vague extractions. > - Use **text anchors** (labels, headings, neighbouring fields) — never > visual cues like colour, font, or position-without-text-context. This > matches the two-stage pipeline rule in > [`cu-sdk-common-knowledge`](../cu-sdk-common-knowledge/SKILL.md#field-description-rule-the-two-stage-pipeline). > - Include **format examples** and **alternative label names** when a value > can appear in multiple wordings or formats. > - Prefer `"method": "extract"` when the value appears verbatim; use > `"generate"` only when the model needs to synthesise (summary, > classification label) and `"classify"` for fixed enumerations. > - Keep the field count focused on what the user actually needs. Extra > fields cost tokens and can dilute extraction quality on the fields that > matter. The template demonstrates **all three extraction methods** (`extract`, `generate`, `classify`) plus the **nested object** and **array of objects** shapes. Delete the example fields you don't need. > **Template comment keys**: any key whose name starts with `_` > (e.g. `_comment`, `_optional_enableOcr`) is stripped from the request body > by `create_and_test.py` before it hits the service. Use them freely as > inline documentation. > **`models.completion` is effectively required**: whenever `fieldSchema` is > present, the service needs a completion model. Leave the `models` block in > the template populated unless you've run > [`samples/sample_update_defaults.py`](../../../samples/sample_update_defaults.py) > to set resource defaults. Omitting it fails with `InvalidRequest: > 'models.completion' is not set` *after* a misleadingly successful > `[CREATE]` log line. `create_and_test.py` prints a `[WARN]` if the schema > is missing it. #### Choosing `baseAnalyzerId` See the prebuilt-analyzer table in [`cu-sdk-common-knowledge` § Choosing `baseAnalyzerId`](../cu-sdk-common-knowledge/SKILL.md#choosing-baseanalyzerid). The local validator (Step 3) rejects any value not on that list. #### Example single-type schema ```json { "baseAnalyzerId": "prebuilt-document", "description": "Extract invoice header and totals.", "config": { "estimateFieldSourceAndConfidence": true, "returnDetails": true }, "models": { "completion": "gpt-5.2", "embedding": "text-embedding-3-large" }, "fieldSchema": { "name": "invoice_v1", "description": "Invoice header fields.", "fields": { "invoiceNumber": { "type": "string", "method": "extract", "description": "Invoice number printed near the 'Invoice #' label at the top of the page.", "estimateSourceAndConfidence": true }, "totalAmount": { "type": "number", "method": "extract", "description": "Grand total at the bottom of the document; typically labelled 'Total' or 'Amount Due'. Excludes any 'Subtotal' value.", "estimateSourceAndConfidence": true } } } } ``` > **Reference**: see > [`samples/sample_create_analyzer.py`](../../../samples/sample_create_analyzer.py) > for the typed-model equivalent. The script accepts a JSON dict directly > (the SDK's `begin_create_analyzer` is overloaded for both). > **Field-description rule (two-stage pipeline):** descriptions must reference > **text content and structure** (labels, headings, neighbouring fields), not > visual appearance (colour, font, size). See > [`cu-sdk-common-knowledge`](../cu-sdk-common-knowledge/SKILL.md) § > "two-stage pipeline". ### Step 3 — Validate the schema locally ```bash python .github/skills/cu-sdk-author-analyzer/scripts/create_and_test.py \ --schema .local_only/schemas/invoice_v1.json \ --input samples/sample_files/sample_invoice.pdf \ --output .local_only/test_results/v1 ``` The script runs the local validator first. If anything is wrong (unknown `baseAnalyzerId`, missing `fieldSchema`, malformed entries) it exits with code **2** *before* any service call. ### Step 4 — Read the stdout summary After the script finishes you get something like: ``` ======================================================================== [SUMMARY] category: (single) (3 document segments) ------------------------------------------ field fill rate avg conf invoiceNumber 100.0% 0.962 totalAmount 66.7% 0.481 lowest-confidence fields: 0.461 totalAmount (mixed_financial_docs) 0.732 invoiceNumber (mixed_financial_docs) ======================================================================== ``` For each input document the script writes two files into `--output`: - `<doc>.json` — full per-document `AnalysisResult` (fields, grounding, confidences). - `<doc>.llm.md` — same result rendered via the SDK's [`to_llm_input`](../../../samples/sample_create_classifier.py) helper: YAML front matter (category, page range, fields) plus the document text. Drop this straight into an LLM prompt, or skim it in VS Code for a fast human review. ### Step 5 — Agent review and iterate > **[COPILOT]** After the summary prints, do the following **automatically** > before asking the user anything: > > 1. Open each `<doc>.llm.md` (and the underlying `<doc>.json` for grounding) > in `.local_only/test_results/` and compare extracted field values