structured-output-llm

name	structured-output-llm
description	Gets machine-parseable JSON out of an LLM reliably — prefer provider-enforced grammar (OpenAI `response_format:{type:"json_schema",strict:true}`, Anthropic tool-calling / `tool_choice:{type:"tool"}`, Gemini `responseSchema`) over a "respond in JSON" prompt, define the contract once as a Pydantic/Zod model and validate every response (never trust the string), use constrained decoding / grammars for open weights (Outlines, llguidance, llama.cpp GBNF, vLLM `guided_json`), keep schemas SHALLOW with required fields + enums + `additionalProperties:false`, and build a repair-and-retry loop that handles refusals, `length`/`max_tokens` truncation, and streaming partial JSON (`partial-json-parser`). Schema-guided generation eliminates parse-error retries; validate-and-repair is the backstop, not the primary mechanism.
when_to_use	You need a model to emit JSON/objects a program consumes — extraction, classification into fixed labels, function arguments, form-filling, or feeding output into another service — and string-parsing or regex on free text keeps breaking. Distinct from prompt-engineering (crafts the instruction/few-shot prose; this skill enforces the output SHAPE via grammar+validation) and agent-tool-mcp-builder (defines the tools/MCP an agent decides to call; this skill is the one-shot "make THIS call return a typed object" half, even when implemented via a single forced tool).

When to Use

Reach for this skill when a program — not a human — consumes the model's output and the shape must be guaranteed:

• "Extract these 8 fields from this invoice/email as JSON I can JSON.parse"
• "Classify into one of {refund, dispute, question} — give me just the label"
• "The model returns ```json fences / prose / trailing commas and my parse breaks ~3% of the time"
• "Function arguments come back malformed or with hallucinated keys"
• "Fill this form schema" / "map this text to my DB row"
• "Open-weights model won't reliably produce valid JSON"
• "Long extraction gets cut off mid-object" or "streaming JSON is unparseable until complete"

NOT this skill:

• Writing the instruction, few-shot examples, role/system prompt, or chain-of-thought prose → prompt-engineering (it shapes WHAT to say; this skill enforces the machine SHAPE of the reply)
• Defining tools/an MCP server an agent autonomously selects among across many turns → agent-tool-mcp-builder (this skill is one forced, schema-constrained call returning a typed object)
• Reliability of the whole LLM app — timeouts, fallbacks, circuit-breakers, idempotency around the call → harden-llm-app-reliability (this skill owns only the output-shape contract)
• Token/latency/cost tuning, caching, model selection → optimize-llm-cost-latency (note: strict schemas cost a cache miss on first use — that skill weighs it)
• Stopping a malicious payload in retrieved/user text from hijacking the model → defend-llm-prompt-injection (orthogonal; a valid-schema output can still be adversarial content)
• Retrieval/chunking/grounding that feeds the prompt → rag-pipeline (this skill structures rag's output step)
• Scoring whether the extracted values are correct across a dataset → llm-eval-harness (this skill guarantees parseable; that one measures accuracy)
• Validating ordinary user-submitted form data (no LLM) → build-form-validation
• The wire/serialization contract between your own services → rest-graphql-contract
• Scraping the HTML/DOM the text came from → scrape-structured-web-data

Steps

1. Prefer provider-enforced schema over a "respond in JSON" prompt — it's a different mechanism, not a stronger hint. A prompt is a request the model may ignore (fences, preamble, extra keys); enforced schema masks the decoder's token logits so only schema-valid tokens are emittable — invalid JSON becomes impossible, not just unlikely. Use the strongest enforcement the provider offers:

   Provider	Enforcement	How
   OpenAI (gpt-4o+, gpt-4.1, o-series)	Structured Outputs, 100% guaranteed	response_format:{type:"json_schema",json_schema:{name,schema,strict:true}} — or strict:true on a function tool
   Anthropic Claude	Tool-calling (no native json_schema)	define one tool with input_schema, set tool_choice:{type:"tool",name:"..."} to force it; read tool_use.input
   Google Gemini	Controlled generation	generationConfig:{responseMimeType:"application/json",responseSchema:{...}} (also supports responseSchema enums)
   Azure OpenAI	same as OpenAI	response_format json_schema on 2024-08-01-preview+
   Open weights (vLLM/TGI/llama.cpp/Ollama)	Constrained decoding (step 5)	guided_json / GBNF grammar / Outlines

   json_object mode (the older {type:"json_object"}) only guarantees syntactically valid JSON, NOT your schema — treat it as a weak fallback, never the goal.

2. Define the contract ONCE as a typed model; let the SDK emit the schema and parse the result. Don't hand-write JSON Schema and a parser separately — they drift. Use the model class as the single source of truth:

   • Python → Pydantic v2. OpenAI: client.beta.chat.completions.parse(..., response_format=MyModel) → .choices[0].message.parsed is a typed instance (or .refusal). Or MyModel.model_validate_json(raw).
   • TS/JS → Zod + the OpenAI helper , then ; or . (Instructor / wrap this with retries.)