agentsop-signature-design

name	agentsop-signature-design
version	0.1.0
description	Decision rubric for promoting a prose prompt into a typed DSPy Signature. This is an ENHANCE overlay on top of the [[dspy]] library skill: it does NOT teach DSPy syntax — it answers the coder-agent decision "when do I stop hand-writing a prompt string and declare it as a `dspy.Signature`, and how do I name/describe its fields so the optimizer and the calling code both get a clean contract." Activate when: a prompt string grows past ~50 lines; the LM output is consumed by code (parsed, branched on, stored) rather than read by a human; the same prompt is reused across >1 call site; or a teammate asks "should this be a Signature?". Do NOT activate for one-shot throwaway prompts, or for HOW-TO questions about DSPy modules /optimizers/compile — defer those to the [[dspy]] skill and the [[agentsop-dspy]] workflow skill. Search keywords: typed prompt, structured prompt, DSPy Signature, prompt as a function, prompt contract, when to formalize a prompt.

Signature-Design — Promote Prose → Typed Contract

"DSPy uses the field names as the only natural-language hint the optimizer has about intent before it sees data. Name them like you'd name function parameters in well-written code." — derived from [dspy.ai/learn/programming/signatures/], see references/R1-source-evidence.md

This skill is the decision layer, not the library layer. It tells you when a prose prompt has become "load-bearing" enough to deserve a typed Signature, and how to shape its fields. For the actual API (dspy.Signature, InputField, OutputField, Predict, ChainOfThought, compile, save) defer to the [[dspy]] skill; for the full program→evaluate→optimize SOP defer to [[agentsop-dspy]].

---

1. 何时激活 (When to activate)

Activate this overlay the moment a hand-written prompt crosses any one of three load-bearing thresholds.

Trigger	Concrete signal	Why it matters
Length	A single prompt string grows past ~50 lines of f-string / template	Long prose prompts hide their I/O contract inside narration; the [[agentsop-dspy]] skill names this exact symptom: "hand-written prompts grow past ~50 lines; brittleness on model swap" (R1, claim S1)
Code-consumed output	The LM response is parsed, branched on, or stored by downstream code (not just shown to a human)	If code reads the output, the output has a type. An untyped prompt forces brittle regex/JSON-scraping at every call site
Reuse	The same prompt (or a copy-pasted variant) is called from >1 call site or in a loop	Reuse means the contract is now an API surface. Drift between copies is a guaranteed bug source

Secondary signals (each strengthens, none alone is sufficient):

• The prompt is about to be model-swapped (GPT → Llama) and you fear it will break — Signatures + recompile is the documented fix (R1, claim S6; see [[agentsop-dspy]] Case B).
• A metric already exists for this task — you are one step from optimization, and optimizers require a Signature.
• The prompt mixes task instruction + few-shot demos + format spec in one blob — Signatures separate these cleanly.

Do NOT activate when:

• The prompt is one-shot ("summarize this one email") — keep it as a raw string; the contract has no second reader.
• The task signature is still changing daily — promoting now just churns boilerplate. Wait for the I/O to stabilize (R1, claim S7).
• The question is HOW to write the Signature class / pick a module / compile — that is the [[dspy]] skill's job, not this rubric's.
• Output must be free-form human prose with no downstream parsing and no reuse — a Signature buys nothing.

---

2. 核心心智模型 (Core mental model)

A Signature is a typed function contract for a single LM call. Promote a prose prompt to a Signature exactly when the prompt becomes load-bearing — when something other than a one-time human reader depends on its shape.

Think of the progression as the same lifecycle a script goes through when it earns a function:

prose prompt string          →   typed Signature
─────────────────────────        ─────────────────────────
"You are an expert... given     class Classify(dspy.Signature):
 the ticket below, output           """Route a support ticket."""
 the category and a one-line        ticket: str  = dspy.InputField()
 reason. Categories are..."         category: Literal[...] = dspy.OutputField()
                                     reason: str  = dspy.OutputField(desc="<=15 words")
inline narration of I/O          explicit, named, typed I/O
human reads / eyeballs           code parses category, logs reason
each caller copies the blob      one contract, N callers import it
optimizer sees nothing           optimizer rewrites instructions, keeps field names

Three load-bearing ideas (all sourced; see references/R1-source-evidence.md):

1. Field names are the contract. Before the optimizer ever sees data, the only intent signal it has is the field names. question -> answer ≠ query -> response. Name fields like function parameters in clean code (R1, claim S2). This is the reason promotion is worth it: you convert narration into a machine-readable intent signal.

2. The Signature shape is YOUR code; the prompt text is the optimizer's. When you compile, the optimizer rewrites instructions and demos — but it never changes field names, field count, or types (R1, claim S5). So the Signature is the stable seam between "what I own" and "what the compiler owns." A prose prompt has no such seam — everything is tangled.

3. Promote on load-bearing, not on aspiration. A Signature you optimize a 5-line one-shot prompt into is pure overhead. The payoff appears only when the prompt is long, code-consumed, or reused. Below that line, raw prompting wins (R1, claims S7, S1).

The PyTorch analogy from [[agentsop-dspy]] holds: a Signature ≈ a forward() shape contract. You don't write a nn.Module for a one-line lambda; you write one when the shape is reused and trained.

---

3. SOP 工作流 (The promotion SOP)

A four-step gate. Run it top-to-bottom; each step has an exit criterion. Implementation of any step lives in the [[dspy]] skill — this SOP only tells you what decision to make at each step.

Step 0 — Gate: is this prompt load-bearing?

Run the §1 trigger table. If zero triggers fire → stop, keep the prose prompt. Promotion is overhead. Exit: at least one of {>50 lines, code-consumed output, reused} is true.

Step 1 — Identify inputs and outputs

Read the prose prompt and extract every variable thing the LM is given (inputs) and every distinct thing it must return (outputs). A common smell: the prose says "output the category and a confidence and a reason" — that is three output fields, not one paragraph to regex later. Exit: you can list inputs and outputs as a flat set of named slots, each with a Python type.

Step 2 — Name fields semantically

Rename each slot to read like a function parameter. text → ticket, out → category, resp → reason. The name carries the optimizer's only pre-data intent signal (R1, claim S2). Avoid generic input/output. Exit: every field name would be self-explanatory to a teammate reading only the field list.

Step 3 — Add descriptions only where the name underspecifies

Add InputField(desc=...) / OutputField(desc=...) only when the field name alone is ambiguous or the value needs a constraint the name can't carry (format, length, units, allowed values). The DSPy cheatsheet's own example adds a desc on output (answer, desc="often between 1 and 5 words") but leaves the input bare (R1, claim S3). Over-describing every field bloats the prompt and fights the optimizer. Exit: descriptions exist for exactly the fields that need disambiguation, and no others.

Step 4 — Hand off module choice to [[dspy]]

Picking Predict vs ChainOfThought vs ReAct, then evaluating and compiling, is out of scope for this decision skill — that is the [[dspy]] skill (modules) and [[agentsop-dspy]] (Stage 1–3 workflow). Your deliverable from this skill is a well-shaped Signature, handed to those skills. Exit: Signature is named, typed, minimally described, and committed; you have switched contexts to [[dspy]].

When to iterate back

If, after compiling (in [[dspy]]), the optimizer plateaus, the most common root cause is an ambiguous Signature — not a bad optimizer (R1, claim S8). Loop back to Step 1: are inputs/outputs really separated? Are field names carrying intent?

---

4. 操作模型 (Operations — Trigger / Action / Output / Evidence)

Eight operations. The first two are the gate; the rest are the shaping rubric. Full Trigger/Action/Output/Evidence records are in intermediate/operation_candidates.json.

4.1 Promote-trigger checklist (the gate)

Promote prose → Signature when you can check ≥1 box. Each box maps to a §1 trigger:

[ ] LENGTH      prompt string > ~50 lines
[ ] CONSUMED    LM output is parsed / branched on / stored by code (not just human-read)
[ ] REUSED      same prompt called from > 1 site, or inside a loop
[ ] (bonus) about to model-swap, OR a metric already exists, OR blob mixes instruction+demos+format

Zero boxes → do not promote. One box → promote. (Source: §1 triggers; R1 S1, S6, S7.)

4.2 Operation table

#	Trigger	Action	Output	Evidence
OP-1	Prompt crosses a §1 threshold	Run the §4.1 checklist	promote / keep-prose decision	R1 S1, S7
OP-2	Decision = promote	Extract variable inputs + distinct outputs into named slots	Flat list of typed slots	R1 S2
OP-3	Slots listed	Rename each to a semantic, parameter-style name	Field names that read as intent	R1 S2
OP-4	Names set	Add desc= only to underspecified fields	Minimal descriptions	R1 S3
OP-5	Output has fixed value set	Type the output field (Literal[...] / bool / int) instead of str	Typed OutputField	R1 S3, S5
OP-6	Reasoning would help quality	Note "needs CoT" but defer module choice to [[dspy]]	Hand-off note	R1 S4 (module table is dspy's)
OP-7	Optimizer plateaus later	Loop back: re-audit Signature for ambiguity before blaming optimizer	Revised Signature	R1 S8
OP-8	Output consumed by code AND must be machine-valid	Pair the Signature with a grammar/JSON enforcer (Outlines) — Signature shapes intent, enforcer guarantees syntax	Signature + enforcement layer	R1 S9

4.3 Field-naming rules (the heart of this skill)

1. Name like a function parameter, not like a prompt. customer_email, not the text the user pasted.
2. Inputs are nouns the LM receives; outputs are nouns the LM produces. Don't smuggle an output into an input name.
3. One concept per field. "category_and_reason" is two fields. Split it (OP-2).
4. Prefer the most specific type the value can hold. Literal["bug","billing","other"] over str when the set is closed (OP-5) — the type is documentation and a parse guard.
5. Reserve desc= for what the name can't say: format, length, units, allowed values, edge-case handling.

4.4 When InputField vs OutputField descriptions matter

Side	Add a desc when…	Skip the desc when…
InputField	The input has a non-obvious format/source ("raw OCR text, may contain noise"), or the LM tends to misread which input is which	The field name fully explains it (question, ticket) — the cheatsheet leaves question bare (R1 S3)
OutputField	You need to constrain the value: length ("≤15 words"), format ("ISO-8601 date"), or allowed set	The output type already constrains it (e.g. Literal[...] or bool carries the spec)

Rule of thumb: input descs prevent confusion; output descs prevent malformed values. Default to fewer descs; add one only when you can name the specific failure it prevents.

---

5. 困境决策案例 (Dilemma cases)

Case A — "The 80-line mega-prompt: promote whole, or split first?"

困境: A support-triage prompt is 80 lines: persona + 4 categories with examples + output-format spec + edge-case rules. The output ("category, confidence, escalate?") is parsed by routing code. Promote it verbatim into one Signature, or restructure first?

约束:

• Output is code-consumed (routing branches on category and escalate) → §1 CONSUMED trigger fires.
• The 80 lines mix instruction + demos + format → the optimizer should own most of that text, not you.
• Three distinct return values are currently scraped from one free-text blob.

决策步骤:

1. Promote — the gate is satisfied (CONSUMED + LENGTH). This is exactly the load-bearing case (R1 S1).
2. Do NOT copy the 80 lines into one giant docstring. Inputs = ticket: str. Outputs = category: Literal[...], confidence: float, escalate: bool (OP-2, OP-5). The four category descriptions and examples are demos/instructions the optimizer will own — drop them from your code (mental model #2, R1 S5).
3. Type the outputs so the router stops regex-scraping (OP-5). escalate: bool replaces parsing the word "yes" out of prose.
4. Add a desc only on confidence ("0–1, calibrated") since the name underspecifies the range (OP-4, §4.4).
5. Hand to [[dspy]] to pick ChainOfThought (reasoning helps category choice) and to compile against the existing routing-accuracy metric.

结果: An 80-line blob collapses to a 5-field typed contract; the router drops all string-scraping; the prose that was the prompt becomes optimizer-owned instructions/demos.

可提取的操作: When promoting a mega-prompt, keep only the I/O shape in code; let the instruction/demo prose become the optimizer's territory. Split fused outputs; type closed-set and boolean outputs.

---

Case B — "One-shot prompt someone wants to 'make robust' — promote or refuse?"

困境: A teammate has a 6-line prompt that runs once in a migration script ("classify these 200 rows once, then we throw the script away") and asks you to "make it a proper Signature so it's robust."

约束:

• Runs once, script is disposable → §1 LENGTH, REUSED both fail.
• Output is consumed by code (it writes a column) → CONSUMED fires.
• No metric, no model-swap planned, signature won't be reused.

决策步骤:

1. CONSUMED fires, so the gate technically passes — but weigh it. The output is consumed, yet the contract has no second reader over time (disposable script). This is the boundary the [[agentsop-dspy]] skill flags: compile only after the I/O contract stabilizes and will be reused (R1 S7).
2. Compromise: declare a minimal Signature for the type-safety of the one column (OP-5 — a Literal output stops bad values landing in the DB), but do not optimize/compile it. A typed dspy.Predict(Sig) with no compile is cheap and gives the parse guard without the compile-loop overhead.
3. Refuse the "robust"/optimize ask. Optimizing a one-shot, no-metric prompt is the documented anti-pattern — DSPy without a metric is just verbose prompting (R1 S10). Say so explicitly.

结果: A 10-line typed Signature with no compile: enough to make the written column type-safe, not enough to waste a compile budget on a script that's about to be deleted.

可提取的操作: CONSUMED alone justifies a typed Signature (parse safety) but NOT optimization. Separate "promote to typed contract" from "compile/optimize" — they have different gates.

---

6. 反模式与边界 (Anti-patterns & boundaries)

Anti-patterns

1. Over-signaturizing one-off prompts. A Signature (let alone a compiled one) for a 5-line throwaway prompt is pure boilerplate. If no §1 trigger fires, the prose prompt is the correct artifact (R1 S7).
2. Under-specifying field semantics. class Sig: input: str; output: str defeats the entire point — the optimizer gets zero intent signal and code still can't trust the output shape. Field names ARE the contract (R1 S2). Generic names are the most common silent failure.
3. One blob output that should be N fields. Returning a single result: str and regex-scraping three values out of it re-creates the fragility you were escaping. Split into typed fields (OP-2, OP-5).
4. Copying the whole prose prompt into the docstring. The instruction/demo text is the optimizer's to own; freezing it in your Signature both bloats your code and fights the compiler (mental model #2, R1 S5).
5. Promoting before the I/O contract is stable. If you're still adding/removing outputs daily, you're churning boilerplate. Stabilize first (R1 S7).
6. Adding desc= to every field reflexively. Descriptions you can't tie to a specific prevented failure are noise that bloats the prompt; the cheatsheet leaves obvious inputs bare (R1 S3).
7. Confusing "promote to Signature" with "guarantee valid JSON". A typed OutputField pushes toward structure but does not enforce grammar — pair with Outlines/Guidance when machine-validity is mandatory (R1 S9, OP-8).

Boundaries (when this skill is NOT the right layer)

• HOW to write the class / pick a module / compile → [[dspy]] (library) and [[agentsop-dspy]] (workflow). This skill stops at "the Signature is shaped."
• Free-form human-read prose with no parsing and no reuse → keep raw prompting; a Signature buys nothing.
• Token-level format guarantees (strict JSON/regex/grammar) → that's the generation layer (Outlines, Guidance, LMQL), orthogonal to Signature design (R1 S9).
• Non-DSPy stacks → the decision ("is this prompt load-bearing enough to deserve a typed contract?") transfers, but the implementation does not (see §7 for the cross-framework mapping).

---

7. 跨框架对照 (Cross-framework comparison)

The decision ("promote prose → typed contract when the prompt becomes load-bearing") is framework-agnostic. Only the artifact differs. This overlay's rubric (§4) tells you when to reach for any column below.

Approach	What the "contract" is	Optimizable?	Enforces output syntax?	Best when
Raw prompt string	None — narration only	No	No	One-shot, human-read, unstable, no reuse (§6 boundary)
DSPy Signature	Named + typed I/O fields; field names carry intent for the optimizer (R1 S2)	Yes — instructions/demos rewritten on compile, field shape preserved (R1 S5)	Pushes toward structure, no hard guarantee (R1 S9)	Load-bearing prompt + a metric exists / model-swap planned. Implementation: [[dspy]]
Pydantic output model	A typed schema the response is validated against after generation	No (it's validation, not prompt-tuning)	Yes — validation raises on mismatch	You need a hard post-hoc type check but are not optimizing the prompt
instructor (Pydantic + LLM)	Pydantic model used both as prompt scaffold and parse target; auto-retries on validation failure	No prompt-optimization loop; retries only	Yes — re-asks the LM until the schema validates	You want structured-output-with-retries on a raw provider SDK, no compile pipeline

How they compose (not mutually exclusive):

• DSPy Signature + Outlines/Guidance: Signature shapes intent and gets optimized; Outlines guarantees the output is valid JSON/grammar at the token level (R1 S9, OP-8).
• DSPy Signature ≈ instructor's Pydantic model at the "declare the I/O shape" step — but DSPy adds the optimizer that instructor lacks, and instructor adds validation-retry that a bare Signature lacks. Pick DSPy when you have a metric and want to compile; pick instructor when you just need structured output + retries on a raw SDK.
• Pydantic is the validation primitive the others build on; reach for it directly when you only need a hard type gate with no prompt machinery.

Bottom line: this skill decides whether the prompt deserves a typed contract. If yes and you're in DSPy with a metric → the contract is a Signature, and you continue in [[dspy]] / [[agentsop-dspy]]. If you only need validation → Pydantic/instructor. If the prompt isn't load-bearing → no contract at all.

---

Cross-skill links

• [[dspy]] — the library skill: Signature/Module/optimizer API, how to actually write and compile. This overlay defers ALL implementation to it.
• [[agentsop-dspy]] — the full DSPy operating workflow (program → evaluate → optimize, 3-stage gate, optimizer selection, cost guardrails). This overlay is the narrow "should this prose become a Signature" slice of its Stage 1.

Source basis

All claims tagged S1–S10 are sourced verbatim in references/R1-source-evidence.md, drawn from the local dspy-sop-skill/SKILL.md Signatures material and the upstream DSPy docs it cites ([dspy.ai/learn/programming/signatures/], [dspy.ai/cheatsheet/], [arxiv.org/abs/2310.03714]).