prompt-engineering

name	prompt-engineering
description	Design, version, and optimize LLM prompts — few-shot examples, chain-of-thought, role framing, constraint injection, A/B testing, and prompt version management. Use when asked to "write a system prompt", "improve this prompt", "prompt isn't working", "add few-shot examples", "prompt A/B test", or "how do I get better outputs from the model". Do NOT use for: RAG retrieval design — use `rag-architect`. UI for streaming responses — use `llm-ui-patterns`.
origin	yamtam-original
license	MIT © 2025 Vũ Văn Tâm
version	1.0.0
compatibility	Any LLM API. Patterns apply to Claude, GPT-4, Gemini, Mistral.

When to Use

• Use when: designing or iterating a system prompt
• Use when: model outputs are inconsistent, verbose, or off-target
• Use when: adding few-shot examples to improve output format
• Use when: managing multiple prompt versions across environments
• Do NOT use for: retrieval/RAG problems — those are a chunking/search issue
• Do NOT use for: model selection trade-offs — that is model-routing-strategy

---

Prompt Anatomy

A complete prompt has these layers (in order):

1. Role framing      — who the model is and its constraints
2. Task definition   — what to do, output format
3. Context injection — user-specific data, retrieved chunks
4. Few-shot examples — 2–5 input/output pairs
5. Chain-of-thought  — "think step by step" or explicit reasoning scaffold
6. Constraints       — what NOT to do (negative space)
7. Output format     — JSON schema, Markdown structure, word limit

Not every prompt needs all 7 — omit layers that add noise.

---

Core Techniques

Role framing

System: You are a senior software engineer reviewing pull requests.
        You give precise, actionable feedback. You do not praise trivially.
        You flag security issues before style issues.

• Be specific about expertise domain, not just "expert"
• Add behavioral constraints ("do not", "always", "never") — models follow negations well
• Keep role consistent — don't contradict it in user turns

Few-shot examples

# 2–5 examples optimal. More doesn't always help, can confuse.
Input: "Summarize this legal clause in plain English."
Output: "The contractor must deliver within 30 days or pay a 5% penalty."

Input: "Summarize this legal clause in plain English."
Output: [your actual example output here]

• Examples must cover the full output format, including edge cases
• Negative examples (what NOT to output) can be more powerful than positive ones

Chain-of-thought (CoT)

# Zero-shot CoT — works on reasoning tasks
"Think step by step before giving your final answer."

# Explicit scaffold — for complex multi-step tasks
"First, identify the problem type. Second, list relevant constraints.
 Third, propose a solution. Finally, check for edge cases."

CoT significantly improves accuracy on math, logic, and multi-step tasks. Not needed for simple classification or extraction.

Constraint injection

Constraints:
- Output must be valid JSON matching this schema: {...}
- Do not include explanations outside the JSON object
- If information is missing, use null — do not hallucinate
- Maximum 3 items in the "recommendations" array

Explicit constraints reduce hallucination and format drift.

---

Output Format Control

JSON output

Output a JSON object with exactly these fields:
{
  "summary": string (max 100 words),
  "sentiment": "positive" | "negative" | "neutral",
  "confidence": float (0.0–1.0)
}
Do not include any text outside the JSON object.

Markdown structure

Use this exact structure:
## Finding
[one paragraph]
## Impact
[one paragraph]
## Recommendation
[bulleted list, max 3 items]

Word/token budget

Answer in exactly 2–3 sentences. No more.

Models respect explicit word/sentence limits better than vague "be concise".

---

Prompt Versioning

Every prompt used in production must be versioned:

prompts/
  review-pr/
    v1.md          ← initial
    v2.md          ← added CoT
    v3.md          ← added JSON constraint
    current → v3   ← symlink or env var

Version commit message format:

prompt(review-pr): v2 — add chain-of-thought scaffold
Reason: v1 missed multi-file refactors (hit rate 0.61 → 0.84 with CoT)
Eval: tested on 50 PR samples, measured format compliance + accuracy

---

A/B Testing Prompts

Minimum viable prompt A/B test:

1. Write variant A and variant B (change one variable only)
2. Run both on same 50-item eval set
3. Measure: format compliance, accuracy, latency, token cost
4. Promote winner; document why loser lost

Never ship a prompt change without an eval baseline. "Felt better" is not evidence.

---

Common Failure Patterns

Symptom	Cause	Fix
Model ignores format instruction	Instruction buried at end	Move format spec to top, before task
Outputs vary wildly	No examples	Add 2–3 few-shot examples
Model adds unwanted preamble	No negative constraint	Add "Do not explain — output only the result"
Model hedges excessively	Role too generic	Tighten role: "You are confident and direct"
Hallucination	No grounding	Add "Only use information from the provided context"
Instruction following degrades	Prompt too long (> 2K tokens)	Split into system + user; move examples to user

---

Anti-Fake-Pass Rules

Before claiming a prompt is production-ready, you MUST show:

• Prompt tested on ≥ 20 representative inputs (not just 1 example)
• Output format validated (JSON parsed, Markdown structure checked)
• Version documented (v1, v2...) with reason for each change
• Failure cases identified and addressed (at least 2 edge cases tested)
• Token cost estimated for p50 / p95 input size

Reference: gates/anti-fake-pass-gate.md