prompt-optimization

# Prompt Optimization ## Description A universal skill for optimizing prompts through systematic analysis, iterative refinement, and evidence-based improvement. Applicable to all agents and task types. ## Activation Keywords - prompt optimization - optimize prompt - improve prompt - prompt refinement - prompt tuning - prompt enhancement - 优化 prompt ## Recommended Model - **sonnet4.5** (Recommended for balanced reasoning and speed) - **opus4.5** (For complex multi-step optimization) ## Tools Used - exec: Run prompt tests and evaluations - read: Analyze existing prompts and context - write: Create optimized prompts and documentation ## Usage Patterns ### General Optimization ``` 优化这个 prompt: [your prompt] ``` ### Specific Technique Application ``` 使用 [CoT/ToT/Few-shot] 优化这个 prompt ``` ### Targeted Optimization ``` 优化这个 prompt 以提高 [accuracy/consistency/clarity] ``` ## Instructions for Agents ### Overview Prompt optimization follows a systematic workflow: ``` ┌─────────────────────────────────────────────────────────┐ │ Prompt Optimization Workflow │ ├─────────────────────────────────────────────────────────┤ │ │ │ ┌─────────────────┐ ┌─────────────────┐ │ │ │ Analyze │──────▶│ Diagnose │ │ │ │ (分析现状) │ │ (诊断问题) │ │ │ └─────────────────┘ └─────────────────┘ │ │ │ │ │ │ ▼ ▼ │ │ ┌─────────────────┐ ┌─────────────────┐ │ │ │ Refine │──────▶│ Test │ │ │ │ (优化改进) │ │ (测试验证) │ │ │ └─────────────────┘ └─────────────────┘ │ │ │ │ │ │ └──────────┬───────────────┘ │ │ ▼ │ │ ┌─────────────────────────────────────────────┐ │ │ │ Iterate & Document │ │ │ │ (迭代并记录结果) │ │ │ └─────────────────────────────────────────────┘ │ │ │ └─────────────────────────────────────────────────────────┘ ``` ### Step 1: Analyze Current Prompt **Goal:** Understand the prompt's current state and context **Analysis Checklist:** 1. **Intent Clarity** ```markdown - What is the main task? - Is the objective clear? - Are there ambiguities? - Could it be misunderstood? ``` 2. **Structure Assessment** ```markdown - Is the prompt well-organized? - Are instructions logically ordered? - Is there unnecessary complexity? - Is the output format specified? ``` 3. **Content Evaluation** ```markdown - Are there examples? - Are constraints specified? - Is the context sufficient? - Are edge cases addressed? ``` 4. **Performance Issues** ```markdown - Known failure modes - Inconsistency problems - Quality issues - Hallucination risk ``` ### Step 2: Diagnose Problems **Common Issues & Solutions:** | Issue | Symptoms | Solution | |-------|----------|----------| | Unclear Intent | Off-topic outputs | Add explicit task statement | | Missing Context | Inconsistent outputs | Add background information | | No Examples | Variable quality | Add few-shot examples | | Vague Constraints | Hallucination | Add explicit boundaries | | No Output Format | Unstructured output | Specify exact format | | Complex Task | Incomplete outputs | Break into steps | **Diagnosis Process:** ```python def diagnose_prompt(prompt, test_results): """Identify specific issues with a prompt.""" issues = [] # Check clarity if has_ambiguous_terms(prompt): issues.append({ 'type': 'clarity', 'severity': 'high', 'description': 'Ambiguous terms detected', 'recommendation': 'Replace with specific terms' }) # Check structure if not has_clear_structure(prompt): issues.append({ 'type': 'structure', 'severity': 'medium', 'description': 'Poor organization', 'recommendation': 'Reorganize with clear sections' }) # Check examples if not has_examples(prompt) and task_is_complex(prompt): issues.append({ 'type': 'examples', 'severity': 'high', 'description': 'No examples for complex task', 'recommendation': 'Add 2-3 representative examples' }) # Check constraints if has_hallucination_risk(prompt): issues.append({ 'type': 'constraints', 'severity': 'critical', 'description': 'Risk of hallucination', 'recommendation': 'Add fact-checking constraints' }) return issues ``` ### Step 3: Apply Optimization Techniques **Technique Selection Guide:** ``` Task Type → Recommended Technique ───────────────────────────────────────────────────── Complex Reasoning → Chain-of-Thought (CoT) Multi-step Problems → Decomposition Decision Making → Tree-of-Thought (ToT) Pattern Matching → Few-shot Examples Quality Sensitive → Self-Consistency Error-Prone Tasks → Verification Steps Creative Tasks → Style Guidelines ``` #### Technique 1: Chain-of-Thought (CoT) **When to use:** Complex reasoning, math, logic problems **Template:** ``` Let's think step by step: 1. [First step] - Analysis: ... - Reasoning: ... 2. [Second step] - Analysis: ... - Reasoning: ... ... Therefore, the answer is: [Conclusion] ``` **Example Application:** Before: ``` Solve: If John has 5 apples and gives 2 to Mary, then buys 3 more, how many does he have? ``` After: ``` Solve: If John has 5 apples and gives 2 to Mary, then buys 3 more, how many does he have? Let's think step by step: 1. Initial state: John has 5 apples 2. First action: John gives 2 apples to Mary - Remaining: 5 - 2 = 3 apples 3. Second action: John buys 3 more apples - New total: 3 + 3 = 6 apples Therefore, John now has 6 apples. ``` #### Technique 2: Few-Shot Learning **When to use:** Pattern-based tasks, format-sensitive outputs **Template:** ``` Task: [Task description] Examples: 1. Input: [Example 1] Output: [Output 1] 2. Input: [Example 2] Output: [Output 2] 3. Input: [Example 3] Output: [Output 3] Now solve: Input: [New input] Output: ``` **Example Application:** Before: ``` Translate these sentences to formal English. ``` After: ``` Translate these sentences to formal English. Examples: 1. Input: "can you help me?" Output: "Could you please assist me?" 2. Input: "I want to know" Output: "I would like to inquire" 3. Input: "it's not good" Output: "This is unsatisfactory" Now translate: Input: "you need to fix this" Output: ``` #### Technique 3: Tree-of-Thought (ToT) **When to use:** Decision making, multi-path reasoning **Template:** ``` Let's explore multiple approaches: Approach 1: [First approach] - Considerations: ... - Advantages: ... - Disadvantages: ... - Success probability: X% Approach 2: [Second approach] - Considerations: ... - Advantages: ... - Disadvantages: ... - Success probability: X% ... Best approach: [Selected approach] Reasoning: [Why this is optimal] ``` #### Technique 4: Self-Consistency **When to use:** High-stakes accuracy, validation needed **Template:** ``` Generate 3 different solutions: Solution A: [First approach] Solution B: [Second approach] Solution C: [Third approach] Analysis: - Common elements: ... - Differences: ... - Most consistent: ... Final answer: [Most reliable solution] ``` #### Technique 5: Structured Output **When to use:** Format-specific requirements **Template:** ``` Task: [Description] Output format: ``` [Field 1]: [Description] [Field 2]: [Description] [Field 3]: [Description] ``` Constraints: - [Constraint 1] - [Constraint 2] - [Constraint 3] ``` ### Step 4: Test & Validate **Testing Framework:** ```python def test_optimized_prompt(prompt, test_cases): """Test optimized prompt with diverse inputs.""" results = { 'passed': 0, 'failed': 0, 'issues': [] } for test in test_cases: output = run_prompt(prompt, test.input) # Evaluate output evaluation = evaluate_output( output, test.expected, criteria=['accuracy', 'completeness', 'format'] ) if evaluation.passes_threshold(threshold=0.8): results['passed'] += 1 else: results['failed'] += 1 results['issues'].append({ 'input': test.input, 'output': output, 'expected': test.expected, 'evaluation': evaluation }) return { 'success_rate': results['passed'] / len(test_cases), 'details': results } ``` **Test Case Design:** ```markdown ## Test Cases for Prompt Optimization ### Normal Cases - Typical inputs - Expected behavior - Standard complexity ### Edge Cases - Boundary conditions - Unusual inputs - Missing information ### Failure Cases - Known problematic inputs - Ambiguous requests - Conflicting constraints ### Stress Cases - Long inputs - Complex requirements - Multiple constraints ``` ### Step 5: Iterate & Document **Iteration Process:** 1. **Compare Results** ```markdown Version 1 → Version 2 Changes: - Added: [What was added] - Modified: [What was changed] - Removed: [What was removed] Performance Impact: - Accuracy: 75% → 85% (+10%) - Consistency: 70% → 90% (+20%) - Format Compliance: 80% → 95% (+15%) ``` 2. **Document Learnings** ```markdown ## Optimization Log ### Date: [Date] ### Prompt: [Prompt name/version] Issues Identified: - [Issue 1] - [Issue 2] Solutions Applied: - [Solution 1] → Result: [Outcome] - [Solution 2] → Result: [Outcome] Key Learnings: - [Learning 1] - [Learning 2] ``` 3. **Track Versions** ```python class PromptVersion: def __init__(self, prompt, version, changes, metrics): self.prompt = prompt self.version = version self.changes = changes self.metrics = metrics self.timestamp = datetime.now() def is_better_than(self, other_version): return self.metrics['overall'] > other_version.metrics['overall'] ``` ## Optimization Checklist ### Before Optimization - [ ] Understand the task and success criteria - [ ] Identify known issues and failure modes - [ ] Gather test cases - [ ] Set baseline metrics ### During Optimization - [ ] Apply appropriate technique(s) - [ ] Test with diverse inputs - [ ] Measure performance - [ ] Document changes ### After Optimization - [ ] Compare against baseline - [ ] Document learnings - [ ] Archive previous versions - [ ] Plan for future iterations ## Common Optimization Patterns ### Pattern 1: Clarity Enhancement ```python def enhance_clarity(prompt): """Improve prompt clarity.""" # Add explicit task statement if not has_task_statement(prompt): prompt = add_task_statement(prompt) # Replace vague terms vague_terms = identify_vague_terms(prompt) for term in vague_terms: specific = get_specific_alternative(term) prompt = prompt.replace(term, specific) # Add structure prompt = add_section_headers(prompt) return prompt ``` ### Pattern 2: Example Augmentation ```python def add_examples(prompt, task_type): """Add relevant examples to prompt.""" examples = generate_examples(task_type, count=3) example_section = "\nExamples:\n" for i, example in enumerate(examples, 1): example_section += f"\n{i}. Input: {example.input}\n" example_section += f" Output: {example.output}\n" return insert_before(prompt, "Now solve:", example_section) ``` ### Pattern 3: Constraint Addition ```python def add_constraints(prompt, risk_areas): """Add constraints to prevent issues.""" constraints = [] if 'hallucination' in risk_areas: constraints.append("- Only use provided information") constraints.append("- State if information is unknown") if 'format' in risk_areas: constraints.append("- Follow exact output format specified") if 'safety' in risk_areas: constraints.append("- Refuse harmful requests")