automatic-stateful-prompt-improver

license	Apache-2.0
name	automatic-stateful-prompt-improver
description	Automatically intercepts and optimizes prompts using the prompt-learning MCP server. Learns from performance over time via embedding-indexed history. Uses APE, OPRO, DSPy patterns. Activate on "optimize prompt", "improve this prompt", "prompt engineering", or ANY complex task request. Requires prompt-learning MCP server. NOT for simple questions (just answer them), NOT for direct commands (just execute them), NOT for conversational responses (no optimization needed).
allowed-tools	mcp__prompt-learning__optimize_prompt,mcp__prompt-learning__retrieve_prompts,mcp__prompt-learning__record_feedback,mcp__prompt-learning__suggest_improvements,mcp__prompt-learning__get_analytics,mcp__SequentialThinking__sequentialthinking
category	AI & Machine Learning
tags	["prompt-engineering","optimization","stateful","improvement","llm"]
pairs-with	[{"skill":"skill-coach","reason":"Optimize skill prompts systematically"},{"skill":"skill-logger","reason":"Track prompt performance over time"}]

Automatic Stateful Prompt Improver

DECISION POINTS

PROMPT ASSESSMENT:
├── Simple question/command (what, when, how)
│   └── Skip optimization → Answer directly
├── Complex task (multi-step, reasoning, technical)
│   ├── Token budget < 1000
│   │   └── APE: 3-5 iterations
│   ├── Token budget 1000-5000
│   │   └── OPRO: 5-10 iterations
│   └── Token budget > 5000
│       └── DSPy compilation: 10-20 iterations
└── Reusable template/system prompt
    └── Full optimization with historical retrieval

OPTIMIZATION TECHNIQUE SELECTION:
├── Instruction rewriting needed
│   └── Use APE (Automatic Prompt Engineer)
├── Parameter tuning with constraints  
│   └── Use OPRO (Optimization by PROmpting)
├── Complex pipeline with multiple modules
│   └── Use DSPy compilation patterns
└── Unknown/exploratory domain
    └── Hybrid APE→OPRO→DSPy cascade

ITERATION CONTROL:
├── Improvement < 1% for 3 rounds → STOP
├── Quality score > 0.95 → STOP  
├── Max iterations reached → STOP
├── User satisfaction confirmed → STOP
└── Continue → Next iteration

FEEDBACK INTEGRATION:
├── Task successful (user confirms/metrics good)
│   └── Record positive feedback + embed for retrieval
├── Task failed/poor quality
│   └── Record negative feedback + analyze failure mode
└── Unclear outcome
    └── Ask user for explicit feedback before recording

FAILURE MODES

Over-Optimization Spiral

• SYMPTOM: Prompt grows to 500+ tokens with many nested constraints
• DIAGNOSIS: Chasing diminishing returns instead of stopping at "good enough"
• FIX: Apply 80/20 rule - if improvement drops below 5% per iteration, stop

Template Obsession

• SYMPTOM: Spending iterations on formatting/structure vs. task clarity
• DIAGNOSIS: Confusing presentation with performance
• FIX: Measure actual task success, not template conformity

Historical Overfitting

• SYMPTOM: Optimized prompt works for past examples but fails on new inputs
• DIAGNOSIS: Training on too narrow a dataset from retrieval
• FIX: Include diverse examples in optimization, test on held-out cases

Capability Misjudgment

• SYMPTOM: Adding extensive scaffolding for tasks model handles natively
• DIAGNOSIS: Assuming model limitations without testing
• FIX: Test baseline capability before heavy prompting

Measurement Blindness

• SYMPTOM: Multiple iterations without clear success metrics
• DIAGNOSIS: Optimizing without knowing what "better" means
• FIX: Define measurable success criteria in first step

WORKED EXAMPLES