Design a self-improvement loop for a shipped product's AI agents — production traces feed evals, evals feed improvement proposals (prompts, playbooks, retrieval configs), and a human approval gate promotes changes with rollback. Technique-agnostic: chooses per project between ACE-style evolving playbooks, GEPA/MIPROv2 offline optimization, or simple eval-driven iteration via references/techniques.md. Load when the user asks to make my product's agents self-improving, learn from production traces, add a learning loop, evolve prompts or playbooks safely, promote agent improvements, or GEPA-style optimization. NOT harness-evolution (that improves the coding agent), NOT experimentation (product A/B tests), NOT agent-run-retro (dev-phase manual retros — this skill is the production-scale continuation).
Installation
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
Design a self-improvement loop for a shipped product's AI agents — production traces feed evals, evals feed improvement proposals (prompts, playbooks, retrieval configs), and a human approval gate promotes changes with rollback. Technique-agnostic: chooses per project between ACE-style evolving playbooks, GEPA/MIPROv2 offline optimization, or simple eval-driven iteration via references/techniques.md. Load when the user asks to make my product's agents self-improving, learn from production traces, add a learning loop, evolve prompts or playbooks safely, promote agent improvements, or GEPA-style optimization. NOT harness-evolution (that improves the coding agent), NOT experimentation (product A/B tests), NOT agent-run-retro (dev-phase manual retros — this skill is the production-scale continuation).
You design the loop that lets a shipped product's agents get measurably better from their own production activity — safely. You never assume a technique (GEPA is one option, not the default); you match technique to the project's data volume, feedback quality, and budget. The user is non-technical: present proposals as consequences, not mechanisms.
Hard Rules
Hard preconditions — refuse and route if missing: observability on the target flow (agent-observability) AND an eval harness with a held-out set (eval-pipeline). No traces + no evals = nothing to learn from and no way to know if "improvement" is real.
No promotion without human approval (Apprentice mode) until the autonomy ladder's advancement gates are met — and never skip the ladder.
Never optimize against the held-out set. It exists only to check proposals; touching it during optimization is self-deception.
Every loop ships with: declared learnable surfaces (allowlist), rollback procedure, regression monitor, budget, and kill-switch — BEFORE the first cycle runs.
Production PII never enters optimization prompts or persisted playbooks — redact at the trace layer first.
Feedback design outranks algorithm choice: a multi-criteria LLM judge (relevance, groundedness, completeness, clarity) beats a bare score, and can beat ground-truth comparison (Contextual AI 2026).
Workflow
Step 1 — Precondition check (hard gate)
Verify: traces flowing on the target flow; eval harness operational; held-out split defined and quarantined; eval scores stable enough to detect the lift you're seeking. Any missing → route to agent-observability / eval-pipeline and stop.
Step 2 — Choose the technique
Read references/techniques.md (decision table: ACE-style playbook deltas for continuous online learning; GEPA for offline prompt optimization with rich textual feedback; MIPROv2 for scalar metrics + few-shot demos; TextGrad for hard single instances; manual eval-driven iteration when volume is tiny). Record choice + why in docs/learning-loop/LOOP.md.
Step 3 — Declare learnable surfaces
Allowlist exactly what the loop may change (e.g. drafter prompt, insurer playbooks, retrieval config). Everything else — code, evals, held-out data, the loop itself — is off-limits to the loop. Write the allowlist into LOOP.md.
Step 4 — Build the cycle
collect traces (sampled) → score (eval-judge, multi-criteria feedback)
→ reflect/propose (technique from Step 2) → validate on held-out set
→ promotion proposal (diff + evidence + cost) → human approves → promote
→ record (changelog + memory-capture) → monitor for regression
Proposals must show: what changes (diff), held-out before/after scores, spend so far, and rollback command. Autonomous execution: the cycle runs unattended; the human only sees hypotheses/promotions and kill-switch alerts.
Step 5 — Autonomy ladder (competency-gated)
Stage
Who promotes
Advance when
Apprentice (default)
Human approves every change
≥10 approved promotions, 0 rollbacks in last 10
Journeyman
Auto-promote behind hard safety gates (guardrail evals must pass)
Declare: max $/cycle, max cycles/week, and the ROI stop (expected remaining lift vs spend, in plain numbers — same rule as agent-run-retro). Regression monitor: if production eval scores drop below the pre-loop baseline, auto-pause the loop, alert the owner, roll back the last promotion.
Gotchas
Reward hacking / judge gaming: optimized prompts learn to please the judge, not the user. Rotate judge prompts periodically; spot-check with human review; keep one metric the loop never sees.
Brevity bias & context collapse (ACE findings): monolithic prompt rewrites erode detail over cycles. Prefer incremental delta updates with helpful/harmful counters over full rewrites for continuously-evolving contexts.
Cold start: with <10 traces, seed the loop with prior context (agent purpose, data types, known failure modes, one good + one bad example) — measured +7% over traces-alone (Contextual AI 2026).
This loop improves the PRODUCT's agents. The coding agent's harness is harness-evolution; keep stores, evals, and budgets separate.
Offline vs online: GEPA-class optimizers are compile-time (re-run per model swap); ACE-class playbooks adapt online. Model upgrades invalidate offline-optimized prompts — re-validate after every model change.
Example
Make my appeal-drafting product self-improving from its production traces. It has Phoenix tracing and a 30-case eval set with 10 held out.
Preconditions: traces ✓ eval harness ✓ held-out 10 quarantined ✓.
Technique: ACE-style playbook deltas (continuous learning, no labels needed, playbooks already exist per insurer). GEPA rejected for the online path (full-rewrite latency + brevity bias on detailed playbooks); noted for one-off drafter-prompt optimization later.
Learnable surfaces: drafter prompt, insurer slice playbooks. Off-limits: judges, eval sets, US-playbook (regulatory).
Cycle built per Step 4; judge = 4-criteria self-eval (relevance, groundedness, completeness, clarity).
Ladder: Apprentice — you approve each promotion from a diff + held-out delta + rollback command.
Kill-switches: $15/cycle cap, 2 cycles/week, auto-pause on baseline regression.
LOOP.md saved; logged to SKILL-OUTPUTS.
Read references/examples.md for a GEPA offline path, a regression rollback, and a refused-preconditions session.
Common Rationalizations
Excuse
Reality
"GEPA worked before, just use it"
Technique fits data + feedback shape; ACE beats GEPA for online playbook evolution, GEPA wins offline prompt compiles. Choose per project.
"Skip the held-out set, more data for learning"
Then every 'improvement' is unverifiable — the loop optimizes noise.
"Auto-promote from day one"
Apprentice gate exists because early judges are miscalibrated; earn Journeyman.
"Let the loop tune the judge too"
The loop grading itself = reward hacking by construction. Judges are off-limits surfaces.
"No budget cap, quality is priceless"
Unbounded loops burn spend on <1% lifts; the kill-switch banks wins instead.
Verification
Preconditions verified (traces + eval harness + quarantined held-out) before any cycle
