| name | self-refine-reflection |
| description | Systematic self-reflection and iterative output refinement for AI agents. Based on Madaan et al. (2023) Self-Refine, Shinn et al. (2023) Reflexion, and Andrew Ng's Reflection pattern. Use when outputs need multi-round critique-and-improve cycles, when initial quality is insufficient, or when complex tasks benefit from self-correction. |
| license | MIT |
| metadata | {"author":"Thomas Zhou","version":"2.0.0"} |
Self-Refine Reflection Skill
An AI agent skill for systematic self-reflection and iterative output refinement.
Based on Madaan et al. (2023), Shinn et al. (2023), and Andrew Ng's Reflection design pattern.
Platform Auto-Detection
At skill load time, detect your runtime environment and adjust capabilities:
| Capability | How to Check | Fallback |
|---|
| File system | Can you read references/reflection-templates.md? | Use the inline templates below instead |
| Persistent memory | Can you write to memory/? | Store reflection notes in conversation context only |
| Long context | Is your context window > 32K tokens? | Cap at Level 2 (skip adversarial review) |
| Tool access | Can you call external tools? | Use mental verification only |
Detection rules:
- If you can read this file's
references/ directory → full mode (all levels + memory)
- If you can read files but not write → full levels, in-conversation memory only
- If you cannot read files at all → use inline templates (copied below), cap at Level 2
- If context is limited (< 8K usable) → default to Level 1, max Level 2
This means every platform gets the best possible experience automatically — no manual configuration needed.
Inline Reflection Templates (for environments without file access)
If you cannot read references/reflection-templates.md, use these directly:
Level 1 Internal Prompt
Review drafted response: (1) Did I answer everything? (2) Logic gaps? (3) Can I cut 20%? (4) User can act on this? Fix → Deliver.
Level 2 Internal Prompt
For each dimension (Logic / Facts / Completeness / Conciseness / Actionability / Consistency):
- Quote exact sentences with issues
- Rate: Critical / Minor / Pass
Fix ALL Critical. Fix Minor if straightforward. Re-read once. Deliver.
Level 3 Internal Prompt
Round 1: Level 2 review.
Round 2: "If a domain expert attacked this, what would they target?" Fix valid attacks.
Round 3: Did fixes introduce new issues? If stable → deliver. If not → fix and deliver.
When to Activate
Activate when you are about to deliver a final response to the user, after you've gathered all information and formed your answer. Reflection happens before delivery, not instead of work.
Core Loop: GENERATE → CRITIQUE → REFINE → CHECK
1. GENERATE — Produce your initial response as normal
2. CRITIQUE — Apply the reflection framework (depth-dependent)
3. REFINE — Fix every issue found in CRITIQUE
4. CHECK — If issues remain AND within budget, loop to step 2
— If clear OR budget exhausted, deliver
Source: Madaan et al., "Self-Refine: Iterative Refinement with Self-Feedback" (2023) — the same LLM acts as generator, critic, and refiner.
Reflection Dimensions
Every critique examines the response across these dimensions. Not all apply to every response — skip irrelevant ones silently.
| # | Dimension | What to Check | Signal Words |
|---|
| 1 | Logical Completeness | Does the reasoning chain have gaps? Are conclusions supported by premises? | "therefore" without prior evidence, jumps in logic |
| 2 | Factual Accuracy | Are there unverified assertions? Claims that could be wrong? | Specific numbers, dates, names, "everyone knows..." |
| 3 | Response Completeness | Did I address every part of the user's question? Are there sub-questions I ignored? | Multiple questions in user message, implicit needs |
| 4 | Conciseness | Is there redundancy? Can paragraphs be merged? Are there filler phrases? | "It's important to note that", "In conclusion", repeated points |
| 5 | Actionability | Can the user act on this immediately? Or do they need to ask follow-ups? | Vague advice without steps, missing specifics |
| 6 | Internal Consistency | Do any parts of my response contradict each other? | Conflicting recommendations, contradictory statements |
Reflection Depth Levels
The depth is determined by task complexity, not user preference. The agent auto-selects.
Level 0: Quick Scan (Skip formal reflection)
Trigger: Simple factual lookups, greetings, trivial questions, single-sentence answers.
Action: Do nothing extra. Just respond. Cost: 0 additional tokens.
Examples: "What time is it?", "Thanks", simple formatting requests.
Level 1: Standard Review
Trigger: Medium-complexity tasks — explanations, how-to guides, code snippets, multi-paragraph responses.
Action: One pass through all 6 dimensions mentally. Fix issues. Deliver.
Budget: 1 refinement round. ~15-20% overhead on response tokens.
Internal process:
After drafting your response, ask yourself:
- Did I answer everything they asked?
- Any logical gaps or contradictions?
- Can I cut 20% of the words without losing meaning?
- Would the user know exactly what to do next?
Fix → Deliver.
Level 2: Deep Audit
Trigger: Complex tasks — technical architectures, multi-step plans, research summaries, anything with 5+ distinct claims.
Action: Explicit dimension-by-dimension review. One full refinement round with documented issues.
Budget: Up to 2 refinement rounds. ~30% overhead.
Internal process:
Draft response.
For each dimension (1-6):
- Identify specific issues (quote the exact sentence)
- Rate severity: Critical / Minor / Pass
If any Critical: refine entire response, then re-check
If only Minor: fix inline, deliver
Level 3: Adversarial Review
Trigger: High-stakes tasks — production code, security decisions, medical/legal adjacent, public-facing content, or user explicitly requests thorough review.
Action: Full audit PLUS an adversarial pass where you actively try to find the weakest point in your response.
Budget: Up to 3 refinement rounds. ~50% overhead.
Internal process:
Draft response.
Round 1: Standard dimension-by-dimension review.
Round 2: Adversarial pass — "If I wanted to prove this response wrong,
what would I attack?" Check those attack vectors.
Round 3 (if needed): Verify fixes from Round 2 didn't introduce new issues.
Convergence Rules
Based on the empirical finding from Madaan et al. that "most gains are in the initial iterations":
- Maximum 3 refinement rounds regardless of depth level.
- Stop early if no Critical or Minor issues found in a round.
- Stop early if the changes between rounds are purely stylistic (no substantive improvement).
- Diminishing returns rule: If Round N fixes fewer issues than Round N-1, stop after N.
Anti-pattern — DO NOT:
- Refine forever trying to reach perfection
- Make changes just to justify another round
- Re-introduce previously fixed issues
Cost Control Strategy
| Depth | Max Rounds | Approx. Token Overhead | When to Use |
|---|
| Level 0 | 0 | 0% | Simple Q&A |
| Level 1 | 1 | ~15% | Most conversations |
| Level 2 | 2 | ~30% | Complex technical |
| Level 3 | 3 | ~50% | High-stakes only |
Principle: Reflection should cost less than the cost of delivering a wrong answer. For low-stakes responses, skip reflection entirely.
Trigger Conditions Summary
Auto-Trigger (always on)
- Response exceeds 3 paragraphs → at least Level 1
- Response makes 3+ factual claims → at least Level 1
- Response includes code → at least Level 1 (check logic + actionability)
Skip Reflection
- User is in a hurry (explicit: "quick", "brief", "just tell me")
- Response is under 2 sentences
- Pure social/chat exchange
User Manual Trigger
- User says "think carefully", "double-check", "make sure this is right" → Level 2+
- User says "this is important", "critical", "production" → Level 3
- User says "reflect" or "self-refine" → Level 2+
Output Format
Reflection is internal — the user should not see the raw critique. However:
After refinement, you MAY append a subtle note:
Level 1: No note (keep it invisible).
Level 2: Optionally: _(response refined via self-review)_
Level 3: Optionally: _(refined through [N]-round adversarial self-review)_
NEVER:
- Show the actual critique/feedback text to the user
- Make the note prominent or distracting
- Add notes for Level 0 or Level 1 responses
Reflexion Memory (Cross-Session Learning)
Inspired by Shinn et al. (2023) — Reflexion stores verbal self-reflections in persistent memory for future tasks.
When to use: After completing a Level 2 or Level 3 reflection where you discovered a significant pattern in your own errors (e.g., "I tend to forget edge cases in X", "I consistently over-explain Y").
How to use: Write a brief note to memory/ capturing the pattern:
Self-reflection: When writing about [topic], I tend to [pattern].
Fix: [specific behavior change].
Date: [today].
This creates a growing corpus of self-corrections that improve future responses.
Relationship to Other Reflection Techniques
This skill synthesizes multiple academic approaches:
| Technique | Source | What We Borrow |
|---|
| Self-Refine | Madaan et al. 2023 | Core GENERATE→CRITIQUE→REFINE loop |
| Reflexion | Shinn et al. 2023 | Persistent memory of self-reflections |
| Chain-of-Verification | Dhuliawala et al. 2023 | Verification questions for factual claims (Level 2-3) |
| Self-Calibration | Kadavath et al. 2022 | Confidence assessment of own outputs |
| Andrew Ng's Reflection | Ng 2024 | Design pattern: agent critiques and improves its own output iteratively |
| CRITIC | Gou et al. 2023 | Tool-interactive critiquing (use tools to verify when possible) |
Quick Reference Card
REFLECT? ──→ Simple/trivial? ──→ NO → Just respond
│
YES
│
├─ Medium complexity → LEVEL 1 (1 round, mental check)
├─ Complex / multi-claim → LEVEL 2 (2 rounds, explicit review)
└─ High-stakes / user requested → LEVEL 3 (3 rounds, adversarial)
For each round:
1. Check 6 dimensions
2. Fix all issues
3. Converged? → Deliver
4. Budget left? → Next round
5. Budget exhausted → Deliver current version
