Manage token budgets, progressive disclosure, context window optimization, summarization strategies, dual-representation compilation (human-readable vs agent-optimized), structured context pruning, attention budget allocation, context retention policies across multi-turn conversations, state ledger design, and working memory vs long-term context separation. Use when maximizing agent performance under context window constraints, designing token-efficient skill instructions, implementing progressive disclosure pipelines, or optimizing multi-turn agent conversations. Handles token budget analysis, context pruning rules, summarization quality validation, and dual-representation compilation. Do NOT use for general text summarization, document compression for human readers, or optimizing non-AI text processing.
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
A direct command skips the review prompt. Inspect the source before running it.
Manage token budgets, progressive disclosure, context window optimization, summarization strategies, dual-representation compilation (human-readable vs agent-optimized), structured context pruning, attention budget allocation, context retention policies across multi-turn conversations, state ledger design, and working memory vs long-term context separation. Use when maximizing agent performance under context window constraints, designing token-efficient skill instructions, implementing progressive disclosure pipelines, or optimizing multi-turn agent conversations. Handles token budget analysis, context pruning rules, summarization quality validation, and dual-representation compilation. Do NOT use for general text summarization, document compression for human readers, or optimizing non-AI text processing.
author
Sandeep Kumar Penchala
license
MIT
portability
works with Claude Code, Copilot CLI, Cursor, OpenClaw, Gemini CLI
Portability target: Spec-level (runs on Claude Code, Copilot CLI, Cursor, OpenClaw, Gemini CLI). No vendor-specific frontmatter fields.
RESEARCH_PREREQUISITE — Execute Before Any Output
This is a HARD GATE. Do not produce ANY output, code, strategy, design, or recommendation without completing this research.
Before you act, you MUST execute every applicable research step. Research-before-acting is the difference between professional work and amateur guessing:
#
Research Step
Why It Matters
Where to Look
RP1
Verify domain currency. Check for breaking changes, deprecations, new standards, or version shifts since the knowledge cutoff.
[STALE_RISK] Outdated advice breaks real systems. API deprecations, framework version bumps, and security advisory changes happen continuously. Outputting based on stale knowledge damages credibility and produces broken results.
Official docs, changelogs, GitHub releases, RFC tracker
RP2
Audit the system or codebase. Read relevant files. Understand existing patterns, constraints, and architecture before proposing changes.
[CONTEXT_VIOLATION] Solutions that ignore existing patterns create technical debt. A change that contradicts the established architecture is worse than no change — it introduces inconsistency that compounds over time.
Cross-reference claims against authoritative sources. Every factual assertion needs a verifiable source. Mark each: [VERIFIED], [COMPUTED], or [ESTIMATED].
[HALLUCINATION_GUARD] Claims without sources are indistinguishable from hallucinations. The #1 cause of incorrect output is treating assumptions as facts. Source tagging prevents this.
Official documentation, peer-reviewed papers, RFCs, specifications
RP4
Identify known failure modes. Before recommending, list what commonly breaks. For each failure mode: trigger condition, detection signal, and mitigation.
[FAILURE_BLINDNESS] Every domain has known failure patterns. Output that doesn't address them is dangerously incomplete. If you cannot name 3+ failure modes for your recommendation, you don't understand it well enough to recommend it.
Quantify impact in concrete units. Replace abstract claims ("faster," "better," "more scalable") with exact numbers, even if estimated.
[VAGUENESS_PENALTY] "Faster" is unverifiable. "Reduces p95 latency from 340ms to 120ms (±15ms)" is verifiable. Abstract adjectives hide ignorance behind confidence. Concrete numbers expose gaps.
Benchmarks, production metrics, pricing data, published performance data
RP6
Map side effects and downstream impacts. What else breaks? Which dependencies are affected? Which downstream consumers need updating?
[CASCADE_BLINDNESS] Changes to one component ripple outward. A fix in module A can break module B that depends on A's old behavior. Map the blast radius before acting.
Dependency graph, cross-skill coordination table, API consumers list
RP7
Verify against non-negotiable quality gates. What are the minimum quality bars for this domain (accessibility, security, performance, accuracy, compliance)?
[QUALITY_FLOOR] Every domain has minimum standards below which output is invalid regardless of functionality. Missing WCAG AA = broken. Leaking credentials = broken. Silent data loss = broken.
Declare explicit limitations and edge cases. What does this NOT handle? What are the known boundaries? What scenarios are explicitly out of scope?
[SCOPE_HONESTY] Declaring limitations is a feature, not an admission of weakness. It prevents misuse, sets correct expectations, and demonstrates true understanding. Every solution has boundaries — naming them is professional.
This SKILL.md, domain literature, edge case databases
If you skip any of these research steps, you are not producing quality output — you are guessing with confidence. Guessing wastes time, breaks systems, and destroys trust. The references, ground rules, and decision trees in this skill exist specifically to prevent guessing. Use them.
Compliance: Research must be executed before any substantial output. For each step, document findings inline in your response using [RESEARCHED] marker: [RESEARCHED: RP1 — Domain verified against changelog v2.4. No breaking changes since cutoff.]. Partial research = partial quality. Zero research = zero credibility.
🔄 Iterative Research Loop — Research at EVERY Decision Point, Not Just Entry
The RP1-RP8 cycle above is NOT a one-time gate. It fires continuously at every material decision point throughout the workflow:
Loop
When It Fires
What Re-research Validates
Loop 0: Pre-Action
Before producing ANY output, code, strategy, or recommendation
At every adjustment, phase transition, scale-out, or significant state change
Has the context changed? Are the original assumptions still valid? Has new information invalidated the Loop 0 conclusions?
Loop 2: Pre-Exit
Before closing, handing off, escalating, or declaring completion
Is the deliverable complete by the quality gates defined in RP7? Are all limitations declared (RP8)? Have failure modes been addressed (RP4)?
Loop 3: Post-Action
After completion: compare expected vs. actual outcome
What was the efficiency ratio (actual / theoretical max)? What learnings emerged? What should be fed back into the pattern database for future decisions?
Integration into Core Workflow:
Every decision point in a skill's Core Workflow must be marked with:
[RESEARCH LOOP: Re-execute RP1-RP8 before proceeding to next phase]
This ensures the agent pauses to re-verify ALL research dimensions before making the next decision. A skill that only researches at entry and then operates on auto-pilot is a skill that makes decisions on stale context.
Markers for output: At each loop, the agent outputs: [RESEARCHED: Loop N — RP1-RP8 re-verified. Key delta from previous loop: ...]
Why this matters: A decision made in Loop 0 may be catastrophically wrong by Loop 2 because the context changed. Markets move. Requirements shift. Dependencies update. The research loop catches context drift before it becomes output error.
Compliance: Research must be executed before any substantial output AND re-executed at every decision point. For each research loop, document findings inline. Partial research = partial quality. Zero research = zero credibility. Stale research = dangerous confidence.
Anti-Hallucination
Rationalization
Reality
"The model has a 200K context window — we don't need compaction."
Context window size is irrelevant to attention quality. Models attend effectively to ~70% of context. Beyond that, signal-to-noise degrades. A 200K window with 180K of noise is worse than a 10K window with 9K of signal. Compaction is about attention quality, not capacity.
"Progressive disclosure adds complexity — just load the full skill."
Loading a 4,500-token skill for a 150-token task burns 4,350 tokens of attention budget on irrelevant content. Across 500 invocations/day, that's 2.1M wasted tokens/day — $6.30/day at $3/M input. Progressive disclosure cuts this to 950 tokens average: $4,650/year saved per skill.
"Dual-representation compilation will introduce errors."
Done correctly, the minified form is semantically equivalent — it's a format transform, not summarization. XML tags like <constraint>NEVER skip validation</constraint> carry identical semantic payload to the markdown "NEVER skip validation." Validation via eval suite catches any regression before deployment.
"Context rotation defense is overengineering — we'll notice redundancy."
Redundancy is invisible without tooling. An agent repeating the same ground rule across 5 turns adds 0 information and consumes 5x the attention budget. Sentence embedding deduplication catches this automatically. Humans cannot track token-level redundancy across 20+ turns.
"We'll just summarize when we hit the limit."
Reactive summarization at 95% window saturation means the summarizer runs on a nearly-full context, produces lower-quality output, and the agent already suffered 15 turns of diluted attention. Proactive compaction at 70% yields better summaries and cleaner context.
Ground Rules — Read Before Anything Else
#
Negative Constraint
Mechanical Trigger
Violation Response
1
NEVER load Tier 3 content without exhausting Tier 1 + Tier 2 first
RELOCATE guardrails to first 200 tokens (primacy zone)
R1
ANCHOR to runtime versions before generating framework-specific code. Never generate Fastify/Express/Django/FastAPI/Prisma/SQLAlchemy API calls from training data alone — your training data may be stale.
Trigger: skill receives code-generation task involving framework-specific APIs → run scripts/runtime-version-detect.sh [project-root] --skill-context to detect installed versions → if detection succeeds, anchor all API calls to detected versions → if detection fails, request version info from user
STOP. Respond: "Detected: {runtime}@{version}, {frameworks}@{versions}. Anchoring all API calls to these versions. I will add // VERIFY: comments on any API call where the detected version is newer than my training cutoff."
R2
RUN the ROI Gate before any non-emergency code change. Every code change that is not (a) a security fix, (b) a compliance requirement, or (c) an active production incident must pass scripts/roi-gate.sh. If the gate returns negative, refuse to write the code.
Trigger: skill receives a code-generation or refactoring task that is NOT a security fix, compliance requirement, or production incident → estimate implementation cost in engineer-hours → compare against annual value of the change → if cost > value, gate fails
STOP. Respond: "ROI Gate analysis: This change costs approximately $[X] to implement but saves $[Y]/year. Payback period: [N] years. If payback > 2 years, I recommend declining this work. See scripts/roi-gate.sh for the full formula."
Admit uncertainty — never fabricate. If you're not certain about an API method, package version, configuration syntax, or command flag, say so explicitly: "I'm not certain this API exists in the latest version. Check the official docs at [URL]." Never invent a function signature or configuration key because it "seems right." Hallucinated code costs hours of debugging.
Flag your knowledge cutoff. If your training data predates the latest SDK release, framework version, or platform change, state your cutoff date and recommend verifying against current documentation. This is especially critical for rapidly evolving domains: cloud IAM policies, JS framework APIs, mobile OS capabilities, and SaaS pricing — all change quarterly or faster.
Never guess security configurations. If you're unsure about the correct CSP header value, OAuth flow parameter, or encryption algorithm choice, do NOT provide a "reasonable default." Say: "Security configurations must be verified against current best practices at [official source]. I cannot provide a definitive answer without current documentation."
Distinguish between what you know and what you infer. Explicitly mark statements as: [VERIFIED] — from official docs, [COMMON-PRACTICE] — widely used but not authoritative, [INFERRED] — your best guess based on patterns, [UNKNOWN] — you're unsure. This helps the user calibrate trust in your output.
The Expert's Mindset
Context compaction is the art of maximizing decision quality per token. You are not compressing text — you are curating attention. Three principles govern everything:
1. Attention is the scarcest resource. A model's effective attention bandwidth is ~70% of its context window. Every token beyond that is invisible or dilutes focus on what matters. Treat each token as an attention allocation decision: "Does this token earn its place by improving the agent's next decision?"
2. Information has a half-life. A ground rule read at turn 1 is only 60% as likely to be followed at turn 15 (recency-weighted attention decay, λ=0.1 per turn). Repetition is not redundancy — it's attention renewal. The 12 context rotation defense patterns exist because stale context is as dangerous as missing context.
3. Format is fungible; semantics are sacred. Markdown is a human convention. XML and JSON-LD are machine conventions. The information payload is identical. Dual-representation compilation achieves 35-50% token reduction without semantic loss because it strips presentation, not meaning.
Cognitive biases to guard against:
Completeness bias — "Include everything just in case" (the fastest path to attention dilution)
Recency anchoring — over-weighting the last error vs. the systemic cause
Format attachment — believing markdown carries information that XML doesn't
Operating at Different Levels
Quick Scan (~30s)
Check context window saturation. If > 85%, trigger Tier 3 eviction. Verify no skill conflicts (2+ active skills sharing domain keywords). Run: python context_dashboard.py --quick
Standard Engagement (~5min)
Full compaction audit: redundancy detection (0.92 threshold), staleness scoring by last-access, attention zone verification (guardrails in primacy zone, output format in recency zone), unproductive-loop check, dual-representation validation. Run: python context_dashboard.py --standard
Deep Dive (~30min)
Architecture review of the entire compaction pipeline. Includes: progressive disclosure tier calibration with real workload traces, token budget optimization across a 5-skill pipeline, dual-representation compiler refinement for new skill types, and attention allocation modeling with exponential decay simulation across 50-turn conversations.
Do NOT use this skill when the problem is not compaction:
Trigger
Route Instead
Why
Prompt phrasing, instruction tuning
llm-engineer
You are editing prompts, not compacting context
Minimizing the cost of an EXISTING payload end-to-end
context-optimizer
The optimizer owns the lever ladder (measure → reduce → cache → compress → cap)
Token pricing, cost math, cache economics
token-efficiency
Cost modeling is that skill's domain
Context hierarchy and structure design
context-engineering
Structure design, not compaction
Fine-tuning or RLHF
ml-ai-engineer
Model training, not context
If your task is compaction algorithms — pruning rules, dual-representation compilation, summarization, attention budgets — this is the right skill. If it is cost optimization of a payload, pricing math, or prompt phrasing — hand off.
"My skill is too large" → Start at "Progressive Disclosure Architecture" (Core Workflow, Step 1)
"Agent forgets instructions mid-conversation" → Start at "Context Rotation Defense Patterns" (Decision Tree 4)
"Token costs are too high" → Start at "Token Budget Management" (Decision Tree 1)
"Agent loops on the same error" → Start at "Unproductive-Loop Detection" (Decision Tree 6)
"Building a skill compiler" → Start at "Dual-Representation Compilation" (Core Workflow, Step 5)
"Multiple skills conflicting" → Start at "Context Fragmentation Prevention" (Decision Tree 5)
Anti-Rationalization (QUICK)
AR-01 No Unbounded Skills: You CANNOT let a skill instruction set exceed its declared budget. "It's only 500 more tokens" is how progressive disclosure collapses and every invocation pays full price.
AR-02 No Uniform Pruning: You CANNOT prune every section by the same percentage. "Everyone gives a little" is how 2 critical ground rules get dropped while 5 verbose examples survive. Priority-based eviction only.
AR-03 No Reactive Compaction: You CANNOT wait until 95% saturation to compact. "We'll summarize when we hit the limit" is how the summarizer runs on a degraded, nearly-full context. Compact proactively at 70%.
AR-04 No Compressing Security Constraints: You CANNOT lossy-compact security or compliance content. "The gist is enough" is how "NEVER store passwords in plaintext" becomes "use secure auth" and the agent picks MD5. Format-only compaction for L1.
AR-05 No Mid-Turn Compaction: You CANNOT compact during active generation. "The agent can still answer" is how pruned references produce corrupted output. Compact at turn boundaries after a state-ledger checkpoint.
AR-06 No Handoff Without a Missing-Skill Check: You CANNOT route a handoff to a role whose skill does not exist in this library. If the target skill is missing, create it autonomously before handing off.
Core Workflow
(STANDARD)
Step 1: Progressive Disclosure Architecture
Design the three-tier loading system for every skill:
Reference-heavy skills: 50-55% reduction (links compressed to inline summaries)
Step 6: Attention Placement Optimization
Position content for maximum model attention:
POSITION EFFECT CONTENT TYPE REASON
──────────────────────────────────────────────────────────────────
First 200 Primacy Ground Rules, "NEVER" Model attends most
tokens constraints, safety strongly to opening
guardrails content
Middle 25-75% Lost-in-middle NONE — avoid placing Model is 20-40% less
anything critical here likely to attend to
mid-context content
Last 100 Recency Output format spec, Model's final context
tokens required structure, shapes its response
next-step instruction most directly
Tier 2/3 Lazy-loaded Detailed examples, Only loaded when
references gotchas, reference docs specifically needed
Step 7: Skill Creation on Handoff — Fill Missing-Skill Gaps Autonomously
[RESEARCH LOOP: Re-execute RP1-RP8 — is the handoff target real, current, and truly missing from the library?]
When a downstream task requires a skill that does not exist in this library, do NOT degrade the handoff. Create the skill autonomously, then hand off.
#
Action
Verify
1
Detect the gap. The handoff target role/domain has no matching skill in skills/.
grep -rl "name: <target>" skills/ returns nothing; no >80% description-similar neighbor found
2
Duplicate check. Search for near-duplicates by name and description before creating. If an equivalent exists, extend it instead.
Validate. Run lint-template.py, lint-yaml.py, lint-markdown.py, and bash scripts/validate-skills.sh.
All gates pass; skill registers in the router
7
Hand off. Invoke the new skill's workflow for the original task, and record the creation in the State Log.
Downstream task completes using the created skill; State Log entry documents the gap + creation
Creation boundary: Only create a skill when (a) the task genuinely recurs or is consequential, (b) no existing skill covers it, and (c) you can fill it to the 10/10 bar. For one-off, low-stakes gaps, record the gap in the State Log and route to the nearest existing skill instead — creating a half-quality skill is worse than routing.
Handoff: Deliver the compacted skill (or the new skill) to the consuming skill via cross-agent-skills-packaging conventions, and confirm the downstream skill's consumes_from includes this skill so the graph stays symmetric.
Decision Trees
(QUICK)
Decision Tree 1: Token Budget Saturation Response
Phase 1 — Gather:
Model context window size (tokens)
Current token count
Active skills with tier level
Conversation turn count
Saturation % = current_tokens / window_size
Phase 2 — Decide:
Saturation level?
├── < 70% → HEALTHY — no action needed
│
├── 70-84% → WARNING
│ ├── Run redundancy detection (Pattern 1, 0.92 threshold)
│ ├── Score staleness (Pattern 2); flag sections > 5 turns unreferenced
│ ├── Prepare Tier 3 eviction candidates (lowest priority first)
│ └── Log: "Prepared {N} sections for eviction if saturation reaches 85%"
│
├── 85-94% → CRITICAL
│ ├── EVICT all Tier 3 content; downgrade to Tier 2 representation
│ ├── Compress conversation history (Pattern 3, λ=0.1 decay)
│ ├── Check for unproductive loops (Pattern 4, > 3 identical cycles)
│ ├── Downgrade > 3 active full skills to Tier 1 (Pattern 9)
│ └── Re-check saturation; if still > 85%, proceed to next level
│
└── ≥ 95% → OVERFLOW — EMERGENCY
├── EVICT all Tier 3 AND Tier 2 content; Tier 1 only
├── Emergency compression: conversation → 1-sentence summary per 5 turns
├── Remove all but 1 example per concept
├── Drop deliberate practice, references, verification sections
└── Log complete eviction manifest for recovery
Decision Tree 2: Progressive Disclosure Tier Assignment
Usage frequency of each section (from agent telemetry)
Criticality: which sections are MUST-HAVE vs NICE-TO-HAVE
Phase 2 — Decide:
For each section in skill:
├── Section is "Route the Request" or headline description?
│ └── TIER 1 (150 tokens) — always loaded for intent matching
│
├── Section is "Ground Rules" or "Decision Trees"?
│ └── TIER 2 (800 tokens total) — loaded on intent match
│ ├── Ground rules: full constraint table
│ ├── Decision trees: complete branch logic
│ └── Core workflow steps: numbered procedure
│
├── Section is "Anti-Patterns", "Examples", "Verification", "Deliberate Practice"?
│ └── TIER 3 (lazy-loaded) — only on explicit branch traversal
│ ├── Anti-Patterns: full table with costs
│ ├── Examples: loaded when agent uncertainty detected
│ └── References: loaded when decision tree references specific file
│
└── Section is "Anti-Rationalization" or "Expert's Mindset"?
└── TIER 3 — motivational/contextual, not operational
Decision Tree 3: Pruning Algorithm Selection
Content type to prune?
├── Conversation history (turns 1-N)
│ └── ALGORITHM: Summarization-based (LLM compression to 20%)
│ ├── Keep: decisions made, facts discovered, state changes, blockers
│ ├── Drop: intermediate reasoning, resolved errors, tool call details
│ └── Format: "Turn [N]: [Action] → [Outcome]. Decision: [X]."
│
├── Source code / implementation files
│ └── ALGORITHM: Rule-based truncation
│ ├── NEVER summarize code — paraphrased code is harmful
│ ├── Replace with: `file:path@hash` reference
│ └── If must include: function signatures + docstrings only
│
├── Skill reference files
│ └── ALGORITHM: Embedding-based retrieval (cosine > 0.85)
│ ├── Embed current task description
│ ├── Retrieve top-N reference sections above similarity threshold
│ └── Budget-fill: include until token budget exhausted
│
├── Ground rules / security constraints
│ └── ALGORITHM: NONE — NEVER prune
│ ├── These are the safety net; loss = potential catastrophe
│ └── If must reduce: compress format only (dual-representation), not content
│
└── Examples and illustrations
└── ALGORITHM: Rule-based truncation by priority
├── Priority 1: keep 1 canonical example per concept
├── Priority 2+: drop; note "N examples available at [ref]"
└── If 0 examples remain: flag for human review
Decision Tree 4: Context Rotation Defense Activation
What pattern is triggered?
├── Redundancy detected? (Pattern 1, similarity > 0.92)
│ └── DEDUPLICATE: keep latest, drop older, log duplicate
│
├── Staleness detected? (Pattern 2, > 5 turns unreferenced)
│ └── DECAY: apply 0.5x relevance multiplier; evict if score < 0.3
│
├── Recency decay needed? (Pattern 3, every turn boundary)
│ └── RE-RANK: apply e^(-0.1 * turns_since_ref) to all section weights
│
├── Unproductive loop? (Pattern 4, > 3 identical attempts)
│ └── HALT + ESCALATE: stop agent, inject triage context, log incident
│
├── Guardrail in mid-context? (Pattern 5, position 25-75%)
│ └── RELOCATE: move to first 200 tokens (primacy zone)
│
├── Output format not in recency zone? (Pattern 6)
│ └── APPEND: add format spec to last 100 tokens
│
├── Skill conflict? (Pattern 8, > 3 shared keywords)
│ └── DOWNGRADE: lower-priority skill → Tier 1; log conflict
│
├── > 3 concurrent full skills? (Pattern 9)
│ └── EVICT: lowest-priority skill → Tier 1
│
├── Context fragmented? (Pattern 11, > 5 skill sources)
│ └── CONSOLIDATE: keep top 3; downgrade rest
│
└── Major decision completed? (Pattern 12)
└── CHECKPOINT: serialize to state ledger; prune decision context
Decision Tree 5: Context Fragmentation Prevention
Phase 1 — Gather:
Active skills list with tier levels
Domain keywords per skill
Current attention allocation (% of context per skill)
Agent's current task description
Phase 2 — Decide:
How many skills have Tier 2+ content loaded?
├── 1-3 → HEALTHY — no fragmentation risk
│
├── 4-5 → MODERATE RISK
│ ├── Check for keyword overlap (Pattern 8)
│ │ ├── Overlap detected → Downgrade lower-priority skill to Tier 1
│ │ └── No overlap → Monitor but allow
│ └── Enforce attention slot limit (Pattern 9)
│
└── > 5 → HIGH RISK — FRAGMENTATION
├── Consolidate: keep top 3 skills at Tier 2+
├── Downgrade remaining skills to Tier 1 (headline only)
├── Log fragmented skills for post-hoc analysis
└── Re-check saturation; may trigger eviction (Decision Tree 1)
Attention distribution check:
├── Any single skill > 40% of context?
│ └── Potential over-focus — verify skill relevance to current task
│
└── All skills < 10% of context each?
└── Context too fragmented — agent lacks depth on any single domain
Decision Tree 6: Unproductive-Loop Diagnosis
Agent appears stuck in loop
├── STEP 1: Hash last 5 (action, outcome) pairs
│
├── STEP 2: Count identical pairs
│ ├── 3+ identical → UNPRODUCTIVE LOOP CONFIRMED
│ │ ├── HALT agent immediately
│ │ ├── Inject escalation context:
│ │ │ "You have attempted the same action 3+ times with the same
│ │ │ outcome. The previous attempts were: [list]. The outcome
│ │ │ each time was: [outcome]. STOP. Consider: (a) is there a
│ │ │ different approach? (b) is the goal achievable? (c) do you
│ │ │ need more context?"
│ │ ├── Log incident with full loop trace
│ │ └── If loop continues after escalation → require human triage
│ │
│ └── < 3 identical → Continue monitoring
│
├─ STEP 3: Root cause analysis
│ ├── Missing context? → Lazy-load Tier 3 references
│ ├── Conflicting ground rules? → Check for skill conflict (Pattern 8)
│ ├── Attention dilution? → Check saturation; evict if > 85%
│ └── Stale context? → Rotate via recency-weighted re-rank (Pattern 3)
│
└─ STEP 4: Prevention
├── Set loop detection threshold per task complexity
├── Simple tasks: > 2 identical = halt
├── Standard tasks: > 3 identical = halt
└── Complex tasks: > 4 identical = halt (allow more exploration)
Error Decoder
Error Message / Situation
Root Cause
Fix
Lesson
"Agent forgot the architecture decision from turn 5 and proposes contradictory design on turn 18"
The decision was compacted out during context rotation without a state ledger entry. The agent's working memory has no record of the binding choice.
Log every pruned decision to the state ledger with a unique key. Before proposing architectural changes, the agent must check the ledger for prior decisions in the same domain.
Context rotation without a state ledger is amnesia. The ledger is the agent's long-term memory — compaction without it is irreversible information loss.
"Compacted skill produces different behavior than original — agent fails tasks it previously passed"
Semantic drift in dual-representation compilation. A negation was dropped during minification: "Do NOT use for production" became "Use for production."
Run behavioral equivalence validation at >= 95% threshold. Test negations and constraints specifically — these are the most fragile during compilation. Add a negation-preservation check to the compiler pipeline.
Format transformation is not semantically neutral. Negations, constraints, and exceptions are the first things to break during compaction.
"Agent repeats the same warning across 5 consecutive turns — context is 30% redundant"
Redundancy detection failed. The agent is retrieving the same skill instruction repeatedly because it appears in multiple skill fragments. The assembler doesn't deduplicate before context assembly.
Enable pre-assembly redundancy detection at 0.92 threshold. Cache deduplication results per skill fragment. If the same content appears in 2+ fragments, retain only the canonical copy with a cross-reference.
Redundancy is invisible to humans but devastating to attention. Sentence embedding dedup catches what manual review cannot.
"Token budget shows 40% over target but compaction isn't triggering"
Compaction threshold is set too high (95%). The pipeline is running at 90% saturation — well above the recommended 70% proactive compaction threshold but below the emergency trigger.
Lower proactive compaction trigger to 70%. At 70%, compact Tier 3 content to headlines. At 85%, evict all Tier 3, compact Tier 2 to summaries. At 95%, emergency eviction — all non-critical content removed.
Proactive compaction at 70% produces better summaries than reactive compaction at 95%. The summarizer works better on less-saturated context.
"Information density scoring marks a critical security constraint as low-priority"
The density scorer weights by token count, not semantic impact. "Use AES-256-GCM" is 4 tokens but carries more security weight than a 200-token code example.
Add a content category classifier BEFORE density scoring. Security constraints, auth patterns, and ground rules get an automatic priority boost regardless of token count. Density scoring applies only within the same category.
Token count ≠ information value. The shortest sentences are often the most important. Classify content by category before scoring by density.
"Sliding window evicts the architecture decision from turn 3 but keeps verbose error logs from turns 7-10"
Pure recency-based eviction: the oldest context is dropped regardless of importance. The architecture decision on turn 3 is still binding; the error logs on turns 7-10 are obsolete.
Replace recency-only eviction with importance-weighted eviction. Score each segment: importance (is this decision still binding?) × recency (how old?) × uniqueness (is this info available elsewhere?). Evict lowest-scored segments — which may be recent but unimportant.
Recency is not relevance. A binding decision from turn 3 matters more than a resolved error from turn 9.
Error Recovery
(STANDARD)
If a command or approach fails, follow this escalation path before giving up:
Symptom
First Action
If That Fails
Last Resort
Tool/command not found
Check installation: which [tool] or [tool] --version. Install via package manager (brew install, npm install -g, pip install)
Check PATH: echo $PATH. Verify the tool binary is in a PATH directory. Symlink or update PATH if installed but unreachable
Use a functionally equivalent alternative tool. If rg is unavailable, use grep -r. If gh is unavailable, use git directly or the GitHub API via curl
Permission denied
Check ownership: ls -la [path]. Fix with chmod or sudo if appropriate. For API errors (401/403), verify credentials haven't expired: echo $TOKEN or check ~/.netrc
Refresh credentials: re-authenticate with the service. For file permissions, check if the file is locked by another process: lsof [path]
Request elevated permissions or use a different authentication method (token vs password, SSH key vs HTTPS)
Command hangs or times out
Kill the process: Ctrl+C. Re-run with a timeout: timeout 30 [command] or gtimeout on macOS. Check system resources: top, df -h, netstat -an
Add verbose/debug flags: --verbose, --debug, -v. Check logs: tail -f [logfile]. Reduce scope: process fewer files, query a smaller time range, limit concurrency
Split the work into smaller batches. Implement a retry loop with exponential backoff (1s, 2s, 4s, 8s). If the issue is network-related, add --retry 3 or equivalent
Unexpected output or error message
Read the error message completely — the solution is often in the last 3 lines. Search the exact error: grep -r "[error text]" in the repo to find prior occurrences
Check GitHub issues for the tool: gh issue list --repo owner/repo --search "[error keyword]". Check Stack Overflow
Simplify the approach. If the complex one-liner fails, break it into 3 sequential commands. If the specialized tool fails, use a more basic tool with more steps
Data integrity concern (wrong output, silent failure)
Verify with a manual check: compare output against a known-correct baseline. Add assertions: `[command]
grep -q "[expected]" && echo "OK"
Hard failure boundary: If 3 different approaches all fail, STOP. Do not iterate infinitely. Log what was tried, capture the error output, and report the blocking issue with full context. Move to the next independent task rather than blocking all progress on one failure.
Cross-Skill Coordination
Scenario
Coordinate With
Handoff
Skill instruction design for token efficiency
llm-engineer
Token budget, tier architecture, prompt placement within skill
Multi-agent pipeline context distribution
system-architect
Context partitioning strategy, handoff state schema
Agent-to-agent state serialization
agent-handoff-protocol
State ledger format, decision checkpointing, recovery paths
Validating compaction doesn't degrade behavior
agent-eval-pipeline
Behavioral equivalence test suite, regression detection
Platform-level context infrastructure
platform-engineer
Compiler pipeline, token monitoring, eviction policies as platform service
Handoff protocol: When delegating context compaction work that intersects another skill, include: (1) the current token budget and saturation %, (2) the active skill manifest with tier levels, (3) the compaction strategy selected and rationale, (4) the recovery path for pruned content.
Upstream Skill
What You Receive
When to Involve
system-architect
System context, integration patterns, deployment constraints
Before designing AI/ML pipelines
mlops-engineer
Model lifecycle, deployment patterns, monitoring requirements
Validate semantic equivalence; reject if behavioral match < 95%
Conversation exceeds 20 turns without summarization
python turn_monitor.py --threshold 20
Force conversation summarization to 20% token density; archive to state ledger
What Good Looks Like
✅ Good — Token Budget Management:
"Monitored 5-skill pipeline at 68% saturation. Proactive redundancy detection identified 3 duplicate ground rule sections (12% token waste). Deduplicated to canonical versions. Evicted 2 stale Tier 3 gotchas unreferenced for 8+ turns. Result: 62% saturation with no decision-quality loss. Projected savings: $1,240/month."
✅ Good — Dual-Representation Compilation:
"Compiled security-reviewer skill: 4,800 tokens markdown → 2,640 tokens minified XML (45% reduction). Frontmatter → JSON-LD, decision trees → nested <branch> elements, gotchas → JSON array. Eval suite: 98.2% behavioral match across 75 test scenarios. Deploying to production pipeline."
✅ Good — Unproductive Loop Resolution:
"Detected loop at turn 14: agent repeatedly attempted same SQL migration fix (3 identical cycles). Halted agent. Injected escalation context with alternative approaches. Agent pivoted to schema redesign approach. Resolved in 2 additional turns vs. projected 8+ turns without intervention. Saved ~$0.45 in loop tokens."
❌ Bad — Reactive Truncation:
"Agent hit 97% saturation so I truncated the oldest 40% of context. Unfortunately that included the database schema the agent needed for the current task. Agent generated queries against wrong column names. 3 hours of debugging." [[Violates Ground Rule #4 — never evict without priority scoring]]
❌ Bad — Lossy Security Summarization:
"Summarized the auth module's security constraints to save tokens. Summary said 'use secure auth' instead of the original 'NEVER store passwords in plaintext; MUST use bcrypt with cost factor ≥ 12.' Agent implemented MD5 hashing because 'secure auth' was ambiguous. Security regression in production." [[Violates Ground Rule #3 — never compact security-critical sections]]
❌ Bad — Missing State Ledger:
"Compacted context at turn 18 without checkpointing. Pruned 3 architecture decisions made at turns 5-8. At turn 22, agent needed to know why microservice A communicates via gRPC not REST. Decision was pruned, rationale lost. Agent proposed REST migration. $15K rework to revert." [[Violates Pattern 12 — always checkpoint before pruning decisions]]
Deliberate Practice
Token Budget Calibration Drill: Take a real 10-skill agent pipeline trace. Calculate actual token consumption per skill vs. budgeted. Identify the top 3 budget overruns. Recalibrate tier boundaries. Target: all skills within 15% of budget.
Dual-Representation Compilation Exercise: Take a 5,000-token skill. Manually compile to minified XML: extract frontmatter → JSON-LD, decision trees → <branch> elements, gotchas → JSON array. Measure token reduction. Validate semantic equivalence by having another agent execute both versions on 10 test tasks. Target: ≥ 40% reduction with ≥ 95% behavioral match.
Redundancy Detection Calibration: Run sentence embedding deduplication on a 50-turn agent conversation. Tune the similarity threshold (start at 0.92). Find the sweet spot: maximize deduplication without removing semantically distinct content. Validate: no unique decisions lost.
Attention Zone Audit: Take a compiled context. Check: (a) are all "NEVER" constraints in first 200 tokens? (b) is output format in last 100 tokens? (c) is any critical content in positions 25-75%? Fix placement violations. Measure decision quality before/after.
Loop Simulation Exercise: Create an agent task that intentionally triggers an unproductive loop (e.g., circular dependency in config). Observe: how many cycles before detection? Does the escalation context break the loop? Tune the detection threshold. Document the optimal halt point.
Eviction Recovery Drill: Take a heavily compacted context (post-95% eviction). Simulate the agent needing pruned information. Can the agent recover from the state ledger? From file references? From conversation summary? Identify unrecoverable information. Strengthen the checkpoint strategy.
Cross-Skill Fragmentation Test: Load 8 skills simultaneously. Observe attention distribution. Detect keyword conflicts. Apply fragmentation prevention (Pattern 11). Measure: does consolidating to top 3 skills improve decision quality on the primary task?
Best Practices
Use structured summarization for decisions, not free-form prose. "We chose PostgreSQL" is ambiguous. "Decision: database=PostgreSQL 16; rationale: team expertise + JSONB support; constraints: GDPR EU-only; reversible: true" is recoverable. Structured summaries enable automated context reconstruction; prose requires re-reading.
Implement sliding window with importance scoring, not simple recency truncation. Dropping the oldest N turns loses early decisions that are still active. Score each context segment by: recency (how old?), relevance (does current task reference this?), decision impact (was a binding choice made here?), and uniqueness (is this information available elsewhere?). Evict lowest-scored segments first.
Prioritize information density over token count. A 20-token decision record ("Use AES-256-GCM with random IV") carries more information than a 200-token prose explanation of encryption choices. When compacting, prefer dense formats (tables, key-value pairs, structured records) over narrative prose. Every token should earn its place by improving decision quality.
Balance lossiness tradeoffs explicitly per content category. Security constraints: lossless (verbatim preservation). Architecture decisions: near-lossless (structured summaries with all key parameters). Code examples: lossy (reference by file:hash, not inline). Prose explanations: highly lossy (compress to one-sentence summaries). Never apply uniform compression — the cost of losing a security constraint is 1000x the cost of losing a verbose example.
Use dual-representation compilation for skills, not for conversations. Skills are static — compile markdown → XML/JSON-LD once, validate semantic equivalence, deploy. Conversations are dynamic — use progressive summarization with turn-boundary compaction. Compiling live conversations introduces latency and risks misrepresenting evolving context.
Measure information density as decisions per token, not tokens per turn. A 500-token turn with 0 decisions is waste. A 50-token turn with 1 binding architectural decision is gold. Track decision density over time — if it's declining, your compaction is preserving noise at the expense of signal.
Implement attention zone placement as a formal step in context assembly. Primacy zone (first 200 tokens): ground rules, security constraints, task definition. Recency zone (last 100 tokens): output format, current step instruction. Mid-context (25%-75%): reference material, examples, supporting context. Critical guardrails in mid-context are invisible to attention — relocate or repeat them.
Use redundancy detection as a pre-assembly step, not a post-hoc audit. Before assembling context for a new turn, detect and remove duplicate information across all candidate segments. Sentence embedding similarity at threshold 0.92 catches near-duplicates. This prevents the common failure mode of repeating the same ground rule across 5 turns while consuming 5x the attention budget.
Design compaction logs for recoverability, not just audit. When you compact a decision out of context, log: what was removed, why, when it was last accessed, and a recovery path (state ledger key, file reference, or checkpoint number). An agent that needs the pruned information should be able to recover it without replaying the entire conversation.
Test compaction with behavioral equivalence suites, not token count checks. A compaction that reduces tokens by 40% but drops a negation ("Do NOT use X" → "Use X") is a regression. Run the same 50-scenario eval suite against the compacted and original context. Pass threshold: >= 95% behavioral equivalence on all dimensions. Token savings without behavioral equivalence is compression, not compaction.
Production Checklist
(STANDARD)
Before deploying any context compaction pipeline to production, verify ALL of:
CR1: Progressive disclosure tiers defined for all active skills — Verification: Tier 1 (overview), Tier 2 (decision trees), Tier 3 (full gotchas/examples) mapped per skill
CR2: Context saturation monitoring active — Verification: alerts at 70% (warn), 85% (Tier 3 eviction trigger), 95% (emergency eviction)
CR3: Redundancy detection running pre-assembly — Verification: sentence embedding dedup at 0.92 threshold, zero duplicates in assembled context
CR4: Attention zones verified — Verification: guardrails in primacy zone (first 200 tokens), output format in recency zone (last 100 tokens)
CR5: No security-critical sections compacted — Verification: all "NEVER," "MUST NOT," security, and auth constraints preserved verbatim
CR6: Dual-representation compilation validated — Verification: behavioral equivalence >= 95% against original markdown, eval suite passing
CR7: Compaction logged with recovery path — Verification: what was removed, why, when, and how to recover recorded for every compaction event
CR8: Unproductive loop detection active — Verification: < 3 identical (action, outcome) pairs in any 10-turn window, escalation context injected at halt
CR9: Skill conflict detection active — Verification: no two active skills share > 3 domain keywords, or namespace prefixing mitigates
CR10: State ledger populated — Verification: all pruned decisions have a recovery path recorded, ledger integrity verified
CR11: Context rotation defense tested — Verification: all 12 defense patterns validated against multi-turn conversation simulation
CR12: Token budget per skill declared and monitored — Verification: actual usage tracked against budget, alerts on > 20% variance
CR13: Compaction metadata recorded per event — Verification: timestamp, tokens before/after, method used, segments affected, recoverability status
CR14: Recovery drill completed — Verification: simulate need for pruned information, verify successful recovery from ledger or file reference
Missing state ledger for pruned decisions: No record of what was removed or why — agent cannot recover context when needed
Irreversible information loss, dead-end agent requiring full restart
$20K-$100K in lost context and rework
5
False equivalence in dual-representation: Minified version dropped a negation — "Do NOT use for X" became "Use for X" — agent applies skill to wrong domain
Agent produces garbage output in wrong domain, erodes trust
$10K-$100K in misapplied AI and reputation damage
6
Attention dilution from over-caution: Retained 15 edge-case gotchas "just in case" — agent's attention spread so thin it missed the 1 critical constraint
Critical constraint ignored; agent focused on irrelevant edge cases
$50K-$500K in missed primary concern
7
Uniform pruning without priority scoring: Removed 30% of tokens uniformly — lost 2 critical ground rules but kept 5 verbose examples
Tier 3 loaded without Tier 2 grounding: Agent received full gotchas but no decision tree context to understand when they apply — misapplied edge-case warnings to standard path
Agent forgets binding architecture decision from turn 3 because it was compacted out without state ledger entry
$100K-$300K in contradictory design decisions and rework
Log every pruned decision to the state ledger with unique key. Before proposing architectural changes, check the ledger for prior decisions in the same domain. Context rotation without state ledger = amnesia.
Negation dropped during minification — "Do NOT use for production" becomes "Use for production"
$50K-$250K in production incidents from semantically inverted instructions
Run behavioral equivalence validation at ≥ 95% threshold. Test negations and constraints specifically — they are the most fragile during compaction. Add negation-preservation check to the compiler pipeline.
Recency-only eviction drops critical old content while keeping verbose recent logs
$50K-$150K/year in wasted tokens and lost context
Replace recency-only eviction with importance-weighted eviction: importance × recency × uniqueness. A binding decision from turn 3 matters more than a resolved error from turn 9.
Compaction threshold set too high (95%) — pipeline runs at 90% saturation without triggering
$50K-$200K/year in degraded model performance from context overflow
Set proactive compaction trigger at 70% saturation. Compact Tier 3 content at 70%, evict Tier 3 at 85%+, emergency eviction at 95%. Proactive compaction at 70% produces better summaries than reactive compaction at 95%.
Verification
#
Complete when...
Verify
☐
Complete when Token budget per skill declared
All active skills have tier assignments with token budgets
☐
Complete when Context saturation < 70%
Healthy operating band; no eviction needed
☐
Complete when Context saturation < 85%
No Tier 3 eviction triggered; monitor for growth
☐
Complete when Context saturation < 95%
Emergency eviction NOT triggered; Tier 2 content preserved
☐
Complete when Redundancy detection passed
Zero duplicate sentences at 0.92 similarity threshold
☐
Complete when No security sections compacted
All "NEVER", "MUST NOT", security/auth constraints preserved verbatim
☐
Complete when Attention zones correct
Guardrails in primacy zone (first 200 tokens); output format in recency zone (last 100 tokens)
☐
Complete when No unproductive loops
< 3 identical (action, outcome) pairs in any 10-turn window
☐
Complete when ≤ 3 concurrent full-skill loads
Active skills at Tier 2+ do not exceed 3
☐
Complete when No skill conflicts
No two active skills share > 3 domain keywords
☐
Complete when Dual-representation validated
Compiled skill passes behavioral equivalence at ≥ 95%
☐
Complete when State ledger populated
All pruned decisions have recovery path recorded in ledger
☐
Complete when Compaction logged
Metadata recorded: what was removed, why, when, recoverable?
☐
Complete when Recovery tested
Simulated need for pruned information → successfully recovered from ledger or file reference
☐
Complete when Code blocks preserved or referenced
No paraphrased code; all code either verbatim or file:hash reference
Verification Guardrails
Before delivering work, verify: self-check against What Good Looks Like, no broken references, continuity with State Log, no fabricated APIs/versions/capabilities, Error Recovery paths exercised, cross-skill dependencies satisfied. If any fail, revise before delivering.
This section documents every irreversible decision made during the session. It is non-negotiable and prevents the agent from revisiting settled questions.
#
Decision
Rationale
Alternatives Considered
Timestamp
1
[no decisions logged yet]
—
—
—
Rules:
Append a new row for each irreversible or hard-to-reverse decision
Never modify past rows — only append
If revisiting a decision, add a NEW row (do not edit the old one)
