Analyse failure and success patterns across many runs of a specific workflow using context-intelligence session data. Use when you want to answer: "How is <workflow> failing?", "What does a successful run look like vs a failing one?", "Which steps are the most common failure points?", or "What patterns appear consistently across sessions?" Triggers on: workflow failure patterns, session failure analysis, compare successful and failing runs, what patterns appear across sessions, session behaviour investigation, how is the workflow failing, success patterns, failure signals, identify session signals.
Analyse failure and success patterns across many runs of a specific workflow using context-intelligence session data. Use when you want to answer: "How is <workflow> failing?", "What does a successful run look like vs a failing one?", "Which steps are the most common failure points?", or "What patterns appear consistently across sessions?" Triggers on: workflow failure patterns, session failure analysis, compare successful and failing runs, what patterns appear across sessions, session behaviour investigation, how is the workflow failing, success patterns, failure signals, identify session signals.
Domain concepts are established. Proceed with the analysis below.
File not found
Phase 0 is incomplete. Delegate to context-intelligence:context-intelligence-design-facilitator with context_depth="none" to run Phase 0 first. Do NOT proceed with any analysis until the facilitator has written domain-concepts.md.
This skill is designed to run AFTER Phase 0 (concept elicitation). Running it before Phase 0
produces poorly scoped signals and forces the user to steer corrective turns.
Systematic investigation of failure and success patterns across many runs of a specific
workflow using context-intelligence session data.
Answers three questions:
What failure patterns appear consistently across many runs?
What does a successful run look like vs a failing one?
Which specific steps or delegations are the most common failure points?
Follows the dual-agent loop established from direct investigation of Amplifier
session corpora:
context-intelligence:graph-analyst — all corpus queries and JSONL inspection
context-intelligence:context-intelligence-design-facilitator — structured artifact
production and quality gating (mandatory gate before any finding is treated as curated)
Produces a findings package in {{output_path}}:
findings.md — failure patterns (P1–Pn) with severity, frequency, mechanism, evidence
domain-signals.md — JSONL-observable signal definitions with corpus-anchored thresholds
session-index.md — full session inventory with signal scores
{{workflow}}: The workflow to analyse — e.g. "amplifier CLI sessions", "git-ops
agent runs", "recipe orchestration", "resolver setup flow". Can be a workspace name,
session characteristic pattern, or a description of the session type.
{{output_path}}: (Optional) Where to write findings artifacts.
Default: .context-intelligence-investigation/
findings/ holds raw analyst output (intermediate). {{output_path}}/ holds curated
artifacts produced by the facilitator.
Then clarify what a "run" means for {{workflow}}:
What session property identifies a run? (workspace slug, duration range, recipe name,
event count range, session characteristics)
Scope: :RootSession only, :ForkedSession only, or both?
Time range: all time or bounded?
Express the scope as an observable session property filter — not as a named-agent filter.
Success criteria: Companion files loaded. Output directories exist. Run definition
expressible as a Cypher WHERE clause or metadata field filter.
2. Enumerate the full corpus before investigating
Before drilling into any specific pattern, build the full corpus inventory.
Delegate to context-intelligence:graph-analyst to:
Find all sessions matching {{workflow}} scope using Q1 from cypher-patterns.md
Split :RootSession from :ForkedSession — failure rates differ structurally
(roots ~73% vs corpus ~55%)
Compute: session count by type, median duration, median event count
Graph vs disk:
Graph server is preferred — Cypher returns cross-session pattern matches as atomic
operations far more efficiently than per-file scanning. Use Q1–Q7 from cypher-patterns.md.
Disk scan is the explicit alternative when the user needs comprehensive coverage of
ALL on-disk sessions regardless of graph indexing state. Disk scan entry:
CONTEXT_INTELLIGENCE_ROOT="${AMPLIFIER_CONTEXT_INTELLIGENCE_BASE_PATH:-$HOME/.amplifier/projects}"# Enumerate captures by the canonical marker (events.jsonl), matching the Python# readers; read the sibling metadata.json for the format/version filter.
find "$CONTEXT_INTELLIGENCE_ROOT" -path "*/context-intelligence/events.jsonl" 2>/dev/null \
| whileread -r ev; doecho"${ev%/events.jsonl}/metadata.json"; done
Filter to format == "context-intelligence" AND version == "1.0.0" (from metadata.json).
Follow @context-intelligence:context/safe-extraction-patterns.md — never load full
llm:request lines.
Validate coverage: compare graph session count against disk count and inform the user
if the graph holds a small fraction. The signal prevalence rates from a graph sample
can still be directionally valid even if the absolute corpus is larger — confirm this
before deciding whether a full disk scan is needed.
Write raw output to findings/session-corpus-raw.md.
Success criteria: Corpus total confirmed (root count, sub count, workspace count).
User is aware of graph vs disk coverage situation.
3. Apply signal set — failing and successful runs
Delegate to context-intelligence:graph-analyst to apply signals S1–S9 from
signals-reference.md to all sessions in scope. Use Q2–Q5 from cypher-patterns.md.
For each signal compute:
Prevalence in failing runs (% of failing sessions where signal fires)
Prevalence in successful runs (% of successful sessions where signal fires)
Delta — the gap between the two rates
The delta is the primary finding. Signals with high delta discriminate between failure
and success for this workflow. Signals that fire equally in both groups are structural
characteristics of the workflow type, not failure indicators.
Classify each run by outcome before scoring:
Successful: status = completed, compound signal score = 0
Degraded: status = completed, compound signal score ≥ 1
Failed: status = cancelled OR (status = running AND stale > 2h), score ≥ 2
Write raw signal scores to findings/session-signal-scores-raw.md.
Success criteria: Per-signal prevalence table with delta column. Top compound-failure
sessions ranked. At least one signal with delta ≥ 20 percentage points between failing
and successful groups.
4. Produce initial structured artifacts
Delegate to context-intelligence:context-intelligence-design-facilitator with the raw
findings from Steps 2–3.
Instruct the facilitator to write:
{{output_path}}/findings.md — patterns (P1–Pn) with severity, frequency, mechanism,
cross-cutting structural findings (Xn), and candidate skill implications (Cn)
{{output_path}}/domain-signals.md — signal definitions calibrated to this workflow
with corpus-anchored thresholds
{{output_path}}/session-index.md — full session inventory with root/sub split,
outcome classification, and compound failure scores
{{output_path}}/diagrams/failure-patterns.dot — taxonomy with severity swimlanes and
prevalence numbers on pattern nodes
Success criteria: All four artifacts written and internally consistent. Corpus numbers
and signal thresholds consistent across all files.
5. Iterative investigation loop
Continue investigating specific patterns as the user steers. Each round follows the
same dual-agent shape — repeat as many times as needed:
a. Delegate to context-intelligence:graph-analyst with the specific question.
Analyst writes raw findings to findings/.
b. Delegate to context-intelligence:context-intelligence-design-facilitator to
update the structured artifacts. Facilitator calls do not need to be 1:1 with analyst
calls — consolidate related findings into a single facilitator update.
Common investigation rounds and their targets:
Signal calibration — when a signal fires at an unexpectedly high or low rate,
inspect actual JSONL event payloads before adjusting any threshold. Key field reference
(from signals-reference.md §Authoritative JSONL field reference):
Hash (tool_name, tool_input) to test for S4c — literal same-input essentially never
fires in stuck sessions. Use S4a (parallel-group shape concentration) and S4b (ritual
instrumentation-bash share) as primary stuck-agent detectors.
Compaction causal analysis — inspect context:compaction event payload (ground truth:
before_tokens, after_tokens, budget, strategy_level). Inspect the last llm:request
before each compaction for context composition — extract character lengths per region,
never render full message content:
Two etiologies: P1a tool-result accumulation (delegation-driven, skill-addressable) vs
P1b thinking-block accumulation (interleaved_thinking=true, model-config issue)
Delegation pattern analysis — inspect delegate tool:post result envelope:
{agent, response, session_id, status}. Measure len(result.response).
Delegation is the gating condition for severe compaction: no root session reaches
S1 (≥3 compactions) from pure file reads alone. Synthesis-output sub-agents
(result > 30K chars) are the compaction multiplier.
Success criteria per round: Raw findings written, structured artifacts refreshed,
signal thresholds validated against corpus evidence rather than assumed.
6. Produce Cypher query set for ongoing monitoring
Delegate to context-intelligence:graph-analyst to produce a reusable, workflow-specific
Cypher query set using templates from cypher-patterns.md:
Q-enum: Enumerate all runs with outcome classification
Q-signals: Detect each high-delta failure signal
Q-compare: Compare signal prevalence between successful and failing runs
Q-risk: Identify in-progress sessions at risk (trajectory)
Q-points: Failure point localisation — events preceding failure signals
Write the query set to {{output_path}}/queries/{{workflow}}-cypher.md.
Success criteria: Working Cypher queries tested against the graph server, scoped to
this workflow's session definition.
7. Close the investigation
Verify the final artifact state is consistent:
findings.md — all patterns have corpus-anchored prevalence and causal mechanisms
domain-signals.md — all thresholds validated against actual JSONL payloads, not assumed
session-index.md — inventory reflects the final corpus scope and outcome classification
diagrams/failure-patterns.dot — updated and re-rendered to PNG
Success criteria: Complete, consistent findings package ready to inform skill design
decisions for the context-intelligence bundle or any downstream action.
Standing Rules
Enumerate the full corpus before investigating. Anchor all signals to
population-level denominators. Do not sample and claim corpus-level conclusions.
Graph-first; disk as explicit mode. Both available via graph-analyst. Offer disk
scan when the user needs full coverage or suspects under-indexing.
Always split root vs sub-sessions. Roots and subs have structurally different
failure profiles. Separate denominators for all prevalence calculations.
Always compare failing vs successful runs. The delta between groups is the finding.
Signals that fire in both groups equally are workflow characteristics, not failure signals.
JSONL-observable behavioral signatures only — never named-agent filters. Express
every pattern as event field conditions observable in any session (see signals-reference.md
§Behavioral classification). Named instances are traceability evidence only.
S4 family only. The consecutive-same-tool-name heuristic is retired (48% FPR at
corpus scale). Use S4a, S4b, S4c, S4d from signals-reference.md.
Facilitator is the mandatory quality gate. Every wave of raw analyst findings must
pass through the facilitator before being treated as curated findings. Facilitator calls
do not need to be 1:1 with analyst calls — consolidate related findings.
Always describe signals in plain language and explain what they point to. In every
report, finding, or summary, signal codes (S1, S3, S9a, etc.) must always be accompanied
by their human-readable name, a plain-language explanation, AND a causal interpretation
of what the signal indicates about session health and what it could be pointing to.
The code is a shorthand for cross-reference only.
Wrong: "S3 fired at SEVERE. S9a also fired."
Correct:
S3 — LLM Iteration Runaway (SEVERE): 316 iterations — 8× the severe threshold of 40.
The session made 316 separate LLM calls. This is not normal volume for productive work;
it means the agent was looping — repeatedly asking the LLM for the next step without
reaching a conclusion. Common causes: an exploration task with no exit condition
(reading files and looking for something without knowing when to stop), a delegation
chain that keeps spawning new investigations, or a goal too vague to converge on.
At this volume, the session is on a trajectory toward context saturation — each LLM
call adds its response to the accumulated context, and without compaction the window
fills until the session stalls or crashes.
For every signal that fires, include:
The signal code and human name (e.g. "S3 — LLM Iteration Runaway")
The severity and measured value with the threshold (e.g. "SEVERE: 316 iterations — 8× threshold of 40")
What it means in plain terms (e.g. "the agent looped without converging")
What it points to — the underlying behaviour or root cause pattern. Use the
"What it points to" text from signals-reference.md for each signal as the source
of this interpretation. Do not summarise it away — quote or paraphrase it fully.
Risk trajectory — the consequence if unaddressed. Use the "Risk trajectory"
text from signals-reference.md as the source. This tells the reader why the
signal matters beyond its raw count.
