| name | mistake-tracking |
| description | Log, analyze, and prevent recurring mistakes. Use when: (1) a mistake or near-miss occurs (self-detected or user-detected), (2) session start/end review of recent patterns, (3) user asks to review or analyze mistakes, (4) after any corrective action to verify the fix was logged. Covers logging JSONL events, running RCA, promoting/demoting guardrails, and testing prevention. |
Mistake Tracking
Each mistake is an iteration on the prevention system. The goal is not just to log what happened, but to make the system better at catching the next one.
Cycle: detect → understand → fix the system → test the fix → monitor
Files
- Global log:
~/.claude/mistakes.jsonl
- Project log:
<repo>/logs/mistakes.jsonl
- Analysis script:
scripts/analyze-mistakes.sh
- Validation script:
scripts/log-mistake.sh
- Known patterns: references/common-antipatterns.md
- Testing guide: references/testing.md
- Eval scenarios:
evals/evals.json
1. Detect & Log
When a mistake or near-miss occurs, use the validation script to log it:
scripts/log-mistake.sh '{"ts":"2026-02-23T00:00:00Z","repo":"~/ws/example","mistake_id":"guide.not_consulted","scope":"global","detector":"user","notes":"what happened","action_taken":"what systemic change was made"}'
The script validates required fields and mistake_id format before appending. It rejects malformed entries.
Required: ts, mistake_id (lowercase dot-separated, e.g. guide.not_consulted), scope (global|project), detector (self|user), notes
Recommended: type (mistake|near-miss|violation|learning), severity (critical|high|medium|low), action_taken, repo, guide, condition
Before logging, check references/common-antipatterns.md — if it matches a known pattern, reuse the existing mistake_id.
2. Understand Why the System Missed It
This is the key step. Don't just categorize the mistake — ask why the existing system didn't prevent it:
- Was there a guardrail? If no → the system has a gap. If yes → the guardrail failed.
- Why did the guardrail fail? Too passive? Wrong location? Doesn't match the trigger condition? I rationalized past it?
- What's the smallest systemic change that would have caught this? A stronger guardrail? A new one? A script? A check in a skill?
3. Fix the System
Based on the analysis, make a concrete change. Run scripts/analyze-mistakes.sh to see if this is part of a pattern:
scripts/analyze-mistakes.sh
scripts/analyze-mistakes.sh --days=30
The script outputs promotion candidates automatically. Act on them:
Promote (when action_needed: true):
- First occurrence → improve or create the relevant skill
- 2+ in 14 days, same repo → add Hot Rule to that repo's CLAUDE.md
- 2+ repos in 14 days → add universal guardrail to root
~/.claude/CLAUDE.md
- Guardrail already exists but miss recurred → "already exists" is never a reason to skip. Promote by rewriting the guardrail stronger: passive wording → imperative STOP gate, name the specific failure pattern, increase prominence.
Demote (when 14-30 quiet days pass):
- Propose removing one-liner hot rules (the skill/guide remains as canonical source)
Always log action_taken in the JSONL event describing the systemic change.
4. Test the Fix
Don't assume the fix works — verify it. See references/testing.md for the full procedure.
For guardrail changes (CLAUDE.md edits, skill updates):
- Check
evals/evals.json for an existing eval matching this mistake_id
- If none exists, create one: write the scenario prompt as the context that triggered the mistake, write expectations that check process (did the agent consult X, did it stop at Y)
- Run the eval — spawn a subagent with the scenario prompt, grade against expectations
- If it fails → the guardrail is insufficient, iterate (strengthen wording, add specificity, rerun)
For script/tooling changes:
- Run the script against real data (not synthetic)
- Verify the output catches the pattern that was missed
- Test edge cases (empty log, old-format entries, boundary dates)
5. Monitor
At session start/end, run scripts/analyze-mistakes.sh:
Acceptance Checks
