| name | ai-learn |
| description | Extracts lessons from merged PR review feedback by analyzing what reviewers caught, identifying missed checks, and writing entries directly to LESSONS.md. Trigger for 'the AI keeps doing X wrong', 'learn from this PR', 'what patterns did reviewers catch', 'update our standards from feedback'. Not for in-session observation; use /ai-session-watch instead. Not for skill-level rewrites; use /ai-skill-improve instead. |
| effort | mid |
| argument-hint | single [pr]|batch |
| tags | ["meta","learning","continuous-improvement"] |
Learn
Continuous improvement from delivery outcomes: analyzes merged PRs to find where AI missed what human reviewers caught, identifies false positives, and writes lessons directly to .ai-engineering/LESSONS.md. Command: /ai-learn single <pr>|batch (single = after a PR merge; batch = periodic).
Workflow
Step 0: read .ai-engineering/LESSONS.md for pre-existing patterns; load stack contexts: read .ai-engineering/manifest.yml providers.stacks and apply .ai-engineering/overrides/<stack>/conventions.md for each stack.
Two modes — single <pr> (one PR) and batch (all merged PRs since last lesson update) — share one loop: read PR comments + diff → classify each into a Pattern Category → dedupe against existing LESSONS.md entries → append new lessons with category + evidence link → sweep for AGENTS.md proposals. Authoritative per-mode steps below.
Modes
single -- Analyze one merged PR
-
Fetch PR data -- gh pr view <pr> --json body,reviews,comments,files,additions,deletions.
-
Collect AI findings -- read the AI-generated PR description, guard advisories, and verify results from the PR.
-
Collect human feedback -- extract all review comments, requested changes, and approval notes.
-
Cross-reference -- compare AI findings with human feedback:
| Category | Description |
|---|
| AI miss | Human reviewer found an issue AI did not flag |
| False positive | AI flagged something human reviewer dismissed or overrode |
| AI hit | AI flagged an issue human reviewer agreed with |
| Novel insight | Human added context AI could not have known |
-
Write lesson -- for each actionable pattern found (AI miss, false positive, or novel insight), append a lesson entry to .ai-engineering/LESSONS.md:
### [Pattern name derived from PR analysis]
**Context**: [What happened in PR #NNN — the specific review feedback]
**Learning**: [The pattern or rule extracted from the feedback]
**Rule**: [Actionable instruction for future sessions]
Only write lessons for patterns that are repeatable and actionable. Skip one-off issues specific to a single PR.
batch -- Process unanalyzed merged PRs
- Read tracking marker -- check
.ai-engineering/LESSONS.md YAML frontmatter for lastAnalyzedAt field. If absent, this is the first batch run.
- Find unanalyzed PRs --
git log --merges --since=<lastAnalyzedAt> --format="%H %s". Extract PR numbers from merge commit messages. If git log --merges yields no results (e.g., squash-merge workflow), fall back to gh pr list --state merged --json number,mergedAt filtered by lastAnalyzedAt.
- Process each -- run single-mode analysis for each unanalyzed PR.
- Update marker -- set
lastAnalyzedAt: <current ISO date> in LESSONS.md frontmatter (add frontmatter if absent).
- Summary -- report total PRs analyzed, lessons written, and emerging patterns.
Pattern Categories
| Pattern | Example | Action |
|---|
| Missed check | AI never flags missing error handling in async code | Write lesson with Rule for future sessions |
| Over-flagging | AI flags every single-letter variable in list comprehensions | Write lesson noting the exception |
| Missing context | Reviewers always explain why a specific pattern is used in this codebase | Write lesson adding the context |
| Style drift | Reviewers consistently request a style AI does not enforce | Write lesson with the style rule |
AGENTS.md proposal mode (spec-121)
Single-PR analysis writes to LESSONS.md. Procedural memory (AGENTS.md, CONSTITUTION.md) is the durable layer agents read on every session — when a category of lessons crosses threshold, it should be reinforced there, not buried in LESSONS.md.
After every batch run (or at the end of a single run), perform a category sweep:
- Group all lessons in
.ai-engineering/LESSONS.md by Pattern Category (Missed check / Over-flagging / Missing context / Style drift / custom).
- For any category whose count is ≥ 5 AND that has not already been reflected in AGENTS.md (grep AGENTS.md for the category name or a representative phrase), draft a proposal block.
- Append the proposal to
.ai-engineering/state/agents-proposals.md (create if absent). Never edit AGENTS.md directly — same constraint as /ai-dream (D-118-04). Humans review and merge proposals manually via PR.
Proposal block format:
## Proposal — <ISO date> — <Category name>
**Trigger**: <N> lessons in category "<Category>" since <oldest>; AGENTS.md does not yet codify this rule.
**Suggested AGENTS.md addition** (under section `## Hard rules` or appropriate):
> <single-sentence imperative rule derived from the lessons>
**Evidence** (lesson titles, PR refs):
- <lesson 1>
- <lesson 2>
- ...
**Action**: open a PR adding the rule above to AGENTS.md if accepted.
Emit a framework_operation event with operation=agents_proposal_drafted, category=<name>, lesson_count=<N> so the audit chain records each proposal cycle.
Examples
User: "synthesize lessons from everything merged this sprint" → /ai-learn batch walks merged PRs since last lesson update, runs single-mode analysis per PR (classify into pattern categories, dedupe, append to LESSONS.md), and drafts AGENTS.md proposals when categories accumulate enough evidence.
Integration
Called by: user directly, post-merge automation. Reads: gh pr view, LESSONS.md. Writes: LESSONS.md (append-only). See also: /ai-note (individual findings), /ai-session-watch (in-session corrections), /ai-skill-improve (acts on accumulated lessons).
$ARGUMENTS
