| name | learn-from-pr |
| description | Analyzes a completed PR to extract lessons learned from agent behavior. Use after any PR with agent involvement - whether the agent failed, succeeded slowly, or succeeded quickly. Identifies patterns to reinforce or fix, and generates actionable recommendations for instruction files, skills, and documentation. |
| metadata | {"author":"dotnet-maui","version":"1.0"} |
| compatibility | Requires GitHub CLI (gh) |
Learn From PR
Extracts lessons learned from a completed PR to improve repository documentation and agent capabilities.
Inputs
| Input | Required | Source |
|---|
| PR number or Issue number | Yes | User provides (e.g., "PR #33352" or "issue 33352") |
Outputs
-
Learning Analysis - Structured markdown with:
- What happened (problem, attempts, solution)
- Fix location analysis (attempted vs actual)
- Failure modes identified
- Prioritized recommendations
-
Actionable Recommendations - Each with:
- Category, Priority, Location, Specific Change, Why It Helps
Completion Criteria
The skill is complete when you have:
When to Use
- After agent failed to find the right fix
- After agent succeeded but took many attempts
- After agent succeeded quickly (to understand what worked)
- When asked "what can we learn from PR #XXXXX?"
When NOT to Use
- Before PR is finalized (use
pr-finalize first)
- For trivial PRs (typo fixes, simple changes)
- When no agent was involved (nothing to analyze)
Workflow
Step 1: Gather Data
gh pr view XXXXX --json title,body,files
gh pr diff XXXXX
Analyze the PR to extract learning:
- PR discussion - Comments reveal what was tried
- Commit history - Multiple commits may show iteration
- Code complexity - Non-obvious fixes suggest learning opportunities
- Similar past issues - Search for related bugs
Focus on: "What would have helped an agent find this fix faster?"
Step 2: Fix Location Analysis
Critical question: Did agent attempts target the same files as the final fix?
gh pr view XXXXX --json files --jq '.files[].path' | grep -v test
| Scenario | Implication |
|---|
| Same files | Agent found right location |
| Different files | Major learning opportunity - document why |
If different files: Answer these questions:
- Why did agent think that was the right file?
- What search would have found the correct file?
Step 3: Analyze Outcome
Determine which scenario applies and look for the relevant patterns:
Scenario A: Agent Failed
| Pattern | Indicator |
|---|
| Wrong file entirely | All attempts in File A, fix in File B |
| Tunnel vision | Only looked at file mentioned in error |
| Trusted issue title | Issue said "crash in X" so only looked at X |
| Pattern not generalized | Fixed one instance, missed others |
| Didn't search codebase | Never found similar code patterns |
| Missing platform knowledge | Didn't know iOS/Android/Windows specifics |
| Wrong abstraction layer | Fixed handler when problem was in core |
| Misread error message | Error pointed to symptom, not cause |
| Incomplete context | Didn't read enough surrounding code |
| Over-engineered | Complex fix when simple one existed |
Scenario B: Agent Succeeded Slowly (many attempts)
| Pattern | Indicator |
|---|
| Correct file, wrong approach | Found right file but tried wrong fixes first |
| Needed multiple iterations | Each attempt got closer but wasn't quite right |
| Discovery was slow | Eventually found it but search was inefficient |
| Missing domain knowledge | Had to learn something that could be documented |
Key question: What would have gotten agent to the solution faster?
Scenario C: Agent Succeeded Quickly
| Pattern | Indicator |
|---|
| Good search strategy | Found right file immediately |
| Understood the pattern | Recognized similar issues from past |
| Documentation helped | Existing docs pointed to solution |
| Simple, minimal fix | Didn't over-engineer |
Key question: What made this work? Should we reinforce this pattern?
Step 4: Find Improvement Locations
find .github/instructions -name "*.instructions.md" 2>/dev/null
find .github/skills -name "SKILL.md" 2>/dev/null
ls docs/design/ 2>/dev/null
ls .github/copilot-instructions.md 2>/dev/null
| Location | When to Add Here |
|---|
.github/instructions/*.instructions.md | Domain-specific AI guidance (testing patterns, platform rules) |
.github/skills/*/SKILL.md | Skill needs new step, checklist, or improved workflow |
/docs/design/*.md | Detailed architectural documentation |
.github/copilot-instructions.md | General AI workflow guidance |
| Code comments | Non-obvious code behavior |
Step 5: Generate Recommendations
For each recommendation, provide:
- Category: Instruction file / Skill / Architecture doc / Inline comment / Linting issue
- Priority: High (prevents class of bugs) / Medium (helps discovery) / Low (nice to have)
- Location: Exact file path
- Specific Change: Exact text to add
- Why It Helps: Which failure mode it prevents
Prioritization factors:
- How common is this pattern?
- Would future agents definitely hit this again?
- How hard is it to implement?
Pattern-to-Improvement Mapping (Failures/Slow Success):
| Pattern | Likely Improvement |
|---|
| Wrong file entirely | Check /docs/design/ for component relationships |
| Tunnel vision | Instruction file: "Always search for pattern across codebase" |
| Missing platform knowledge | Platform-specific instruction file |
| Wrong abstraction layer | Reference /docs/design/HandlerResolution.md |
| Misread error message | Code comment explaining the real cause |
| Over-engineered | Skill enhancement: "Try simplest fix first" |
Pattern-to-Improvement Mapping (Quick Success - reinforce):
| Pattern | Improvement |
|---|
| Good search strategy | Document the search pattern that worked in skills |
| Documentation helped | Note which docs were valuable, ensure they stay updated |
| Recognized pattern | Add to instruction files as known pattern |
Step 6: Present Findings
Present your analysis covering:
- What happened and what made it hard
- Where agent looked vs actual fix location
- Which patterns applied and evidence
- Prioritized recommendations with full details (category, priority, location, exact change, why it helps)
Error Handling
| Situation | Action |
|---|
| PR not found | Ask user to verify PR number |
| No agent involvement evident | Ask user if they still want analysis |
| Can't determine failure mode | State "insufficient data" and what's missing |
Constraints
- Analysis only - Don't apply changes (use learn-from-pr agent for that)
- Actionable recommendations - Every recommendation must have specific file path and text
- Don't duplicate - Check existing docs before recommending new ones
- Focus on high-value learning - Skip trivial observations
- Respect PR scope - Don't recommend improvements unrelated to the PR's learnings
Examples
Example: Wrong File Entirely
PR #33352 - TraitCollectionDidChange crash on MacCatalyst
What happened:
- Issue title: "ObjectDisposedException in ShellSectionRootRenderer"
- Agent made 11 attempts, ALL in
ShellSectionRootRenderer.cs
- Actual fix was in
PageViewController.cs
Failure Mode: Trusted issue title instead of searching for pattern.
Recommendation:
- Category: Instruction File
- Location:
.github/instructions/ios-debugging.instructions.md
- Change: "When fixing iOS crashes, search for the PATTERN across all files, not just the file named in the error"
- Why: Prevents tunnel vision on named file
Example: Slow Success
PR #34567 - CollectionView scroll position not preserved
What happened:
- Agent took 5 attempts to find fix
- First 3 attempts were in wrong layer (handler vs core)
- Eventually found it after reading more context
- Final fix was simple once the right layer was identified
Pattern: Wrong abstraction layer - fixed handler when problem was in core.
Recommendation:
- Category: Architecture Doc
- Location:
.github/architecture/handler-vs-core.md
- Change: Document layer responsibilities - handlers map properties, core handles behavior
- Why: Helps agent identify correct layer faster
Example: Quick Success
PR #35678 - Button disabled state not updating
What happened:
- Agent found fix in 1 attempt
- Searched for "IsEnabled" pattern across codebase immediately
- Found similar past fix in another control and applied same approach
- Simple, minimal change
Pattern: Good search strategy - recognized pattern from similar code.
Recommendation:
- Category: Skill Enhancement
- Location:
.github/skills/try-fix/SKILL.md
- Change: Add to search strategy: "Search for same property pattern in other controls"
- Why: Reinforces successful discovery technique
Integration
- pr-finalize → Use first to verify PR is ready
- learn-from-pr skill → Analysis only (this skill)
- learn-from-pr agent → Analysis + apply changes
For automated application of recommendations, use the learn-from-pr agent instead.
