sg-improve

name	sg-improve
description	Use after an audit, visual run, or debugging session when learnings should be captured — records local hints and mistakes, and proposes sanitized improvement issues.
context	conversation
argument-hint	[--local-only] [--github-only] [--dry-run] [--history] [--rollback[=#N]] [--keep-all]

/sg-improve — Self-Improving Feedback Loop

After an audit or visual-test session, extract what went well and what didn't, then feed those insights back into ShipGuard so the next run is better.

Recommended model: Sonnet 4.6. Mechanical retrospective work — use /model sonnet to save Opus quota.

Two outputs:

1. Local learnings (.shipguard/learnings.yaml) — project-specific knowledge, read automatically by the next /sg-code-audit
2. GitHub issue (on {target_repo}, Phase 5) — generic improvements for all ShipGuard users

Phases: 0 snapshot · 1 collect data · 2 extract signals · 3 classify · 4 learnings + 4b mistakes · 5 GitHub (sanitized) · 6 summary.

Invocations

Command	Behavior
/sg-improve	Full loop — local learnings + GitHub issue
--local-only	Save learnings locally, skip GitHub
--github-only	GitHub issue only; skip local writes AND snapshot creation (nothing local changes)
--dry-run	Preview exact target changes; no target-file or GitHub writes
--rollback	Revert to the previous snapshot (undo last run)
--rollback=#N	Revert to snapshot #N; consumes #N and all newer snapshots
--history	List all snapshots with dates and summary
--keep-all	Disable snapshot pruning (default keeps the last 5)

Sandbox note: real mode writes .shipguard/ and may call GitHub — use --dry-run first in restricted environments (../../docs/sandbox.md).

---

Dry-run Preview

With --dry-run, run Phases 1-3 normally, then stop before snapshots, local writes, and GitHub writes. Write the preview to visual-tests/_results/sg-improve-preview.md, or to .shipguard/preview/sg-improve-preview.md if that directory does not exist. .shipguard/preview/ is the one exception to "dry-run must not modify .shipguard/" — nothing else under .shipguard/, no GitHub issues, and no source files may be touched.

The preview must include: the target files that would be written; the exact YAML entries, mistakes.md sections, and GitHub issue/comment body; signals ignored and why; the snapshot path real mode would create.

Deterministic dry-run smoke test (also validates fixture schemas against the producer contracts; --keep-tmp/--debug to inspect):

node "$SHIPGUARD_PLUGIN_ROOT/skills/sg-improve/improve-dry-run-smoke-test.mjs"

---

Phase 0 — Snapshot (Safety Net)

Before touching ANY file, take a snapshot — the rollback point. Skipped with --github-only (nothing local changes). Targets: .shipguard/learnings.yaml and .shipguard/mistakes.md.

mkdir -p .shipguard/history/{timestamp}

# Copy current state (only files that exist)
cp .shipguard/learnings.yaml .shipguard/history/{timestamp}/ 2>/dev/null
cp .shipguard/mistakes.md    .shipguard/history/{timestamp}/ 2>/dev/null

# Metadata — record per-file PRE-EXISTENCE, not a fixed list
cat > .shipguard/history/{timestamp}/meta.yaml << EOF
timestamp: "{ISO 8601}"
trigger: "sg-improve"
mode: "{flags used}"
audit_bugs: {count from audit-results.json or "unknown"}
files:
  learnings.yaml: {existed|absent}
  mistakes.md: {existed|absent}
github: []
EOF

{timestamp} = YYYYMMDD-HHMMSS. After Phase 5, append every created GitHub issue/comment URL to github: in this meta.yaml.

Retention

Keep the last 5 snapshots; when creating the 6th, delete the oldest. --keep-all disables pruning.

--rollback

1. List snapshots: ls -1t .shipguard/history/ (newest first).
2. Show the newest snapshot's metadata and ask: Rollback? (yes/no)
3. If confirmed, apply the per-file pre-states from meta.yaml: existed → restore from the snapshot copy; absent → delete the file (the run created it; rollback removes it). Then delete the consumed snapshot directory.
4. If github: has entries, print: GitHub issue #N was NOT reverted — close it manually if needed.
5. Print: Rolled back to state before {timestamp}.

--rollback=#N

Restore from snapshot #N with the same per-file rules, then delete #N and all newer snapshots (#N is consumed like plain --rollback; newer pre-states no longer describe reality). Print the GitHub warning for every deleted snapshot listing URLs. Then: Rolled back to state #N ({date}). {M} newer snapshots removed.

--history

  #1  2026-04-14 07:30  79 bugs  learnings.yaml + mistakes.md  (latest)
  #2  2026-04-13 22:30  47 bugs  learnings.yaml
  Rollback: /sg-improve --rollback     Specific: /sg-improve --rollback=#2

Deterministic rollback smoke test (covers restore-existing and first-run-delete cases):

node "$SHIPGUARD_PLUGIN_ROOT/skills/sg-improve/improve-rollback-smoke-test.mjs"

---

Phase 1 — Collect Structured Data

Gather the hard data before scanning the conversation.

Step 1: Find audit-results.json

Check these paths in order (first found wins):

1. visual-tests/_results/audit-results.json
2. .code-audit-results/audit-results.json (legacy fallback)
3. audit-results.json (repo root)

If found, extract:

• summary.total_bugs, summary.by_severity, summary.by_category
• summary.duration_ms, agent_count (fallback agents.length for legacy reports — never a bare agents)
• prompt_hash — store it in this session's history entry (Phase 4)
• scope_info.mode, scope_info.total_in_scope (if diff mode)
• Bug counts: fix_applied: false (deferred), confidence: "low" (uncertain)
• verification.checked/.confirmed/.rejected; count of unverified_bugs

Prefer TOON: if audit-results.toon exists, use it for LLM analysis (Phase 2+, ~40% fewer tokens); use the JSON for field extraction here.

If not found, log: "No audit-results.json — extracting from conversation only."

Step 2: Read zone JSON files

Glob visual-tests/_results/zone-*-r*.json or .code-audit-results/zone-*-r*.json. Per file: zone, files_audited, bugs count, and whether the zone ID has an a/b suffix (a re-split happened).

Step 3: Read regressions

If visual-tests/_regressions.yaml exists, count entries; note any with consecutive_passes >= 2 (about to auto-remove — a success signal).

Step 3b: Read visual results

If visual-tests/_results/visual-results.json exists, extract summary.pass, summary.fail, summary.error, summary.stale — these feed Phase 2's Success Signals.

Step 4: Read git log

git log --oneline --since="12 hours ago" | grep -c "audit-r[0-9]"

Count audit commits; check for reverts (signal of a bad fix).

Step 5: Read existing learnings

If .shipguard/learnings.yaml exists, read it — Phase 4 merges with existing data, never overwrites.

---

Phase 2 — Extract Friction Signals

Combine Phase 1 data with conversation context — check both per signal type.

Error Signals

Signal	How to detect
Context overflow	Zone IDs with a/b suffix, or "Prompt is too long"
API overload	"529", "overloaded_error"
Post-merge syntax error	"IndentationError"/"SyntaxError"/"NameError" after a worktree merge
Browser collision	agent-browser returning / instead of the requested URL
Session expiry	Re-login needed mid-session
Docker failure	"unhealthy", "dependency failed" after rebuild
Merge conflict	"git merge --abort"

Performance Signals

Signal	How to compute
Overflow rate	(zones with a/b suffix) / (total original zones)
Retry count	"retry"/"retrying"/"attempt" mentions in conversation
Wall clock	summary.duration_ms, or first-to-last audit commit timestamps
Agent waste	Zone JSONs with duplicate zone paths

Quality Signals

Signal	How to compute
Noise ratio	summary.by_severity.low / summary.total_bugs
Top noise pattern	Most frequent category among low severity bugs
Deferred count	fix_applied: false count in bugs array
Post-audit regression	Docker/build failure AFTER audit commits (conversation)

User Friction Signals

Scan the user's messages (never assistant messages) for correction and frustration patterns — wrong behavior even when no error was thrown.

Signal	Patterns (case-insensitive)	Priority
COMMAND_FAILURE	Tool exit code != 0, stderr with "error", "failed", "not found"	100
USER_CORRECTION	I said, you didn't, that's wrong, no not	80
SKILL_OVERRIDE	skip that, don't do, ignore	75
REDO_REQUEST	redo, try again, re-run	70
REPETITION	Same instruction 2+ times (Jaccard > 0.5 over last 10 user messages)	60
TONE_ESCALATION	3+ uppercase words in a row, 2+ exclamation marks, STOP	40

A message can trigger multiple signals; COMMAND_FAILURE comes from tool results, not user messages. Priority >= 70 must produce a local learning (project-specific) or a GitHub insight (generic).

Success Signals

These matter just as much — they tell us what NOT to change.

Signal	How to detect
Clean merge rate	(total zones - merge conflicts) / total zones
Critical bug value	critical + high bugs with fix_applied: true
Visual verification	summary.pass from visual-results.json (Step 3b)
Zero-retry zones	Zones completed on first attempt

Record every signal as:

- signal: "user_correction"
  count: 2
  details: "User corrected the applied audit hint twice"
  impact: "~10 min lost"
  type: "friction"          # error | performance | quality | success | friction
  priority: 80              # friction only
  quote: "no, wrong file"   # exact user quote ≤100 chars, friction only

---

Phase 3 — Classify

Classification	Rule of thumb	Destination
project-specific	Mentions a file path, service, port, or timing specific to this repo	.shipguard/learnings.yaml only
generic	Would help ANY repo using ShipGuard — missing step, bad default, design flaw	GitHub issue
both	Generic pattern observed through a project-specific symptom	Both (GitHub side sanitized per Phase 5)

When in doubt, classify as LOCAL ONLY — a lost generic insight is recoverable; leaked project content is not.

---

Phase 4 — Local Learnings

Write to {repo_root}/.shipguard/learnings.yaml (create .shipguard/ if needed). Sections: zone_hints, infra_hints, audit_hints, noise_filters, success_patterns, session_history. Full v2 schema: see references/formats.md.

Update Rules

1. Read first. Load the existing file entirely before changing it.
2. Merge, don't overwrite. Match by path (zone_hints), service (infra_hints), pattern (audit/noise/success). On match: update last_seen, increment occurrences, enrich note. Append entries for new signals.
3. session_history: prune to the last 10 entries. Each entry records the audit's prompt_hash. When the current hash differs from the previous session's, flag: "baseline discontinuity — audit prompt changed; treat count deltas across this boundary as non-comparable."
4. Never delete zone_hints, audit_hints, or success_patterns — the user prunes manually. Mark stale entries (last_seen > 90 days) with possibly_stale: true instead.
5. Preserve comments and manual edits. Parse the YAML, modify in memory, write back with comments intact.

---

Phase 4b — Mistakes File (Coding Memory)

Alongside learnings.yaml, maintain a human-readable journal at {repo_root}/.shipguard/mistakes.md, read at every coding session. Header: # Mistakes not to repeat. Entry format: see references/formats.md.

Only real bugs found in THIS project — not generic best practices. Each entry: the bad pattern actually written, the fix, and context (files, instance count, what broke).

Update rules: read the existing file first — don't overwrite. For each critical/high audit bug: if a similar pattern exists, update instance count and context; otherwise add an entry under the language section. No low severity entries. One screen of code max per entry. Refresh the "last updated" date.

CLAUDE.md wiring

The skill suggests the reference line below for the project's CLAUDE.md and never edits CLAUDE.md itself — show it to the user and let them add it, so every session (not just audits) benefits:

## Recurring Mistakes
**Full mistakes journal: `.shipguard/mistakes.md`** — READ this file at the start of every session.

---

Phase 5 — GitHub Issue

Target repo

Define {target_repo} once: the origin remote of the ShipGuard plugin installation directory (never the project under test); fallback bacoco/ShipGuard. Sanity check: if the resolved name does not contain "ShipGuard"/"shipguard", warn and fall back to bacoco/ShipGuard. Use {target_repo} in every gh command below.

Pre-flight

Check gh auth status — if not authenticated, skip this phase: "GitHub CLI not authenticated. Run gh auth login to enable issue filing. Local learnings saved."

Sanitization (MANDATORY — the target repo is public)

Issue and comment bodies must contain:

• NO code excerpts from the project under test
• NO file paths from the project under test
• NO internal URLs or hostnames
• NO secrets (keys, tokens, credentials)
• NOT the project name, unless the user explicitly consents

Describe frictions generically — "a FastAPI project with a Celery queue", never repo names or paths.

Confirmation gate: in real mode, render the full body and ask the user before every gh issue create and gh issue comment (--dry-run already previews without filing).

Deduplication and cardinality

One session issue per run, maximum; individual insights matching an existing open issue become comments on it.

gh issue list --repo {target_repo} --state open --label improvement --limit 30 --json number,title,body

For each insight: extract keywords from its title; if an open issue matches, add a comment with the new data point (builds cross-user evidence instead of duplicates). All remaining insights go into the single new session issue:

gh issue create --repo {target_repo} --title "{title}" --label improvement --body "{sanitized body}"

Issue body format, comment format, and label taxonomy: see references/formats.md.

Record every created issue/comment URL in the snapshot's meta.yaml github: list (Phase 0).

---

Phase 6 — Summary

/sg-improve complete

Local (.shipguard/learnings.yaml):
  + {N} zone hints   + {M} audit hints   + {K} noise filters   + {S} success patterns
  Session #{H} recorded {— baseline discontinuity flagged, if prompt_hash changed}

GitHub:
  {Created {target_repo}#{number} — "{title}" | Commented on #{number} | Skipped (reason)}

Next /sg-code-audit will: cap {path} zones at {max_files} files,
add {N} patterns to agent checklists, batch {K} noise patterns.

---

How sg-code-audit Consumes Learnings

sg-code-audit reads .shipguard/learnings.yaml at its Step 1.5 (zone_hints caps, audit_hints and noise_filters injected into agent prompts). See that skill. The loop: a max_files hint saved today re-splits that zone on the next audit; a saved audit_hint lands in every agent's checklist.

---

Edge Cases

• No audit-results.json: extract signals from conversation only; learnings will be thinner.
• First run (no learnings.yaml): create it from scratch; session_history starts with one entry.
• /sg-improve twice in one session: update today's session_history entry, don't append a duplicate.
• gh CLI missing or unauthenticated: skip Phase 5; print the sanitized issue body for manual filing.
• Very long conversation (>100K tokens): rely on Phase 1 data, the last 20 messages, and a grep-style scan for error keywords.
• No ShipGuard skills used this session: ask: "I don't see a ShipGuard session in this conversation. What would you like me to analyze?"

---

Final Checklist

• Snapshot taken BEFORE any modification; per-file pre-existence in meta.yaml (skipped only with --github-only)
• audit-results.json (incl. prompt_hash), visual-results.json, zone JSONs, git log read where present
• Conversation scanned for error/performance/quality/friction/success signals
• Each signal classified (project-specific / generic / both; when in doubt → LOCAL ONLY)
• .shipguard/learnings.yaml merged; prompt_hash recorded; discontinuity flagged if changed (unless --github-only)
• .shipguard/mistakes.md updated with critical/high bug patterns (unless --github-only)
• GitHub bodies sanitized (no code, paths, hostnames, secrets, or project name without consent)
• User confirmed before every gh issue create / gh issue comment (unless --local-only or --dry-run)
• At most one new session issue; matching insights become comments; URLs recorded in meta.yaml
• Summary displayed with concrete "next run will..." predictions