harness-refine

name	harness-refine
description	Improve the harness based on session errors. Classifies each error as harness gap (missing docs, wrong version, anti-pattern, rule violation) or inherent complexity. Patches gaps, documents complexity. Run after any session with notable errors.
disable-model-invocation	true

Harness Refine

Improve the harness based on session errors: $ARGUMENTS

GROUNDING RULE: Every proposed fix must be backed by evidence — from the error itself (classifying the root cause) or from research (verifying the fix). No speculative patches.

---

Phase 1: Input + Harness Read

Parse $ARGUMENTS:

• If a file path: read the file as session log
• If natural language / error descriptions / stack traces: treat as-is
• If empty: ask the user to describe what went wrong — errors seen, corrections made, patterns noticed

Read existing harness:

• CLAUDE.md, all .claude/skills/, .claude/rules/, .claude/agents/
• .claude/settings.json, .mcp.json
• project-wiki.md if it exists — use Reuse Map and Anti-Patterns to inform classification (wiki entries are project-specific; rules are generic)

---

Phase 2: Error Extraction + Classification

Extract each distinct error or correction from the input. For each, determine the class:

Class	Signal	Harness fix
API knowledge gap	Agent used non-existent method, wrong endpoint, wrong field name	MCP or mcpdoc addition
Version mismatch	Deprecated API, wrong method signature, behavior changed in newer version	mcpdoc pointing to wrong version
Repeated anti-pattern	Same mistake N≥2 times, no existing rule covers it	New rule file
Rule exists but violated	Rule is in harness but agent ignored or misapplied it	Strengthen rule: rewrite explicitly or elevate key point to CLAUDE.md
Missing verify step	Error type that a deterministic check would have caught	Patch verify skill or ci-gates rule
Missing workflow step	Process gap — agent skipped a step or did steps out of order	Patch primary skill
Inherent complexity	Strategic decision, business context only user has, one-off edge case without pattern	Document in references/ only — no harness patch

Build a classification table:

Error / Correction             | Class                  | Root cause                    | Proposed fix
───────────────────────────────────────────────────────────────────────────────────────────────────
Used `stripe.charge.create` ×2 | API knowledge gap      | Charges API deprecated in v3  | Add mcpdoc for stripe v3
Same raw SQL pattern ×3        | Repeated anti-pattern  | No rule against raw SQL       | Add rule: parameterized queries
Wrong auth header format ×2    | API knowledge gap      | No auth API docs in context   | Add MCP for auth service
`tsc` errors not caught early  | Missing verify step    | No compile hook               | Add PostToolUse tsc hook
Business direction corrected   | Inherent complexity    | Strategic call, not knowledge | Note in references/

---

Phase 3: Targeted Research

Run only for gaps that need verification before patching. Skip if fix is self-evident from the error.

Agent V — Verify proposed fixes

Run if: API knowledge gap or version mismatch.

For each gap:

• Find correct current docs (official source, not blog)
• Verify mcpdoc supports this library: search <library> llms.txt
• Or find a working MCP server: search npm mcp-server-<library>, check mcp.so
• Confirm exact correct API — method, endpoint, field, signature
• Note exact version where behavior changed (version mismatch cases)

Run if: rule gaps.

For each anti-pattern:

• Find authoritative source confirming it's a real anti-pattern
• Find the canonical "correct way" to cite in the rule body

Return: source URL + one-sentence evidence for each proposed fix.

---

Phase 4: Refinement Report

Show the user:

Session analysis: <N> errors extracted

Harness gaps (<N>):
  API knowledge gap:      <error> → <proposed fix>
  Version mismatch:       <error> → <proposed fix>
  Repeated anti-pattern:  <error> → <proposed fix>
  Rule violated:          <error> → <proposed fix>
  Missing verify step:    <error> → <proposed fix>
  Missing workflow step:  <error> → <proposed fix>

Inherent complexity (<N>) — will document only:
  <error> — <why it's inherent>

Research: <N> fixes verified with sources.
<N> patches to apply.

If no harness gaps found:

No harness gaps identified in this session.
All errors appear to be inherent complexity:
  <list with reasoning>

Stop here.

Get implicit confirmation — proceed unless the user objects within one reply.

---

Phase 5: Surgical Patch

Apply only the fixes from the classification table.

API knowledge gap → mcpdoc: Add to .mcp.json with install command from research verbatim. Add to enabledMcpjsonServers in .claude/settings.json.

API knowledge gap → skill reference: Add correct method/pattern to the relevant skill's references/ directory.

Version mismatch: Update .mcp.json mcpdoc entry to point to the correct version. Add a rule: "Always use <lib> version X.Y — method Z was removed/changed in vX."

Repeated anti-pattern → new rule: One file in .claude/rules/<topic>.md. Invariant format: "Always X", "Never Y, because [reason from research]".

Rule exists but violated: Read the existing rule. Rewrite with more explicit language. If the violation is severe, elevate the key point into CLAUDE.md as a MUST NOT (not a copy — a one-liner pointer is enough).

Missing verify step: Read existing verify skill or .claude/rules/ci-gates.md. Add the missing check. If neither exists, create ci-gates.md with the check and wire it as a PostToolUse hook in .claude/settings.json.

Missing workflow step: Read existing primary skill. Insert the missing step at the correct position in the workflow.

Inherent complexity: Append to .claude/skills/<relevant-skill>/references/known-edge-cases.md — document the situation, context, and how it was resolved. Create the file if it doesn't exist.

Reflection check (after each patch)

• Is this patch directly tied to a classified error?
• Is the fix backed by research or clearly evident from the error?
• Did I touch anything not in the classification table?

Hard stop at 8 patches.

---

Append to harness-research.md

Add a section:

## Refinement: <date>

### Session errors classified
[classification table from Phase 2]

### Fixes applied
[each fix with evidence source]

### Inherent complexity documented
[list with context]

If harness-research.md doesn't exist, create it with this section only.

---

Completion Report

Harness refined from <N> session errors.

Effective immediately:
  <rules added/strengthened, skill content patches, references/ additions>

Requires restart:
  <MCP servers added to .mcp.json>
  <new skills — auto-invocation index rebuilds on restart>

Inherent complexity documented in references/:
  <list>

Gaps not addressed (research inconclusive): <list if any>

If anything is listed under "Requires restart": tell the user to restart Claude Code before the next session.