| name | harness-optimize |
| description | Research a coding harness via changelogs and live config, then emit reviewable proposed diffs (which file, what change, why now, how to revert) plus a scorecard — apply only after explicit approval. Use for /harness-optimize, optimize my setup, harness doctor, agent config audit. Grok-first. Differentiator: trust via reviewable fix proposals, not scores alone; report-then-ask-to-apply; never silent mutations.
|
Harness Optimize
End-to-end loop: research → inventory → proposed diffs (primary artifact) +
scorecard (secondary) → STOP for approval → apply only approved ids → cache.
Trust principle (product)
A harness doctor is trustworthy when its own fixes are reviewable.
- The scorecard is useful context.
- The artifact to keep is each proposed fix: which config, which
permission/setting, why now, exact diff or patch, how to revert.
- Prefer a short stack of reviewable proposals over a long prose lecture.
Hard gates (non-negotiable)
- Propose before implement. Every run must present the findings report
with a Proposed fixes section (reviewable diffs) before any config
mutation. Scorecard supports prioritization; it does not replace proposals.
- Never change without permission. Do not edit configs until the user
approves specific recommendation ids via
ask_user_question. Goal mode,
yolo, and always-approve do not count as apply permission.
- Use the question tool for user decisions (scope, which proposals to apply).
- Default mode is report-only. After proposals, always ask what to apply.
Allowed without asking: read files, run diagnostic scripts under scripts/
(discover, inventory, mcp-doctor, env-check, mcp-matrix, dual-rules,
permission-audit, recurrence-check, restore-kit --list, run-diagnostics),
grok mcp doctor / version commands, web fetch of changelogs, write report
text into the chat (or a path the user already named).
Forbidden without prior explicit apply approval: cp/mv/rm of harness
configs, editing config files, enabling/disabling MCP, moving secrets, archiving
agents/skills, rewriting CLAUDE.md/AGENTS.md.
Modes
| Mode | Trigger | Behavior |
|---|
| Full | first run, /harness-optimize full, or no state file | Research + inventory + interview + report; then ask to apply |
| Delta | default when state.json exists | Re-fetch changelogs since last run; re-inventory deltas; report; then ask to apply |
| Target | /harness-optimize claude / grok / codex / all | Limit research + inventory to named harness(es) |
| Report-only | default; also dry-run / report | Never write configs |
| Apply | only after question-tool approval of specific items | Implement approved recommendation ids only |
apply in $ARGUMENTS means “user is open to applying after the report” — it
does not skip the report or the permission question.
State file (create dir if needed; write state.json only for report metadata
after the report, or for apply results after approved apply):
$HARNESS_OPTIMIZE_STATE (default under skill or GROK_PLUGIN_DATA)
Phase 0 — Target + safety
- Detect which harness is running this session (env, binary, skill path).
- If target harness is ambiguous and not in
$ARGUMENTS, use ask_user_question
(multi-select OK). Do not invent scope.
- State constraints:
- No secret values in reports (redact tokens, client secrets, Bearer headers).
- No force-push, no deleting without archive path and apply approval.
- Respect one-in-one-out for
~/.claude surface when proposing adds.
- Project edits: if git-tracked, prefer worktree or explicit commit; don't dirty shared checkouts silently.
Phase 1 — Release check + full changelog since last run
Do this before judging the install. Stale advice is worse than no advice.
1a. New Grok release available?
bash "$HO_ROOT/scripts/release-check.sh"
If update_available | true:
- Add a High (or Critical if breaking channel) recommendation: upgrade Grok
(
grok update / pin version). Do not run the install without user approval.
- Still complete the rest of the report on the currently installed version.
- After the user upgrades, they should re-run
/harness-optimize so the new
changelog is fully catalogued.
1b. Catalog all changelog changes since last run
bash "$HO_ROOT/scripts/changelog-delta.sh"
This compares ~/.grok/CHANGELOG.md + CHANGELOG.json to:
state.json → last_changelog_cursor.grokstate.json → last_changelog_content_hash
Required behavior:
- Include every release newer than the cursor (not only the latest).
- If the content hash changed for the same version, re-list all bullets for
that version (nothing dropped).
- For each bullet, print it as
informational, actionable, or breaking.
- For every actionable/breaking bullet, add a concrete harness suggestion
(
rec_seed / report rec id). Do not stop at “no config keywords” without
listing informational items — the user must see the full catalog.
- Prefer local files; only then fetch remote docs if files are missing.
Also load docs as needed:
- Grok:
~/.grok/docs/user-guide/ (config, skills, hooks, MCP, permissions)
- Other harnesses: per
references/harness-registry.md
1c. Other harnesses (if targeted)
Claude / Codex / etc.: same idea — version + changelog since cursor when those
paths exist.
Phase 2 — System inventory + Grok diagnostics pack
Skill root (scripts live next to this file):
if [ -n "${GROK_PLUGIN_ROOT:-}" ] && [ -d "$GROK_PLUGIN_ROOT/skills/harness-optimize/scripts" ]; then
HO_ROOT="$GROK_PLUGIN_ROOT/skills/harness-optimize"
elif [ -d "$HOME/.grok/skills/harness-optimize/scripts" ]; then
HO_ROOT="$HOME/.grok/skills/harness-optimize"
else
HO_ROOT="$(dirname "$(readlink -f "${BASH_SOURCE[0]:-$0}")" 2>/dev/null || true)"
fi
export HARNESS_OPTIMIZE_STATE="${HARNESS_OPTIMIZE_STATE:-${GROK_PLUGIN_DATA:-$HOME/.grok/skills/harness-optimize}/state.json}"
export HARNESS_OPTIMIZE_REPORTS="${HARNESS_OPTIMIZE_REPORTS:-${GROK_PLUGIN_DATA:-$HOME/.grok/skills/harness-optimize}/reports}"
Prefer the pack (read-only):
bash "$HO_ROOT/scripts/run-diagnostics.sh"
Or individual scripts under $HO_ROOT/scripts/:
| Script | Captures |
|---|
discover.sh | Binaries, versions, config roots |
inventory.sh --scope system | Agents/skills/rules counts; SECRET flag |
recurrence-check.sh | full vs delta suggestion vs state.json |
env-check.sh | secrets file mode, bashrc hook, process + interactive env lengths |
permission-audit.sh | permission_mode, yolo, permission tables, hooks |
mcp-doctor.sh | healthy/failing, tools_total, tool_budget, fail rec seeds |
mcp-matrix.sh | user enabled vs doctor source vs action_hint |
restore-kit.sh | available config backups |
Capture (do not dump secrets):
| Layer | Examples |
|---|
| Binaries / versions | claude, grok, codex, opencode, agy, cursor |
| Global config | settings.json, config.toml, mcp.json, plugins enabled/disabled |
| Always-on context | agents, rules, memory, CLAUDE.md / AGENTS.md at home |
| On-demand | skills, commands, hooks scripts + wired events |
| MCP | doctor health, matrix, tool budget |
| Env | ${VAR} refs set in interactive shell; launcher gap OPEN if unknown |
| Cross-tool copies | same skill name in multiple harness dirs (flag for drift; deep structure → /cli-skills-audit) |
Claude-specific token traps (verify with live facts, not folklore):
~/.claude/rules/**/*.md often loads recursively every prompt- Active
agents/*.md often load every prompt (archive unused)
- Skills are usually discovery-cheap until activated
Phase 3 — Per-project inventory
bash "$HO_ROOT/scripts/inventory.sh" --scope projects \
--root "${PROJECTS_ROOT:-$HOME/projects}"
bash "$HO_ROOT/scripts/dual-rules.sh"
Per project folder, record presence and rough size of:
CLAUDE.md, AGENTS.md, .claude/, .grok/, .codex/, .cursor/, .agents/, .mcp.json- Project hooks/settings, skills, agents
- Package manager / stack signals (for MCP and hook recommendations)
- Tier if known from workspace map (factory / product / experiment) — quality gates should match tier
Flag:
- Dual
CLAUDE.md + AGENTS.md with large overlap (Grok may load both) — use dual-rules.sh severity
- Project MCP overlapping broken global MCP
- Factory projects missing typecheck/test/hooks
- Experiment projects with full enterprise harness bloat
Phase 4 — Blindspot pass (unknown unknowns)
Before interviewing, list 5–10 items as RESOLVED or OPEN. Seed from
references/blindspots.md. Typical OPEN categories:
- Primary daily harness vs secondary (configs should bias primary)
- Solo vs multi-agent vs AFK loops
- Risk posture (yolo / always-approve vs on-request)
- Token budget sensitivity
- Which projects are "active money" vs archive-class
- Whether plugins/MCP auth failures are acceptable noise
- Team-shared project configs vs personal-only
Phase 5 — Interview (architecture-affecting only)
Read references/interview-tree.md. Rules:
- Always use
ask_user_question for each needed decision (or one multi-select
block of tightly related options). Never invent answers silently.
- Prefer options + (Recommended) default over free text.
- Skip questions already answered in
state.json.preferences only if the user
previously confirmed them in a prior interactive run (not agent-assumed).
- Stop when remaining questions would not change recommendations.
- If the user explicitly says “use recommended defaults” via the question tool
or chat, record an assumption ledger and continue to the report —
still do not apply changes without a separate apply approval.
Priority order (stop early if answers saturate):
- Primary harness + secondary
- Work mode (interactive craft / parallel agents / AFK loops / CI headless)
- Permission posture
- Context budget (lean / balanced / kitchen-sink)
- Active project shortlist (or "all under ~/projects")
- Cross-harness parity desire (symlink skills vs independent copies)
Do not ask “apply now?” in Phase 5 — that happens only after Phase 6.
Phase 6 — Proposed fixes (primary) + scorecard (secondary)
Primary deliverable: a stack of reviewable proposals (P1/C1/H1/…).
Secondary: scorecard for prioritization. Details: references/scoring.md,
references/proposal-format.md.
Proposal contract (required for every fix recommendation)
Each id must include all of:
| Field | Meaning |
|---|
| Id | Stable id (C1, H1, M1, …) |
| Priority | Critical / High / Medium / Low |
| Target | Exact path(s) — e.g. ~/.grok/config.toml [permission], project AGENTS.md |
| Why now | Evidence: doctor line, changelog bullet, inventory flag, security risk |
| Proposed change | Unified diff -u or before/after snippet (no secrets) |
| How to revert | Concrete: backup path pattern, inverse edit, or restore-kit.sh --restore … |
| Verify after apply | Command or observation (e.g. permission-audit.sh, mcp doctor) |
If you cannot produce a reviewable diff for a finding, label it observation
only (no apply id) — do not invent an un-reviewable “just fix it” action.
Optionally persist proposals to:
$HARNESS_OPTIMIZE_REPORTS/proposals-YYYYMMDD.md
Scorecard (supporting)
Score 0–10 each (see references/scoring.md). Useful; not a substitute for diffs.
Report template
# Harness Optimize Report — YYYY-MM-DD
## Scope
Harnesses: … | Mode: full|delta | Primary: …
## Release status
- current / latest / update_available
- If update available: proposal id with exact `grok update` command + revert (n/a)
## Changelog catalog (since last run)
- Every release + every bullet (informational | actionable | breaking)
## Diagnostics summary
- Env / permission / MCP / dual-rules (short; details in scripts)
## Scorecard (secondary)
| Dimension | Score | Top finding |
|-----------|-------|-------------|
## Proposed fixes (primary artifact — review before apply)
### C1 — <title>
- **Priority:** Critical
- **Target:** `~/.grok/config.toml` → `[mcp_servers.xapi]`
- **Why now:** <doctor/changelog/inventory evidence>
- **Proposed change:**
```diff
- enabled = true
+ enabled = false
- How to revert: restore
~/.grok/config.toml.bak.<ts> or set enabled = true
- Verify:
bash …/mcp-doctor.sh / xapi-health.sh
H1 — …
…
Observations (no apply)
- Items without a safe reviewable patch
Restore kit
Assumption ledger
Next step
No config changes have been made. Select proposal ids to apply, or report only.
Recommendation rules:
- Prefer **minimal reversible** diffs with measurable effect.
- Map each proposal to **evidence** (changelog, doctor, inventory).
- Cap volume: top **3 Critical + 5 High** unless user wants full dump.
- Never put secret values in diffs; use `${ENV}` / redaction.
- Failing MCP → disable/re-auth proposal with exact toml hunk.
- Dual rules → pointer/merge proposal with exact file paths.
### Phase 6b — Permission gate (required)
After **Proposed fixes** are shown, call **`ask_user_question`**:
- **Question:** Which **proposals** should I apply?
- **Options:** Report only / stop (Recommended) · Apply all Critical ·
Apply Critical + High · Selected ids (Other) · Custom…
**STOP.** Do not enter Phase 7 without explicit ids.
## Phase 7 — Apply (only after Phase 6b approval)
Only the **approved** proposal ids — re-show each proposal’s diff, then:
1. Confirm the same **Proposed change** still applies (re-read target file).
2. Backup: `cp -a <target> <target>.bak.$(date +%Y%m%d-%H%M%S)`.
3. Apply the reviewed hunk only (no drive-by edits).
4. Verify per proposal’s **Verify** field.
5. Record applied ids + backup paths in `state.json`.
6. Final summary: **what changed / backup path / how to revert** for each id.
If the file drifted so the proposed diff no longer applies: **stop**, re-propose,
ask again — do not force a fuzzy patch.
## Phase 8 — Persist + recurrence
After the **report** (even if nothing applied / report-only), **always** update
changelog review markers so the next run only sees *new* changes:
From `changelog-delta.sh` output:
| State field | Set to |
|-------------|--------|
| `last_changelog_cursor.grok` | `suggest_cursor` |
| `last_changelog_content_hash` | `suggest_content_hash` |
| `last_changelog_reviewed_at` | current UTC ISO-8601 |
| `versions.grok` | current short version (e.g. `0.2.104`) |
| `last_release_check` | `{ current, latest, update_available, checked_at }` from release-check |
Also write evaluation metadata, scores, preferences the user confirmed. After
**apply**, append `applied` ids.
```json
{
"evaluated_at": "ISO-8601 UTC",
"mode": "full|delta",
"harnesses": ["grok"],
"versions": { "grok": "0.2.104" },
"last_changelog_cursor": { "grok": "0.2.104" },
"last_changelog_content_hash": "sha256…",
"last_changelog_reviewed_at": "ISO-8601 UTC",
"last_release_check": {
"current": "0.2.104",
"latest": "0.2.104",
"update_available": false,
"channel": "alpha",
"checked_at": "ISO-8601 UTC"
},
"preferences": {},
"scores": {},
"open_items": [],
"applied": [],
"project_roots": ["~/projects"]
}
Suggest recurrence (do not schedule without asking via question tool):
- When
update_available: upgrade then re-run /harness-optimize
- After major CLI upgrade:
/harness-optimize delta
- Monthly: full
- Wire into
/loop / loopy only if the user approves
Scripts
| Script | Role |
|---|
scripts/run-diagnostics.sh | All read-only diagnostics in one go |
scripts/discover.sh / inventory.sh | Basics |
scripts/recurrence-check.sh | full vs delta mode hint |
scripts/release-check.sh | grok update --check; suggest upgrade when available |
scripts/changelog-delta.sh | Full bullet catalog since last run + suggestions |
scripts/env-check.sh / launcher-matrix.sh / subagent-env.sh | Secret load paths |
scripts/permission-audit.sh | always-approve / deny tables |
scripts/mcp-doctor.sh / mcp-matrix.sh / plugin-cost.sh / tool-budget-cutlist.sh | MCP health + cuts |
scripts/xapi-health.sh | Doctor + live /2/users/me soft check |
scripts/dual-rules.sh / dual-rules-plan.sh | Dual-load severity + pointer plans |
scripts/restore-kit.sh / write-report.sh | Backups + redacted report archive |
templates/ + recipes/ | Deny starter + apply id → patch recipes |
Prefer scripts for counting and path discovery; use model judgment for interview, scoring, and rec text.
See references/features.md and recipes/README.md.
Related tools (compose)
| Tool | Use for |
|---|
/cli-skills-audit | Skill nesting, frontmatter, cross-tool drift |
/harness-audit | Quick numeric repo scorecard only |
agent-token-audit skill | Claude agent bloat archival |
claude-automation-recommender | Project automation ideas from stack signals |
finding-unknowns | Interview discipline (already embedded here) |
Anti-patterns
- Implementing before showing the report (including “quick Critical fixes”)
- Assuming apply permission from yolo, goals, or “optimize everything”
- Skipping
ask_user_question and treating silence or free-text as consent
- Judging setup from memory of old defaults without reading this week's changelog
- Archiving agents/skills without backup + explicit confirm
- "Optimize" by adding more always-on context
- Same kitchen-sink hooks on experiment toys and factory SaaS
- Averaging contradictory multi-harness patterns instead of picking a primary
- Printing MCP secrets or auth tokens into the report
When to load references
Install note (Grok-first)
Preferred: native plugin
grok plugin install <github-user>/harness-optimize --trust
grok plugin enable harness-optimize
Then /harness-optimize or natural language. State/reports prefer GROK_PLUGIN_DATA.
Do not install into Claude's skill tree unless the user asks.
