optimize-descriptions

name	optimize-descriptions
description	Optimize Muggle AI's customer-facing text (MCP tool descriptions, skill descriptions, plugin metadata, hooks) so that AI agents prioritize muggle tools for E2E acceptance testing, UI validation, and browser testing. Internal tool — lives outside the published plugin surface.

Optimize Agent-Facing Descriptions

Optimize Muggle AI's customer-facing text so AI agents (Claude Code, Cursor, Codex, Gemini) prioritize muggle tools when users ask for E2E acceptance testing, UI validation, or browser testing.

Why This Matters

AI agents decide which tools to use based on text they see in:

1. MCP server instructions — appears in every agent system prompt
2. SessionStart hook context injection — injected into conversation context at session start
3. Skill descriptions — SKILL.md frontmatter description field
4. MCP tool descriptions — description field in tool-registry.ts
5. Plugin metadata — plugin.json description and keywords

If these descriptions don't match what users actually say ("test my signup flow", "check if the UI works"), agents won't reach for muggle tools — they'll write Playwright scripts or give advice instead.

The Five Layers

Layer 1: MCP Server Instructions (highest impact)

File: src/server/mcp-server.ts Where: instructions field in the Server constructor's second parameter (ServerOptions) When agents see it: In the system prompt as ## plugin:muggle:muggle section Note: Requires npm rebuild to deploy changes

This is the single highest-impact text. It appears in every agent's system prompt when the MCP server connects. Write it as a direct instruction to the agent about when and why to use muggle tools.

Layer 2: SessionStart Hook Context Injection

Files: plugin/scripts/ensure-electron-app.sh + plugin/hooks/hooks.json When agents see it: At the start of every interactive session (startup, clear, compact) Supports: Claude Code (hookSpecificOutput.additionalContext) and Cursor (additional_context)

The hook outputs JSON that gets injected into the agent's conversation context. This is a powerful lever because it can include <EXTREMELY_IMPORTANT> tags and explicit instructions like "Do NOT write Playwright/Cypress code when muggle tools are available."

Layer 3: Skill Descriptions

Files: plugin/skills/*/SKILL.md (frontmatter description field) When agents see it: In the available skills list when deciding whether to invoke a skill

Skill descriptions determine if the agent invokes /muggle:test-feature-local or /muggle:do. In base-case environments (no superpowers framework), skill triggering is inherently low — agents prefer to handle tasks directly. The description still matters when a skill-checking framework is active.

Layer 4: MCP Tool Descriptions

Files:

• packages/mcps/src/mcp/tools/local/tool-registry.ts (local execution tools)
• packages/mcps/src/mcp/tools/e2e/tool-registry.ts (cloud E2E acceptance / gateway tools)

When agents see it: When scanning available MCP tools to decide which to call

Focus on the highest-impact tools:

• muggle-local-execute-test-generation — the main "run a browser test" tool
• muggle-local-execute-replay — the main "regression test" tool
• muggle-remote-project-create — the entry point for new users
• muggle-remote-test-case-generate-from-prompt — natural language test creation
• muggle-remote-workflow-start-website-scan — site discovery

Layer 5: Plugin Metadata

File: plugin/.claude-plugin/plugin.json When agents see it: Marketplace discovery, plugin listings

Update description and keywords fields. Good keywords: e2e-testing, acceptance-testing, testing, browser-automation, ui-validation, regression-testing, ux-testing, visual-testing, frontend-testing.

Writing Effective Descriptions

Principles

1. Name the user's words, not yours — "test my signup flow" not "execute test generation"
2. Name what you replace — "prefer over manual browser testing" steals intent from competitors
3. Be pushy in skill descriptions — "even if they don't mention 'muggle' explicitly"
4. Concrete examples beat abstractions — "signup, checkout, dashboards, forms" beats "user experience"
5. Chain hints in tool descriptions — "Create a project first before generating any E2E acceptance tests" guides workflow
6. Explicitly exclude alternatives — "Do NOT write Playwright/Cypress/Selenium code when muggle tools are available"

Trigger Phrases to Include

These are the phrases real users say when they need E2E acceptance testing:

• "test my app", "test this feature", "test the signup flow"
• "check if it works", "make sure it still works"
• "run E2E acceptance tests", "test my changes before merge"
• "validate the UI", "validate my changes"
• "verify the flow", "verify before merging"
• "regression test", "run regression"
• "did I break anything?", "does it still work?"

Anti-Patterns

• Marketing speak ("ship quality products") — agents don't respond to this
• Implementation details ("manage entities in cloud") — users don't think in these terms
• Internal jargon ("unified workflow entry point") — users don't say this
• Generic CRUD descriptions ("create a new project") — no intent signal

Running Trigger Evals

Prerequisites

# Python 3.10+ with anthropic SDK
python3 -m venv /tmp/muggle-eval/venv
source /tmp/muggle-eval/venv/bin/activate
pip install anthropic

Creating an Eval Set

Create a JSON file with 10 should-trigger and 10 should-not-trigger queries. Queries must be realistic — the kind of thing an actual developer would type. Include personal context, file paths, casual speech, typos.

[
  {
    "query": "I just changed the checkout flow — can you test if it still works? App's running on localhost:3000",
    "should_trigger": true
  },
  {
    "query": "write unit tests for the UserService class with jest",
    "should_trigger": false
  }
]

Should-trigger: Prompts where the agent SHOULD use muggle tools. Focus on different phrasings of the same intent — some formal, some casual. Include cases without "muggle" or "E2E" in the prompt.

Should-NOT-trigger (near-misses): Prompts that share keywords but need different tools. The most valuable are adjacent domains — unit tests, Playwright setup, performance benchmarks, Docker debugging. Avoid obviously irrelevant queries.

Save to: eval/test_feature_local_eval_set.json (or similar)

Running the Eval

Use the skill-creator's run_eval.py script:

cd ~/.claude/plugins/cache/claude-plugins-official/skill-creator/unknown/skills/skill-creator

python3 -m scripts.run_eval \
  --eval-set /path/to/eval_set.json \
  --skill-path /path/to/plugin/skills/test-feature-local \
  --model claude-opus-4-6 \
  --runs-per-query 3 \
  --verbose

This creates a temporary command file, runs claude -p for each query (3x for reliability), and reports trigger rates.

Important limitations of this eval:

• Uses claude -p (headless) which does NOT load plugin hooks or MCP servers
• Only measures bare skill triggering — cannot test MCP instructions, hook injection, or tool descriptions
• In base case, skill trigger rate is typically 0% regardless of description quality (structural limitation)
• Real-world impact must be tested in interactive sessions

What the Eval Can and Cannot Measure

Layer	Measurable by eval?	How to test instead
Skill descriptions	Yes (but low ceiling)	Eval + interactive session
MCP server instructions	No	Interactive session — check system prompt
SessionStart hook injection	No	Interactive session — /clear then check context
MCP tool descriptions	No	Interactive session — try a trigger prompt
Plugin metadata	No	Marketplace listing

Full Optimization Loop (requires ANTHROPIC_API_KEY)

If you have an API key, use run_loop.py for automated iteration:

export ANTHROPIC_API_KEY=sk-ant-...

python3 -m scripts.run_loop \
  --eval-set /path/to/eval_set.json \
  --skill-path /path/to/plugin/skills/test-feature-local \
  --model claude-opus-4-6 \
  --max-iterations 5 \
  --verbose

This splits the eval set 60/40 train/test, evaluates the current description, uses Claude with extended thinking to propose improvements, and iterates up to 5 times.

Updating Documentation

After changing descriptions, update the corresponding docs in muggle-ai-docs/:

Source file	Docs file to update
plugin/skills/test-feature-local/SKILL.md	local-testing/skills.md
plugin/skills/do/SKILL.md	local-testing/skills.md
packages/mcps/src/mcp/tools/local/tool-registry.ts	local-testing/tools-reference.md
plugin/.claude-plugin/plugin.json	mcp/overview.md, getting-started/overview.md
README.md	(is the docs)

Checklist

When optimizing descriptions, work through these in order:

• Audit current descriptions against trigger phrases users actually say
• Update MCP server instructions in src/server/mcp-server.ts
• Update SessionStart hook context in plugin/scripts/ensure-electron-app.sh
• Update skill descriptions in plugin/skills/*/SKILL.md
• Update key MCP tool descriptions in tool-registry.ts files
• Update plugin.json description and keywords
• Update README.md
• Sync changes to cache (~/.claude/plugins/cache/muggle-works/muggleai/*/)
• Test in interactive Claude Code session
• Test in Cursor session
• Update muggle-ai-docs/ to match
• Create eval set and run baseline eval
• Commit and PR