mcp-server-tester

Run AI-driven tool-metadata optimization experiments for MCP servers using runVariantExperiment. Use when asked to improve tool descriptions, input schemas, or parameter descriptions, fix tool triggering, optimize tool discoverability, or run variant experiments against an eval dataset. Produces a measured, regression-guarded improvement proposal — never edits server source unless explicitly asked.

Generate LLM host simulation evals for MCP servers. Use when asked to test tool discoverability, write mcp_host evals, or validate tool descriptions with real LLM calls. Produces eval datasets and test runners where an LLM discovers and calls tools from natural language scenarios.

Reference guide for @gleanwork/mcp-server-tester — the Playwright-based testing and evaluation framework for MCP servers. Covers import paths, all 11 matchers, transport config, eval datasets, reporter setup, CLI commands, auth patterns, and common anti-patterns. Use when working with MCP server tests or evals.

Generate data-driven eval datasets for MCP server testing. Use when asked to create evals, evaluation datasets, or data-driven tests for MCP tools. Produces JSON eval datasets and Playwright test runners for @gleanwork/mcp-server-tester.

Generate Playwright tests for MCP server tools. Use when asked to write MCP tests, test MCP tools, or create test files for an MCP server. Produces direct-mode tests using @gleanwork/mcp-server-tester fixtures.