| name | mcp-conformance |
| description | Use when running, interpreting, baselining, or wiring the MCP conformance harness for a client, remote HTTP server, authorization flow, or CI pipeline. |
| version | 0.1.0 |
MCP Conformance
Operating rules
- Server mode is HTTP-only. It requires
--url; it cannot launch or test a stdio server. Stdio server testing remains open issue #258.
- Pin
@modelcontextprotocol/conformance@0.1.16 for this skill. Do not silently switch to 0.2.0-alpha.
- Start the server, then run the harness against the real endpoint. A test file alone is not end-to-end evidence.
- Keep expected failures empty unless a known upstream issue has a narrow, reviewed waiver.
Compatibility boundary: version 0.1.16 rejects --spec-version 2026-07-28; its valid filters are 2025-03-26, 2025-06-18, 2025-11-25, draft, and extension. Do not present it as a 2026 conformance gate. Use it for supported legacy/draft coverage and separately test the modern request contract.
Choose mode and suite
| Mode | Target | Suites |
|---|
server | A remote HTTP server URL | active (default), all, pending |
client | A client command launched by the harness | all, core, extensions, auth, metadata, backcompat, draft, sep-835 |
authorization is not a CLI mode. Test authorization behavior through a supported client/server scenario. Server mode does not accept the client-only --timeout option.
Server workflow
npx @modelcontextprotocol/conformance@0.1.16 server \
--url http://localhost:3000/mcp \
--suite active \
--spec-version 2025-11-25 \
--verbose
Use --scenario, --suite, --spec-version, --expected-failures, --output-dir, and --verbose deliberately. --timeout is client-only. Every scenario also reports wire-schema validation or harness errors; distinguish those from the server's functional result.
Expected failures
server:
- server-stateless:some-check-id
client: []
The top level may contain only server and client, with arrays of strings. A mapping form such as - scenario: check-id is invalid. Do not waive both a scenario and one of its checks.
Exit status is intentionally strict:
| Result | Exit |
|---|
| Passing, no baseline entry | 0 |
| Failing, baseline entry | 0 |
| Failing, no baseline entry | 1 |
| Passing, baseline entry | 1 — stale waiver |
CI
Use the root composite action, modelcontextprotocol/conformance@v0.1.16. Its relevant inputs are mode, url, command, expected-failures, suite, scenario, timeout, verbose, and node-version; the action defaults to Node 22. Keep a zero-waiver baseline when possible.
Reporting
Report the pinned harness version, its supported spec filter, mode, suite/scenario, target URL or client command, exit status, output directory, every unbaselined failure, and every stale waiver. Use scripts/run_conformance.py to select server/client form and summarize emitted checks.json files.
References
references/versions.md — harness version and verification commandscripts/run_conformance.py — thin runner and result summary
