field-test

## Context Unit tests (`add-test` skill) verify handler logic with mocked context. Field testing exercises the real HTTP transport with real JSON-RPC: starts the server, calls `initialize`, surfaces the catalog, runs inputs, and checks what a client actually sees. It catches what unit tests miss — awkward input shapes, unhelpful errors, missing format output, drift between `structuredContent` and `content[]`, edge-case surprises. **Actively call the tools. Don't read code and guess.** ### Transport coverage This skill drives an HTTP server because curl + JSON-RPC is the most reliable harness for shell-based agents. The same handlers run on both transports — only the framing differs — so HTTP exercises the full functional surface. Both HTTP session modes are covered: a durable `Mcp-Session-Id` session, and the sessionless initialization a `MCP_SESSION_MODE=stateless` server performs. **Stdio coverage is a boot check only — run this before Step 1.** Run `bun run rebuild && bun run start:stdio`, confirm the startup logs look clean (banner, expected tool/resource counts, no errors/warnings, no missing-config gripes), then kill it. Pino logs go to stderr in stdio mode (stdout is reserved for JSON-RPC), so they print straight to the terminal when you run interactively. No need to call tools over stdio — the HTTP pass already covered handler behavior. --- ## Steps ### 1. Start the server Generate a 10-character alphanumeric ID (e.g. `9DJ73-K103L`) and write the helper to `/tmp/<project-name>-field-test-<ID>.sh`. Use that exact path in every subsequent Bash call. **Two agents in the same project tree must pick different IDs** — that's what keeps their helper files, server logs, and call scratch from colliding. The helper itself is **stateless** — every function takes the IDs it needs (server `pid`, `url`, `port`, MCP `sid`, server log path) as positional args. `mcp_start` prints them; the agent threads them through every later call. No env vars, no shared state files. ```bash # Pick your ID — example below uses 9DJ73-K103L. Substitute your own. # (Helper path also encodes the project name so /tmp/ stays grep-friendly.) cat > /tmp/<project-name>-field-test-9DJ73-K103L.sh <<'HELPER_EOF' #!/bin/bash # Field-test helper: stateless wrappers around an MCP HTTP server + JSON-RPC # session. Every function takes the IDs it needs as positional args — the agent # threads pid/url/port/sid/log through each call rather than relying on a state # file or env vars (the Bash tool wipes shell state between calls, and a # pointer file would race the same way two agents race on shared state). # See https://github.com/cyanheads/mcp-ts-core/issues/90, #144. # # Surfaces failures aggressively — field test is for finding things that fail, # so the helper auto-tails logs and prints HTTP status/body on errors instead # of swallowing them. # Usage: mcp_start /path/to/server [startup-timeout-seconds] (default: 30) # Builds, starts the HTTP server in the background, waits for the listen line, # and prints: ready pid=<n> url=<u> port=<n> log=<path> # Capture these — every later helper takes them as args. Raise the timeout for # servers that build a local index at boot. mcp_start() { local dir="${1:-$PWD}" local timeout="${2:-30}" local build_log; build_log=$(mktemp /tmp/mcp-field-test-build.XXXXXX) echo "building $dir ..." >&2 if ! (cd "$dir" && bun run rebuild) >"$build_log" 2>&1; then echo "BUILD FAILED — last 30 lines of $build_log:" >&2 tail -30 "$build_log" >&2 return 1 fi rm -f "$build_log" local server_log; server_log=$(mktemp /tmp/mcp-field-test-server.XXXXXX) echo "starting server ..." >&2 (cd "$dir" && bun run start:http) >"$server_log" 2>&1 & local pid=$! local line="" local waited=0 while [ "$waited" -lt "$((timeout * 4))" ]; do line=$(grep -Eo 'listening at http://[^" ]+/mcp' "$server_log" | head -1) [ -n "$line" ] && break if ! kill -0 "$pid" 2>/dev/null; then echo "server exited during startup — last 30 lines of $server_log:" >&2 tail -30 "$server_log" >&2 rm -f "$server_log" return 1 fi sleep 0.25 waited=$((waited + 1)) done if [ -z "$line" ]; then echo "server failed to start within ${timeout}s — last 30 lines of $server_log:" >&2 tail -30 "$server_log" >&2 kill "$pid" 2>/dev/null rm -f "$server_log" return 1 fi local url="${line#listening at }" local port; port=$(echo "$url" | sed -E 's|.*:([0-9]+)/.*|\1|') echo "ready pid=$pid url=$url port=$port log=$server_log" } # Internal: report a failed initialize with the raw exchange, then clean up. _mcp_init_fail() { local msg="$1"; local body_file="$2"; local hdr="$3" echo "init failed — $msg" >&2 echo "--- response body ---" >&2 if [ -s "$body_file" ]; then cat "$body_file" >&2; else echo "(empty)" >&2; fi echo "--- response headers ---" >&2 if [ -s "$hdr" ]; then cat "$hdr" >&2; else echo "(none)" >&2; fi rm -f "$hdr" "$body_file" return 1 } # Usage: mcp_init <url> # Runs `initialize`, sends `notifications/initialized`, prints: # ready sid=<id-or-empty> protocol=<negotiated-version> requested=<want> (HTTP <code>) # The initialize *result* is what decides success — a session ID is optional. # A server started with MCP_SESSION_MODE=stateless mints none, and the session # header is then omitted from every later request. Capture BOTH `sid` and # `protocol`: mcp_call takes the protocol as its 5th arg, which is what carries # the negotiated revision when there is no session to carry it. # A negotiated version older than the requested one means the server capped it # — note that in the report; you are then testing an older protocol than a # current client would use. mcp_init() { local url="$1" [ -z "$url" ] && { echo "usage: mcp_init <url>" >&2; return 1; } local want="${MCP_FIELD_TEST_PROTOCOL:-2025-11-25}" local hdr; hdr=$(mktemp) local body_file; body_file=$(mktemp) local code curl_rc code=$(curl -sS -D "$hdr" -o "$body_file" -w '%{http_code}' -X POST "$url" \ -H "Content-Type: application/json" \ -H "Accept: application/json, text/event-stream" \ -d "{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"initialize\",\"params\":{\"protocolVersion\":\"$want\",\"capabilities\":{},\"clientInfo\":{\"name\":\"field-test\",\"version\":\"1.0.0\"}}}") curl_rc=$? if [ "$curl_rc" -ne 0 ] || [ -z "$code" ] || [ "$code" = "000" ]; then _mcp_init_fail "transport failure — curl exit $curl_rc, http_code '${code:-none}'; nothing listening at $url" "$body_file" "$hdr" return 1 fi [ "$code" -ge 400 ] && { _mcp_init_fail "HTTP $code" "$body_file" "$hdr"; return 1; } # Unwrap SSE framing when present; a plain JSON body is used as-is. local payload; payload=$(sed -n 's/^data: //p' "$body_file") [ -z "$payload" ] && payload=$(cat "$body_file") local reply; reply=$(printf '%s\n' "$payload" | grep -E '"(result|error)"' | head -1) [ -z "$reply" ] && reply="$payload" if printf '%s' "$reply" | grep -q '"error"'; then _mcp_init_fail "server returned a JSON-RPC error" "$body_file" "$hdr" return 1 fi if ! printf '%s' "$reply" | grep -q '"result"'; then _mcp_init_fail "HTTP $code but no JSON-RPC result in the body" "$body_file" "$hdr" return 1 fi local got; got=$(printf '%s' "$reply" | grep -o '"protocolVersion":"[^"]*"' | head -1 | cut -d'"' -f4) if [ -z "$got" ]; then _mcp_init_fail "initialize result declares no protocolVersion" "$body_file" "$hdr" return 1 fi local sid; sid=$(grep -i '^mcp-session-id:' "$hdr" | awk '{print $2}' | tr -d '\r\n') local init_headers=(-H "Content-Type: application/json" -H "Accept: application/json, text/event-stream" -H "MCP-Protocol-Version: $got") [ -n "$sid" ] && init_headers+=(-H "Mcp-Session-Id: $sid") curl -sS -X POST "$url" "${init_headers[@]}" \ -d '{"jsonrpc":"2.0","method":"notifications/initialized"}' >/dev/null rm -f "$hdr" "$body_file" echo "ready sid=$sid protocol=$got requested=$want (HTTP $code)" } # Usage: mcp_call <url> <sid> <method> [JSON_PARAMS] [protocol] # Prints the JSON-RPC response. SSE framing is stripped when present, and only # the reply is emitted (a single POST can also carry progress notifications, so # emitting every event would break `| jq .result`). A transport failure or an # HTTP >= 400 prints the details and returns non-zero — it never returns 0 with # empty output. Pipe to `jq`. # `sid` may be empty ('') for a stateless server; the session header is then # omitted. Pass the `protocol` mcp_init printed as the 5th arg — with no # session carrying the negotiation, MCP-Protocol-Version is what tells the # server which revision the request speaks. mcp_call() { local url="$1"; local sid="$2"; local method="$3"; local params="${4:-}"; local protocol="${5:-}" [ -z "$url" ] || [ -z "$method" ] && { echo "usage: mcp_call <url> <sid> <method> [params] [protocol]" >&2; return 1; } local body if [ -z "$params" ]; then body=$(printf '{"jsonrpc":"2.0","id":%d,"method":"%s"}' "$RANDOM" "$method") else body=$(printf '{"jsonrpc":"2.0","id":%d,"method":"%s","params":%s}' "$RANDOM" "$method" "$params") fi local resp_file; resp_file=$(mktemp) local code curl_rc local headers=(-H "Content-Type: application/json" -H "Accept: application/json, text/event-stream") [ -n "$sid" ] && headers+=(-H "Mcp-Session-Id: $sid") [ -n "$protocol" ] && headers+=(-H "MCP-Protocol-Version: $protocol") code=$(curl -sS -o "$resp_file" -w '%{http_code}' -X POST "$url" "${headers[@]}" -d "$body") curl_rc=$? if [ "$curl_rc" -ne 0 ] || [ -z "$code" ] || [ "$code" = "000" ]; then echo "TRANSPORT FAILURE calling $method — curl exit $curl_rc, http_code '${code:-none}'." >&2 echo "Server not reachable at $url (check it is still running: mcp_log <log>)." >&2 rm -f "$resp_file" return 1 fi if [ "$code" -ge 400 ]; then echo "HTTP $code from $method — response:" >&2 cat "$resp_file" >&2 rm -f "$resp_file" return 1 fi local sse; sse=$(sed -n 's/^data: //p' "$resp_file") if [ -n "$sse" ]; then local reply; reply=$(printf '%s\n' "$sse" | grep -E '"(result|error)"') printf '%s\n' "${reply:-$sse}" else cat "$resp_file" fi rm -f "$resp_file" } # Usage: mcp_log <server-log-path> [N] (default: 50 lines) # Tail the per-server log printed by mcp_start. Useful when a call surprises # you — pino startup banner, definition lint diagnostics, request handler # errors, upstream calls, and rate-limit warnings all land here. mcp_log() { local log="$1"; local n="${2:-50}" [ -z "$log" ] && { echo "usage: mcp_log <log-path> [n]" >&2; return 1; }