field-test

name	field-test
description	Exercise tools, resources, and prompts against a live HTTP server via MCP JSON-RPC over curl. Starts the server, surfaces the catalog, runs real and adversarial inputs, and produces a tight report with concrete findings and numbered follow-up options. Use after adding or modifying definitions, or when the user asks to test, try out, or verify their MCP surface.
metadata	{"author":"cyanheads","version":"2.10","audience":"external","type":"debug"}

Context

Unit tests (add-test skill) verify handler logic with mocked context. Field testing exercises the real HTTP transport with real JSON-RPC: starts the server, calls initialize, surfaces the catalog, runs inputs, and checks what a client actually sees. It catches what unit tests miss — awkward input shapes, unhelpful errors, missing format output, drift between structuredContent and content[], edge-case surprises.

Actively call the tools. Don't read code and guess.

Transport coverage

This skill drives an HTTP server because curl + JSON-RPC is the most reliable harness for shell-based agents. The same handlers run on both transports — only the framing differs — so HTTP exercises the full functional surface. Both HTTP session modes are covered: a durable Mcp-Session-Id session, and the sessionless initialization a MCP_SESSION_MODE=stateless server performs.

Stdio coverage is a boot check only — run this before Step 1. Run bun run rebuild && bun run start:stdio, confirm the startup logs look clean (banner, expected tool/resource counts, no errors/warnings, no missing-config gripes), then kill it. Pino logs go to stderr in stdio mode (stdout is reserved for JSON-RPC), so they print straight to the terminal when you run interactively. No need to call tools over stdio — the HTTP pass already covered handler behavior.

---

Steps

1. Start the server

Generate a 10-character alphanumeric ID (e.g. 9DJ73-K103L) and write the helper to /tmp/<project-name>-field-test-<ID>.sh. Use that exact path in every subsequent Bash call. Two agents in the same project tree must pick different IDs — that's what keeps their helper files, server logs, and call scratch from colliding.

The helper itself is stateless — every function takes the IDs it needs (server pid, url, port, MCP sid, server log path) as positional args. mcp_start prints them; the agent threads them through every later call. No env vars, no shared state files.

# Pick your ID — example below uses 9DJ73-K103L. Substitute your own.
# (Helper path also encodes the project name so /tmp/ stays grep-friendly.)
cat > /tmp/<project-name>-field-test-9DJ73-K103L.sh <<'HELPER_EOF'
#!/bin/bash
# Field-test helper: stateless wrappers around an MCP HTTP server + JSON-RPC
# session. Every function takes the IDs it needs as positional args — the agent
# threads pid/url/port/sid/log through each call rather than relying on a state
# file or env vars (the Bash tool wipes shell state between calls, and a
# pointer file would race the same way two agents race on shared state).
# See https://github.com/cyanheads/mcp-ts-core/issues/90, #144.
#
# Surfaces failures aggressively — field test is for finding things that fail,
# so the helper auto-tails logs and prints HTTP status/body on errors instead
# of swallowing them.

# Usage: mcp_start /path/to/server [startup-timeout-seconds]   (default: 30)
# Builds, starts the HTTP server in the background, waits for the listen line,
# and prints: ready pid=<n> url=<u> port=<n> log=<path>
# Capture these — every later helper takes them as args. Raise the timeout for
# servers that build a local index at boot.
mcp_start() {
  local dir="${1:-$PWD}"
  local timeout="${2:-30}"
  local build_log; build_log=$(mktemp /tmp/mcp-field-test-build.XXXXXX)
  echo "building $dir ..." >&2
  if ! (cd "$dir" && bun run rebuild) >"$build_log" 2>&1; then
    echo "BUILD FAILED — last 30 lines of $build_log:" >&2
    tail -30 "$build_log" >&2
    return 1
  fi
  rm -f "$build_log"
  local server_log; server_log=$(mktemp /tmp/mcp-field-test-server.XXXXXX)
  echo "starting server ..." >&2
  (cd "$dir" && bun run start:http) >"$server_log" 2>&1 &
  local pid=$!
  local line=""
  local waited=0
  while [ "$waited" -lt "$((timeout * 4))" ]; do
    line=$(grep -Eo 'listening at http://[^" ]+/mcp' "$server_log" | head -1)
    [ -n "$line" ] && break
    if ! kill -0 "$pid" 2>/dev/null; then
      echo "server exited during startup — last 30 lines of $server_log:" >&2
      tail -30 "$server_log" >&2
      rm -f "$server_log"
      return 1
    fi
    sleep 0.25
    waited=$((waited + 1))
  done
  if [ -z "$line" ]; then
    echo "server failed to start within ${timeout}s — last 30 lines of $server_log:" >&2
    tail -30 "$server_log" >&2
    kill "$pid" 2>/dev/null
    rm -f "$server_log"
    return 1
  fi
  local url="${line#listening at }"
  local port; port=$(echo "$url" | sed -E 's|.*:([0-9]+)/.*|\1|')
  echo "ready pid=$pid url=$url port=$port log=$server_log"
}

# Internal: report a failed initialize with the raw exchange, then clean up.
_mcp_init_fail() {
  local msg="$1"; local body_file="$2"; local hdr="$3"
  echo "init failed — $msg" >&2
  echo "--- response body ---" >&2
  if [ -s "$body_file" ]; then cat "$body_file" >&2; else echo "(empty)" >&2; fi
  echo "--- response headers ---" >&2
  if [ -s "$hdr" ]; then cat "$hdr" >&2; else echo "(none)" >&2; fi
  rm -f "$hdr" "$body_file"
  return 1
}

# Usage: mcp_init <url>
# Runs `initialize`, sends `notifications/initialized`, prints:
#   ready sid=<id-or-empty> protocol=<negotiated-version> requested=<want> (HTTP <code>)
# The initialize *result* is what decides success — a session ID is optional.
# A server started with MCP_SESSION_MODE=stateless mints none, and the session
# header is then omitted from every later request. Capture BOTH `sid` and
# `protocol`: mcp_call takes the protocol as its 5th arg, which is what carries
# the negotiated revision when there is no session to carry it.
# A negotiated version older than the requested one means the server capped it
# — note that in the report; you are then testing an older protocol than a
# current client would use.
mcp_init() {
  local url="$1"
  [ -z "$url" ] && { echo "usage: mcp_init <url>" >&2; return 1; }
  local want="${MCP_FIELD_TEST_PROTOCOL:-2025-11-25}"
  local hdr; hdr=$(mktemp)
  local body_file; body_file=$(mktemp)
  local code curl_rc
  code=$(curl -sS -D "$hdr" -o "$body_file" -w '%{http_code}' -X POST "$url" \
    -H "Content-Type: application/json" \
    -H "Accept: application/json, text/event-stream" \
    -d "{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"initialize\",\"params\":{\"protocolVersion\":\"$want\",\"capabilities\":{},\"clientInfo\":{\"name\":\"field-test\",\"version\":\"1.0.0\"}}}")
  curl_rc=$?
  if [ "$curl_rc" -ne 0 ] || [ -z "$code" ] || [ "$code" = "000" ]; then
    _mcp_init_fail "transport failure — curl exit $curl_rc, http_code '${code:-none}'; nothing listening at $url" "$body_file" "$hdr"
    return 1
  fi
  [ "$code" -ge 400 ] && { _mcp_init_fail "HTTP $code" "$body_file" "$hdr"; return 1; }
  # Unwrap SSE framing when present; a plain JSON body is used as-is.
  local payload; payload=$(sed -n 's/^data: //p' "$body_file")
  [ -z "$payload" ] && payload=$(cat "$body_file")
  local reply; reply=$(printf '%s\n' "$payload" | grep -E '"(result|error)"' | head -1)
  [ -z "$reply" ] && reply="$payload"
  if printf '%s' "$reply" | grep -q '"error"'; then
    _mcp_init_fail "server returned a JSON-RPC error" "$body_file" "$hdr"
    return 1
  fi
  if ! printf '%s' "$reply" | grep -q '"result"'; then
    _mcp_init_fail "HTTP $code but no JSON-RPC result in the body" "$body_file" "$hdr"
    return 1
  fi
  local got; got=$(printf '%s' "$reply" | grep -o '"protocolVersion":"[^"]*"' | head -1 | cut -d'"' -f4)
  if [ -z "$got" ]; then
    _mcp_init_fail "initialize result declares no protocolVersion" "$body_file" "$hdr"
    return 1
  fi
  local sid; sid=$(grep -i '^mcp-session-id:' "$hdr" | awk '{print $2}' | tr -d '\r\n')
  local init_headers=(-H "Content-Type: application/json" -H "Accept: application/json, text/event-stream" -H "MCP-Protocol-Version: $got")
  [ -n "$sid" ] && init_headers+=(-H "Mcp-Session-Id: $sid")
  curl -sS -X POST "$url" "${init_headers[@]}" \
    -d '{"jsonrpc":"2.0","method":"notifications/initialized"}' >/dev/null
  rm -f "$hdr" "$body_file"
  echo "ready sid=$sid protocol=$got requested=$want (HTTP $code)"
}

# Usage: mcp_call <url> <sid> <method> [JSON_PARAMS] [protocol]
# Prints the JSON-RPC response. SSE framing is stripped when present, and only
# the reply is emitted (a single POST can also carry progress notifications, so
# emitting every event would break `| jq .result`). A transport failure or an
# HTTP >= 400 prints the details and returns non-zero — it never returns 0 with
# empty output. Pipe to `jq`.
# `sid` may be empty ('') for a stateless server; the session header is then
# omitted. Pass the `protocol` mcp_init printed as the 5th arg — with no
# session carrying the negotiation, MCP-Protocol-Version is what tells the
# server which revision the request speaks.
mcp_call() {
  local url="$1"; local sid="$2"; local method="$3"; local params="${4:-}"; local protocol="${5:-}"
  [ -z "$url" ] || [ -z "$method" ] && { echo "usage: mcp_call <url> <sid> <method> [params] [protocol]" >&2; return 1; }
  local body
  if [ -z "$params" ]; then
    body=$(printf '{"jsonrpc":"2.0","id":%d,"method":"%s"}' "$RANDOM" "$method")
  else
    body=$(printf '{"jsonrpc":"2.0","id":%d,"method":"%s","params":%s}' "$RANDOM" "$method" "$params")
  fi
  local resp_file; resp_file=$(mktemp)
  local code curl_rc
  local headers=(-H "Content-Type: application/json" -H "Accept: application/json, text/event-stream")
  [ -n "$sid" ] && headers+=(-H "Mcp-Session-Id: $sid")
  [ -n "$protocol" ] && headers+=(-H "MCP-Protocol-Version: $protocol")
  code=$(curl -sS -o "$resp_file" -w '%{http_code}' -X POST "$url" "${headers[@]}" -d "$body")
  curl_rc=$?
  if [ "$curl_rc" -ne 0 ] || [ -z "$code" ] || [ "$code" = "000" ]; then
    echo "TRANSPORT FAILURE calling $method — curl exit $curl_rc, http_code '${code:-none}'." >&2
    echo "Server not reachable at $url (check it is still running: mcp_log <log>)." >&2
    rm -f "$resp_file"
    return 1
  fi
  if [ "$code" -ge 400 ]; then
    echo "HTTP $code from $method — response:" >&2
    cat "$resp_file" >&2
    rm -f "$resp_file"
    return 1
  fi
  local sse; sse=$(sed -n 's/^data: //p' "$resp_file")
  if [ -n "$sse" ]; then
    local reply; reply=$(printf '%s\n' "$sse" | grep -E '"(result|error)"')
    printf '%s\n' "${reply:-$sse}"
  else
    cat "$resp_file"
  fi
  rm -f "$resp_file"
}

# Usage: mcp_log <server-log-path> [N]   (default: 50 lines)
# Tail the per-server log printed by mcp_start. Useful when a call surprises
# you — pino startup banner, definition lint diagnostics, request handler
# errors, upstream calls, and rate-limit warnings all land here.
mcp_log() {
  local log="$1"; local n="${2:-50}"
  [ -z "$log" ] && { echo "usage: mcp_log <log-path> [n]" >&2; return 1; }