dev-mcp

# MCP in Capsem Capsem has two MCP entry points: 1. **`@capsem/mcp`** (host): a standalone TypeScript stdio server. It presents typed tools and uses `@capsem/sdk` for authenticated gateway HTTP. It does not open the service UDS, discover local services, or read VM runtime state. 2. **Guest MCP relay + confined proxy endpoint**: bridges AI agents inside a VM to built-in and external MCP servers through bounded framed records on vsock port 5002. The native installer does not install Node.js or download the npm host package. The guest relay, aggregator, and built-in Rust components remain native product components. ## Using the npm MCP tools for fast debugging Install and register the package with an explicit gateway URL, bearer token, and transport timeout: ```sh npm install --global @capsem/mcp CAPSEM_GATEWAY_TOKEN="$(cat ~/.capsem/run/gateway.token)" \ capsem-mcp --gateway-url http://127.0.0.1:19222 --timeout-ms 30000 ``` The token comes from `CAPSEM_GATEWAY_TOKEN` or `--token-file`; `--token` is refused because argv is world-readable. The bearer token belongs only to the host MCP process; never copy it into a VM, container, guest tool argument, or workload. stdout is protocol-only and sanitized diagnostics use stderr. The current registry is defined in `mcp/typescript/src/host-tools.ts`, `network-tools.ts`, and `mcp-tools.ts`. Parameters follow the TypeScript SDK names, including `vm_id`, `vcpu`, `memory`, `env`, and `timeout_secs`. Canonical tools include `capsem_pause` and `capsem_status`; the retired `capsem_suspend`, `capsem_version`, and duplicate `capsem_service_logs` names are not exposed. The host tools cover: - VM lifecycle, one-shot execution, persistence, files, statistics, logs, timelines, panic extraction, and triage. - Private network lifecycle, membership, and cursor-based audit logs. - Typed MCP server discovery, refresh, permission inspection, and invocation through the running VM's existing relay and security engine. ### Debug workflow ```text 1. capsem_create 2. capsem_exec { vm_id: "...", command: "capsem-doctor -k net" } 3. capsem_read_file { vm_id: "...", path: "/tmp/capsem-init.log" } 4. capsem_timeline { vm_id: "...", layers: "net,tool,fs", limit: 50 } 5. Fix and rebuild the owning native component when required. 6. capsem_delete { vm_id: "..." } ``` The `--timeout-ms` option bounds the SDK HTTP request. `timeout_secs` on execution tools bounds the guest command. Cancellation closes the local request; it does not delete the VM or undo an accepted mutation. The SDK does not retry mutations. ### When to use MCP tools vs just recipes | Scenario | Use | | --- | --- | | Targeted guest behavior | `capsem_exec` | | Read a guest file | `capsem_read_file` | | Inspect typed audit evidence | `capsem_timeline`, statistics, and network logs | | Run capsem-doctor | `capsem_exec` | | Full regression suite | `just test` | | Focused native validation | `just focus-test functional` | ## npm host MCP server ```text Host AI agent -> @capsem/mcp (stdio) -> @capsem/sdk -> authenticated capsem-gateway HTTP -> capsem-service ``` The npm process owns tool presentation and typed input/output only. The gateway authenticates control requests. The service, VM owner, security engine, and logger retain lifecycle, trusted identity, policy, credentials, and audit storage. The npm process receives no virtualization entitlement, registry credential, CA private key, direct database access, or service socket access. ### Key source files | File | Purpose | | --- | --- | | `mcp/typescript/src/cli.ts` | Protocol-only stdio executable | | `mcp/typescript/src/server.ts` | SDK client and tool registration | | `mcp/typescript/src/host-tools.ts` | VM, file, and diagnostic tools | | `mcp/typescript/src/network-tools.ts` | Private network tools | | `mcp/typescript/src/mcp-tools.ts` | MCP server discovery and calls | | `mcp/typescript/src/results.ts` | Structured results and sanitized errors | | `sdk/typescript/src/` | Typed gateway client and validators | ## MCP subprocess architecture The guest MCP path is not a single process. The per-session `capsem-proxy` owns the guest-facing framed endpoint, MCP policy, and ledger admission. `capsem-process` owns a scoped adapter to the aggregator and VM exposure tools; it cannot reopen the session ledger and the proxy cannot reach the aggregator except through that granted connected channel. The service supervises two confined native workers behind the adapter: | Crate | Role | Privileges | |-------|------|-----------| | `capsem-mcp-aggregator` | Parses MCP transports and routes tools over a connected MessagePack RPC channel. | Inherited HTTP/2 capability for configured endpoints and a narrow builtin-peer spawn channel; no ambient files, direct network, credentials, execution, signals, VM or ledger authority. | | `capsem-mcp-builtin` | Implements HTTP tools (`fetch_http`, `grep_http`, `http_headers`) and `echo` over a connected MCP channel. | Inherited HTTP capability only. The service performs current URL/address policy and credential handling; HTTP ledger facts return in `_meta` `dev.capsem/ledger`. No policy-file or database access. | Rationale: isolating external-server connections in a low-privilege subprocess means a compromised third-party MCP server cannot reach the host filesystem or the session DB. The built-in tool server runs in its own process for the same reason. The shared `capsem-mcp-transport` crate owns connected HTTP capabilities and role bootstrap; it stays independent of VM core, policy, credentials and RMCP. The service supplies role/incarnation bootstrap records and connected descriptors. Each native entrypoint clears inherited environment/descriptors, prepares a current-thread runtime, installs Seatbelt on macOS or Landlock plus seccomp on Linux, and attests denial before acknowledging readiness. Parent watching starts after confinement. Linux aggregator/builtin profiles allow threads but deny forked processes; approved external executable profiles must retain their own sandbox. The owner receives one full-duplex RPC descriptor, never native child handles. MessagePack framing is `[4-byte big-endian length][payload]`; builtin MCP uses JSON-RPC lines over its own connected descriptor. Use async stream halves in both directions: stdin adoption sets nonblocking flags, so a blocking stdout adapter over a duplicate descriptor fails under backpressure. The service owns destinations, current policy, credentials and process reaping. Refresh preserves inherited capabilities. HTTP streams cancel on policy changes or retirement; generation completion awaits broker cleanup and positively reaped native children. Separate PIDs, parent watchers or `kill_on_drop` alone are not that completion proof. Settings configure external HTTP servers; there is no public arbitrary host stdio command/file-grant setting. The native spawn broker accepts only predefined builtin peer indices. New external stdio support needs explicit executable/file grants under `McpExternal`, never the trusted companion `with no-sandbox` rule. `/run/capsem-mcp-server` relays guest JSON-RPC as bounded frames over vsock 5002. The proxy evaluates policy before invoking the scoped owner adapter. Tool facts reach `tool_calls` through the proxy's typed ledger capability; the native MCP workers never open the ledger. Binaries land in `~/.capsem/bin/` at install time: `capsem-mcp-aggregator`, `capsem-mcp-builtin`. ## Guest MCP Endpoint The guest MCP relay bridges AI agents in the guest VM to the confined proxy endpoint. It runs over vsock port 5002 using bounded length-prefixed MCP frames that carry JSON-RPC payloads and per-frame process attribution. Framed guest MCP over `vsock:5002` must be tested as the default transport, not as an opt-in benchmark mode. The minimum hardening matrix for that path is: - parser/interpreter: bounded frames, invalid JSON, malformed flags, stream-id reuse, notification/request-id mismatch - dispatch: `initialize`, `tools/list`, builtin `tools/call`, configured external HTTP `tools/call`, `resources/list`, `prompts/list`, and method error mapping - policy: live policy mutation, per-tool block, resource URI rule, argument-name rule, argument-value rule, return-value rule, deny-over-allow precedence, and proof that blocked requests/responses do not leak original data - telemetry: `session.db` rows for success, denial, timeout, process attribution, request/response previews, policy fields, and terminal errors - boundary: aggregator and VM owner remain DB-free; `capsem-proxy` writes MCP audit facts through its role-bound ledger capability - VM E2E: boot a real VM, run `/run/capsem-mcp-server` with no transport override, then inspect the typed timeline/ledger routes ### Architecture ``` Guest (Claude/Gemini) -> capsem-mcp-server (stdin/stdout relay) -> framed vsock:5002 -> capsem-process descriptor handoff -> capsem-proxy -> SecurityEvent rule check -> scoped VM-owner MCP adapter -> builtin tools | external MCP servers (via aggregator) -> typed ledger capability -> tool_calls plus security_rule_events evidence ``` ### Workloads: streamable HTTP at `mcp.capsem.internal` An OCI workload cannot open vsock (seccomp), so it reaches the same endpoint over HTTP: `POST http://mcp.capsem.internal/mcp`, one JSON-RPC message per request (200 with the answer, 202 for notifications, 405 for GET/DELETE). The host DNS answers that one name in every VM with `192.0.2.1` before any private-network member is asked, and never upstream. The VM's port-80 rule carries the connection to the MITM proxy, where `mitm_proxy/mcp_http.rs` claims it before HTTP policy, body sniffing or any dial and hands it to `mcp_frame::dispatch_http_message` -- the framed path's parser, in-flight bound, MCP security rail and `tool_calls` ledger, with `transport = 'http'` and the connection's process name. Attribution is the proxy instance: one per VM, with that VM's endpoint and session ledger. Every other name in `capsem.internal`, and the MCP name on a non-default port, is refused with a `capsem.internal.reserved` net event. The official images configure their agent's `capsem` server at that URL (`images/*/Dockerfile`). ### Wire format Length-prefixed MCP frames over vsock. Each frame contains a bounded JSON-RPC payload plus a stream id, flags, and sanitized process name. #### Handshake Guest sends NUL-prefixed metadata line first: ``` \0CAPSEM_META:claude\n ``` Then JSON-RPC messages: ```json {"jsonrpc":"2.0","id":1,"method":"initialize","params":{}} {"jsonrpc":"2.0","id":2,"method":"tools/list"} {"jsonrpc":"2.0","id":3,"method":"tools/call","params":{"name":"github__search","arguments":{"q":"rust"}}} ``` ### Tool namespacing Tools are namespaced with `__` (double underscore) to prevent collisions: - `github` + `search_repos` -> `github__search_repos` - `builtin` + `http_get` -> `builtin__http_get` The endpoint parses the namespace to route to the correct server. ### Supported methods | Method | Behavior | |--------|----------| | `initialize` | Return protocol version + capabilities | | `notifications/initialized` | Notification (no response) | | `tools/list` | Return builtin + all external server tools | | `tools/call` | Policy check -> route to server -> call via rmcp | | `resources/list` | Return resource catalog from all servers | | `resources/read` | Lookup URI -> read via rmcp | | `prompts/list` | Return prompt catalog | | `prompts/get` | Lookup name -> get via rmcp | ### Security evaluation ``` 1. Parse MCP frame into typed `SecurityEvent` MCP fields. 2. Apply the shared security engine plugin/rule rail. 3. Dispatch only if the effective action allows it. 4. Log canonical tool row, optional MCP protocol row, and matched security rule rows. ``` Config hierarchy: built-in defaults, then the user's `settings.toml`, then the corp config, which wins (and whose `corp_locked` rules hold). `[mcp]` in settings.toml declares servers; tool permissions are settings-managed `profiles.rules.*` rules and `default.mcp`, edited through `/mcp/...` routes. The service merges all three into each session's `vm/active_policy.toml`. There is no MCP-specific decision provider or `user.toml` override rail. Decisions use the shared security action enum: `allow`, `ask`, `block`, `rewrite`, `preprocess`, and `postprocess`. `ask` waits for an approval or denial before dispatch; `block` returns a policy JSON-RPC error without calling the tool. ### Built-in tools `capsem-mcp-builtin` exposes the HTTP tools `fetch_http`, `grep_http`, and `http_headers`, plus `echo`, a zero-I/O probe for benchmarking MCP transport overhead. Guests see them under the `local__` namespace (e.g., `local__fetch_http`). ### Endpoint key source files | File | Purpose | |------|---------| | `crates/capsem-core/src/net/mitm_proxy/mcp_frame.rs` | Framed transport parser, stream lifecycle, disconnect metrics | | `crates/capsem-core/src/net/mitm_proxy/mcp_endpoint.rs` | JSON-RPC handler, policy, dispatch, telemetry logging | | `crates/capsem-core/src/mcp/types.rs` | JsonRpcRequest/Response, McpToolDef, annotations | | `crates/capsem-mcp-aggregator/src/server_manager.rs` | rmcp client pool, tool routing, catalog | | `crates/capsem-core/src/mcp/policy.rs` | Tool/server allow/warn/block decisions | | `crates/capsem-core/src/mcp/mod.rs` | Tool cache, server detection, collision detection | | `crates/capsem-agent/src/mcp_server.rs` | capsem-mcp-server binary (stdin/stdout relay) | ### Telemetry (tool_calls and security events) `tool_calls` is the canonical user/security ledger for all tool origins: model-native, built-in/local, and MCP-origin. UI counts, CEL rules, and forensic tool activity must use this table. MCP protocol facts such as initialize/list/resource frames are security events. A visible MCP `tools/call` without a matching `tool_calls` row is a bug. Every request/response logged with: timestamp, server_name, method, tool_name, compact request/response display fields, decision, duration_ms, error_message, process_name, bytes sent/received. Full MCP request/response payloads share the same `event_body_blobs` ledger table as HTTP and model traffic; do not add a second MCP-only body rail. Read `references/mcp-wire.md` for the full wire format details. ## Testing The npm package owns its unit, type, coverage, and package-build checks: ```sh pnpm --dir mcp/typescript test pnpm --dir mcp/typescript pack ``` `tests/capsem-sdk/test_mcp_cli_parity.py` guards canonical CLI/MCP behavior. `tests/ironbank/test_mcp_settings_ledger.py` packs the npm artifact, launches it over stdio with fixture-issued gateway credentials, drives a real VM and guest MCP path, and verifies correlated tool, network, and security ledger evidence. `tests/capsem-installed/test_winterfell_gateway.py` verifies the installed native HTTP cohort without requiring Node.js. The guest path retains its Rust tests and real-VM diagnostics: ```sh cargo test -p capsem-core mcp just exec "capsem-doctor -k mcp" ``` ## Lessons learned 1. **Never prepend headers to JSON output.** MCP tool responses with `format=json` must return raw, parseable JSON. Do not wrap JSON in pagination headers, content-length prefixes, or any other text. A JSON-format tool once broke because `paginated_response()` prepended `"Content length: ...\nShowing: ...\n"` to the JSON array, making `json.loads()` fail. Rule: if a tool offers both text and JSON formats, branch early and return JSON directly without passing through text-oriented helpers like `paginated_response()`.