manage-cursor-tracing

# Setup Cursor Tracing Configure OpenInference tracing for Cursor IDE sessions to Arize AX (cloud) or Phoenix (self-hosted). Spans are sent directly to the backend from hooks -- no background process or backend-specific dependencies are needed in the user's environment. ## How to Use This Skill **This skill follows a decision tree workflow.** Start by asking the user where they are in the setup process: 1. **First, determine the install type.** Was Arize tracing installed as a **Cursor plugin** (via `/add-plugin` in Cursor 2.5+, which registers hooks automatically), or **manually** by running `install.sh`? - If unsure, ask the user directly: "Did you install Arize tracing as a Cursor plugin (via /add-plugin), or by running install.sh?" - Heuristic 1: Check Cursor's installed-plugins location for a `cursor-tracing` plugin directory. - Heuristic 2: Check whether `.cursor/hooks.json` already contains `arize-hook-cursor` entries -- if so, this is a manual `install.sh` install; if absent, it's a plugin install (or a fresh setup). - This branch only matters for the [Activate Cursor hooks](#activate-cursor-hooks) step below. The credentials/config step is identical for both install types. 2. **Do they already have credentials?** - Yes -> Jump to [Configure Settings](#configure-settings) - No -> Continue to step 3 3. **Which backend do they want to use?** - Phoenix (self-hosted) -> Go to [Set Up Phoenix](#set-up-phoenix) - Arize AX (cloud) -> Go to [Set Up Arize AX](#set-up-arize-ax) 4. **Are they troubleshooting?** - Yes -> Jump to [Troubleshoot](#troubleshoot) **Important:** Only follow the relevant path for the user's needs. Don't go through all sections. **Plugin users, please read:** The marketplace install via `/add-plugin` registers Cursor hooks automatically but cannot run an interactive credentials wizard. Run this skill once after `/add-plugin` so it can write `~/.arize/harness/config.json` with your backend credentials -- without that file, the hooks will fail open (no-op) and no spans will be sent. ## Set Up Phoenix Phoenix is self-hosted. No Python dependencies are needed for tracing -- spans are sent directly via `send_span()` using stdlib `urllib`. ### Install Phoenix Ask if they already have Phoenix running. If not, walk through: ```bash # Option A: pip pip install arize-phoenix && phoenix serve # Option B: Docker docker run -p 6006:6006 arizephoenix/phoenix:latest ``` Phoenix UI will be available at `http://localhost:6006`. Confirm it's running: ```bash curl -sf http://localhost:6006/v1/traces >/dev/null && echo "Phoenix is running" || echo "Phoenix not reachable" ``` Then proceed to [Configure Settings](#configure-settings) with the Phoenix endpoint. ## Set Up Arize AX Arize AX is available as a SaaS platform or as an on-prem deployment. Users need an account, a space, and an API key. **First, ask the user: "Are you using the Arize SaaS platform or an on-prem instance?"** - **SaaS** -> Uses the default endpoint (`otlp.arize.com:443`). Continue below. - **On-prem** -> The user will need to provide their custom OTLP endpoint (e.g., `otlp.mycompany.arize.com:443`). Ask for it and note it for the [Configure Settings](#configure-settings) step. ### 1. Create an account If the user doesn't have an Arize account: - **SaaS**: Sign up at https://app.arize.com/auth/join - **On-prem**: Contact their administrator for access to the on-prem instance ### 2. Get Space ID and API key Walk the user through finding their credentials: 1. Log in to their Arize instance (https://app.arize.com for SaaS, or their on-prem URL) 2. Click **Settings** (gear icon) in the left sidebar 3. The **Space ID** is shown on the Space Settings page 4. Go to the **API Keys** tab 5. Click **Create API Key** or copy an existing one Both `api_key` and `space_id` are required for the shared config. **No Python dependencies are needed.** Both Phoenix and Arize AX use HTTP/JSON — no additional Python dependencies are needed. Then proceed to [Configure Settings](#configure-settings). If the user is on an on-prem instance, remind them to provide their custom endpoint. ## Configure Settings **Important:** Users must run this setup before tracing will work. The `send_span()` function requires `~/.arize/harness/config.json` to exist for backend credential resolution. ### Ask the user for: 1. **Backend choice**: Phoenix or Arize AX 2. **Credentials** (only if no existing config): - Phoenix: endpoint URL (default: `http://localhost:6006`), optional API key - Arize AX: API key and Space ID 3. **OTLP Endpoint** (Arize AX only, optional): For hosted Arize instances using a custom endpoint. Defaults to `otlp.arize.com:443`. 4. **Project name** (optional): defaults to `"cursor"`, stored under `harnesses.cursor.project_name` 5. **User ID** (optional): Set `ARIZE_USER_ID` env var to identify spans by user (useful for teams) ### Write the config The config file at `~/.arize/harness/config.json` is the single source of truth for backend credentials and per-harness settings. Create the directory structure if needed: `mkdir -p ~/.arize/harness/{bin,run,logs,state/cursor}` **Important: read-merge-write.** If `~/.arize/harness/config.json` already exists, read it first, then merge in the new or updated fields (e.g., add/update the `harnesses.cursor` entry) while preserving existing backend credentials. Only prompt for backend credentials if no existing config is found. **Phoenix:** ```json { "harnesses": { "cursor": { "project_name": "cursor", "target": "phoenix", "endpoint": "<endpoint>", "api_key": "" } } } ``` **Arize AX:** ```json { "harnesses": { "cursor": { "project_name": "cursor", "target": "arize", "endpoint": "otlp.arize.com:443", "api_key": "<key>", "space_id": "<id>" } } } ``` If the user has a custom OTLP endpoint, set it in `harnesses.cursor.endpoint`. ### Activate Cursor hooks **This step depends on install type** (see [How to Use This Skill](#how-to-use-this-skill) step 1). Pick the branch that matches the user's install: #### Plugin install (Cursor `/add-plugin`) **Skip this step entirely.** The plugin's bundled `hooks/hooks.json` already registers every Cursor hook event automatically when the plugin is installed. There is nothing to write to `.cursor/hooks.json`. After saving credentials in the previous step, tell the user to start a new Cursor session -- traces will begin flowing on the next interaction. > **Warning:** Do NOT manually add `.cursor/hooks.json` entries on top of a plugin install. Cursor will then route each event to the handler twice (once via the plugin and once via your project file), producing duplicate spans for every hook. If the user already has manual `.cursor/hooks.json` entries pointing at `arize-hook-cursor` from a previous `install.sh` setup, remove those entries before relying on the plugin. #### Manual install (`install.sh`) Cursor uses a `.cursor/hooks.json` file in the project root to route hook events to the handler. All events route to a single `arize-hook-cursor` CLI entry point, which dispatches based on `hook_event_name` in the JSON payload. Create `.cursor/hooks.json` in the user's project (or merge into it if it already exists): ```json { "version": 1, "hooks": { "sessionStart": [{ "command": "~/.arize/harness/venv/bin/arize-hook-cursor" }], "sessionEnd": [{ "command": "~/.arize/harness/venv/bin/arize-hook-cursor" }], "beforeSubmitPrompt": [{ "command": "~/.arize/harness/venv/bin/arize-hook-cursor" }], "afterAgentResponse": [{ "command": "~/.arize/harness/venv/bin/arize-hook-cursor" }], "afterAgentThought": [{ "command": "~/.arize/harness/venv/bin/arize-hook-cursor" }], "beforeShellExecution": [{ "command": "~/.arize/harness/venv/bin/arize-hook-cursor" }], "afterShellExecution": [{ "command": "~/.arize/harness/venv/bin/arize-hook-cursor" }], "beforeMCPExecution": [{ "command": "~/.arize/harness/venv/bin/arize-hook-cursor" }], "afterMCPExecution": [{ "command": "~/.arize/harness/venv/bin/arize-hook-cursor" }], "beforeReadFile": [{ "command": "~/.arize/harness/venv/bin/arize-hook-cursor" }], "afterFileEdit": [{ "command": "~/.arize/harness/venv/bin/arize-hook-cursor" }], "stop": [{ "command": "~/.arize/harness/venv/bin/arize-hook-cursor" }], "beforeTabFileRead": [{ "command": "~/.arize/harness/venv/bin/arize-hook-cursor" }], "afterTabFileEdit": [{ "command": "~/.arize/harness/venv/bin/arize-hook-cursor" }], "postToolUse": [{ "command": "~/.arize/harness/venv/bin/arize-hook-cursor" }] } } ``` If the user already has a `.cursor/hooks.json` with other hooks, merge the Arize entries into the existing arrays for each event. ### Validate 1. **Config exists**: Run `cat ~/.arize/harness/config.json` to verify the config file exists and has correct backend credentials. 2. **Phoenix** (if applicable): Run `curl -sf <endpoint>/v1/traces >/dev/null` to check connectivity. 3. **Hooks active**: - **Manual install**: Verify `.cursor/hooks.json` exists in the project root and contains the Arize hook entries. - **Plugin install**: No project-level `.cursor/hooks.json` is required -- the plugin registers hooks itself. Confirm the `cursor-tracing` plugin is listed as installed in Cursor. ### Confirm Tell the user: - Config saved to `~/.arize/harness/config.json` - Cursor hooks activated via `.cursor/hooks.json` - Spans are sent directly to the backend from hooks — no background process needed - After saving, open a new Cursor session and traces will appear in their Phoenix UI or Arize AX dashboard under the project name - Mention `ARIZE_DRY_RUN=true` to test without sending data (set as env var before launching Cursor) - Mention `ARIZE_VERBOSE=true` for debug output - Errors are always written to `~/.arize/harness/logs/cursor.log`; set `ARIZE_VERBOSE=true` in the shell before launching Cursor to also capture routine hook activity ## Hook Events ### IDE Hooks Cursor IDE fires 15 hook events. Here's what each one traces: | Event | Span Name | Kind | Description | |-------|-----------|------|-------------| | `sessionStart` | Session Start | CHAIN | CLI fallback parent with session metadata; suppressed for IDE sessions | | `beforeSubmitPrompt` | User Prompt | CHAIN | Starts the deferred root for one turn; `stop` closes it | | `afterAgentResponse` | Agent Response | LLM | Adds response text to one turn-level LLM span, which `stop` emits with token usage | | `afterAgentThought` | Agent Thinking | CHAIN | Point span for unique, exact thinking text within the turn | | `beforeShellExecution` | (state push) | -- | Saves command and start time to disk state | | `afterShellExecution` | Shell | TOOL | Merged command span; numeric exit codes set `OK` or `ERROR`, while missing or invalid codes stay `UNSET` | | `beforeMCPExecution` | (state push) | -- | Saves tool name, input, and start time | | `afterMCPExecution` | MCP: {tool} | TOOL | Merged span with tool input and result | | `beforeReadFile` | Read File | TOOL | File path being read | | `afterFileEdit` | File Edit | TOOL | File path and edit details | | `beforeTabFileRead` | Tab Read File | TOOL | Tab file read (file path) | | `afterTabFileEdit` | Tab File Edit | TOOL | Tab file edit (path and edits) | | `postToolUse` | Tool: {name} | TOOL | Generic tool span; postToolUse is suppressed for tools with a dedicated handler (Shell, Read, File Edit, Tab ops, MCP) to avoid duplicate spans | | `stop` | Agent Stop | CHAIN | Zero-duration turn terminal; closes the root once and supplies token totals to the LLM span | | `sessionEnd` | Session End | CHAIN | End-of-session point span; closes a pending turn and keeps only `cursor.*` session totals | Shell and MCP events use a disk-backed state stack to merge before/after context into single spans with both input and output. ### CLI Hooks Cursor CLI currently emits a smaller hook surface than the IDE. The supported CLI hooks in this package are: - `sessionStart` - `sessionEnd` - `beforeShellExecution` - `afterShellExecution` - `afterFileEdit` - `postToolUse` - `stop` Cursor CLI hooks do not currently emit afterAgentResponse or afterAgentThought. Full Cursor CLI assistant and thinking coverage requires parsing --output-format stream-json, which is out of scope for this change. ### What We Capture - **`sessionStart`** is suppressed for IDE sessions because user turns provide the useful roots. For CLI sessions, it produces a fallback CHAIN parent with session metadata because CLI can omit `beforeSubmitPrompt`. - **`beforeSubmitPrompt`** starts one deferred `User Prompt` root for each turn. All turn spans share its trace ID and use it as their parent. The `stop` event closes the root. A `sessionEnd` event or the next prompt closes a pending root once. If persisting the deferred turn fails, the turn is closed immediately through the same closure path `stop`/`sessionEnd`/next-prompt use, rather than being silently dropped. - **`afterAgentResponse`** adds response fragments to one `Agent Response` LLM span. The span keeps text in observation order and bounded in count. Its inferred interval starts after the preceding observed turn activity and ends at the final response. The `cursor.llm.usage.scope` and `cursor.llm.timing.scope` attributes identify this turn scope. A user identity supplied here is copied into the turn. - **`stop`** produces a zero-duration `Agent Stop` CHAIN span. Its token totals occur once on the turn LLM span; when neither a turn nor token counts exist, `llm.model_name` still lands on the `Agent Stop` span rather than being dropped. The prompt total includes the uncached input plus cache-read and cache-write tokens. Zero token values remain present. Duplicate/stale `stop` delivery is rejected by a durable, bounded per-generation terminal marker (not a single "latest generation" marker), so a late `stop` for an old generation cannot resolve to and close the currently active turn. - **`sessionEnd`** produces a `Session End` CHAIN point span. Useful session totals remain under `cursor.session.token_count.*`. The span has no OpenInference token totals. It removes active state and retains bounded terminal history so late duplicate stops remain suppressed. - **`afterShellExecution`** keeps `shell.exit_code` as a string. Numeric zero maps to `OK`, and a numeric nonzero value maps to `ERROR`. Missing or invalid values map to `UNSET`. Output text never determines failure when Cursor omits the code. - **`postToolUse`** produces a generic `Tool: <name>` span ONLY for tools without a dedicated handler. Shell, file read/edit, tab file ops, and MCP execution are handled by their dedicated `before*`/`after*` events; the generic postToolUse is suppressed for these to avoid duplicate spans.