troubleshoot

# Troubleshoot ## Purpose This skill investigates and explains unexpected chat agent behavior using direct log files. Use this skill for questions like: - Why did this request take so long? - Why was a tool or subagent called? - Why did instruction/skill/agent files not load? - Why was a tool call blocked or failed? - Why did the model not follow expectations? Base conclusions on evidence from logs. Do not guess. ## Data Source - Target session log directory/directories for analysis: `{{VSCODE_TARGET_SESSION_LOG}}` Use direct debug log files written by Copilot Chat: ``` debug-logs/<sessionId>/ main.jsonl — always start here; primary conversation log models.json — (optional) snapshot of available models at session start system_prompt_0.json — (optional) full system prompt sent to the model (untruncated) system_prompt_1.json — (optional) written when the model changes mid-session tools_0.json — (optional) tool definitions sent to the model tools_1.json — (optional) written when the model changes mid-session runSubagent-<agentName>-<uuid>.jsonl — (optional) subagent's tool calls & LLM requests searchSubagent-<uuid>.jsonl — (optional) search subagent work title-<uuid>.jsonl — (optional, UI-only) title generation categorization-<uuid>.jsonl — (optional, UI-only) prompt categorization summarize-<uuid>.jsonl — (optional, UI-only) conversation summarization ``` Always read `main.jsonl` first — it has the full conversation flow. Child files only appear when those operations occurred. `main.jsonl` contains `child_session_ref` entries that link to each child file by name. Title, categorization, and summarize files are UI housekeeping and rarely relevant to troubleshooting. When investigating model availability or selection issues, read `models.json` — it contains the full list of models (with capabilities, billing, and limits) that were available when the session started. When investigating what the model was told (system prompt, instructions), read the `system_prompt_*.json` file referenced by a `system_prompt_ref` entry in `main.jsonl`. The file contains the full untruncated system prompt as `{ "content": "..." }`. When investigating which tools were available, read the `tools_*.json` file similarly. If the model changed mid-session, multiple numbered files exist — each `llm_request` entry has a `systemPromptFile` attr indicating which file was active for that request. Each line is a JSON object. Common fields: `ts` (epoch ms), `dur` (duration ms), `sid` (session ID), `type`, `name`, `spanId`, `parentSpanId`, `status` (`ok`|`error`), `attrs` (type-specific details). ### Event Type Reference with Examples #### discovery — customization file loading (instructions, skills, agents, hooks) ```jsonl {"ts":1773200251309,"dur":0,"sid":"62f52dec","type":"discovery","name":"Load Instructions","spanId":"2cb1f2f4","status":"ok","attrs":{"details":"Resolved 0 instructions in 0.0ms | folders: [/c:/Users/user/.copilot/instructions, /workspace/.github/instructions]","category":"discovery","source":"core"}} {"ts":1773200251415,"dur":0,"sid":"62f52dec","type":"discovery","name":"Load Agents","spanId":"38a897d8","status":"ok","attrs":{"details":"Resolved 3 agents in 0.0ms | loaded: [Plan, Ask, Explore] | folders: [/workspace/.github/agents]","category":"discovery","source":"core"}} {"ts":1773200251431,"dur":0,"sid":"62f52dec","type":"discovery","name":"Load Skills","spanId":"472eb225","status":"ok","attrs":{"details":"Resolved 6 skills in 0.0ms | loaded: [agent-customization, troubleshoot, ...]","category":"discovery","source":"core"}} ``` Key attrs: `details` (human-readable summary with folder paths, loaded items, skip reasons), `category` (always `"discovery"`), `source` (`"core"`). #### tool_call — tool invocation (success or failure) ```jsonl {"ts":1773200222647,"dur":4,"sid":"62f52dec","type":"tool_call","name":"manage_todo_list","spanId":"000000000000000b","parentSpanId":"0000000000000003","status":"ok","attrs":{"args":"{\"operation\":\"read\"}","result":"No todo list found."}} {"ts":1773200234047,"dur":8937,"sid":"62f52dec","type":"tool_call","name":"run_in_terminal","spanId":"000000000000000d","parentSpanId":"0000000000000003","status":"error","attrs":{"args":"{\"command\":\"echo rama\"}","result":"ERROR: conpty.node missing","error":"A native exception occurred during launch"}} ``` Key attrs: `args` (JSON string of tool input), `result` (tool output or error text), `error` (present when `status:"error"`). #### llm_request — model round-trip ```jsonl {"ts":1773200231010,"dur":3001,"sid":"62f52dec","type":"llm_request","name":"chat:gpt-4o","spanId":"000000000000000c","parentSpanId":"0000000000000003","status":"ok","attrs":{"model":"gpt-4o","inputTokens":15025,"outputTokens":126,"ttft":1987,"maxTokens":32000,"systemPromptFile":"system_prompt_0.json","userRequest":"echo hello","inputMessages":"[{...}]"}} ``` Key attrs: `model`, `inputTokens`, `outputTokens`, `ttft` (time to first token in ms), `maxTokens`, `temperature`, `topP`, `systemPromptFile` (references a system prompt file in the session directory), `toolsFile` (references a tools file in the session directory), `userRequest` (the full user message content, untruncated), `inputMessages` (full messages array as JSON, truncated to the configured `maxAttributeSizeChars` — unlimited by default), `error` (when failed). #### agent_response — model output (text + tool calls) ```jsonl {"ts":1773200234011,"dur":0,"sid":"62f52dec","type":"agent_response","name":"agent_response","spanId":"agent-msg-000000000000000c","parentSpanId":"0000000000000003","status":"ok","attrs":{"response":"[{\"role\":\"assistant\",...}]","reasoning":"The user wants me to run a command."}} ``` Key attrs: `response` (JSON-encoded array of message parts; may be truncated), `reasoning` (optional — the model's chain-of-thought/thinking text when thinking mode is active; may be truncated). #### user_message — user input ```jsonl {"ts":1773200251345,"dur":0,"sid":"62f52dec","type":"user_message","name":"user_message","spanId":"000000000000000f","status":"ok","attrs":{"content":"using subagent count .md"}} ``` Key attrs: `content` (the user's message text). #### subagent — subagent invocation ```jsonl {"ts":1773200254954,"dur":7921,"sid":"62f52dec","type":"subagent","name":"Explore","spanId":"0000000000000014","parentSpanId":"0000000000000013","status":"ok","attrs":{"agentName":"Explore"}} ``` Key attrs: `agentName`, `description` (optional), `error` (when failed). #### generic — miscellaneous events ```jsonl {"ts":1773200260000,"dur":0,"sid":"62f52dec","type":"generic","name":"some-event","spanId":"abc123","status":"ok","attrs":{"details":"Additional context","category":"some-category"}} ``` Special generic entries: - `system_prompt_ref` — references a `system_prompt_*.json` file in the session directory. `attrs.file` is the filename, `attrs.model` is the model it was written for. Read this file to see the full system prompt. - `tools_ref` — references a `tools_*.json` file. `attrs.file` is the filename, `attrs.model` is the model. #### session_start — session metadata (appears once at session start) ```jsonl {"ts":1773200251300,"dur":0,"sid":"62f52dec","type":"session_start","name":"session_start","spanId":"session-start-62f52dec","status":"ok","attrs":{"copilotVersion":"0.43.2026033104","vscodeVersion":"1.99.0"}} ``` Key attrs: `copilotVersion`, `vscodeVersion`. Useful for identifying which build produced the logs. #### turn_start / turn_end — tool-calling loop iteration boundaries ```jsonl {"ts":1773200251400,"dur":0,"sid":"62f52dec","type":"turn_start","name":"turn_start:0","spanId":"turn-start-X-0","status":"ok","attrs":{"turnId":"0"}} {"ts":1773200255000,"dur":0,"sid":"62f52dec","type":"turn_end","name":"turn_end:0","spanId":"turn-end-X-0","status":"ok","attrs":{"turnId":"0"}} ``` Key attrs: `turnId` (iteration number within a single user request's tool-calling loop). Use these to identify which iteration events belong to and to count total loop iterations. ### Reading the event hierarchy Events form a tree via `spanId`/`parentSpanId`. A typical chain: 1. `user_message` (spanId: `X`) — the user's turn 2. `llm_request` (parentSpanId: `X`) — model call for that turn 3. `agent_response` (parentSpanId: `X`) — what the model returned 4. `tool_call` (parentSpanId: `X`) — tool executed from the response 5. Another `llm_request` (parentSpanId: `X`) — next model call after tool result Subagent calls create nested hierarchies: the `tool_call` for `runSubagent` (spanId: `Y`) becomes the parent for a child `subagent` span, which in turn parents its own `llm_request`/`tool_call` events. ## Tooling Strategy (important) Debug log files live outside the workspace (in user storage), so workspace-scoped search tools like `grep_search` cannot access them. Use the terminal instead. **Do not use `grep_search` for log files — it only works on workspace files.** ### macOS / Linux / WSL / Git Bash Use `run_in_terminal` with `grep` or `jq`: - Find errors: `grep '"status":"error"' <logPath>` - Find discovery events: `grep '"type":"discovery"' <logPath>` - Find slow events (duration > 5s): `jq -c 'select(.dur > 5000)' <logPath>` - Find tool calls: `grep '"type":"tool_call"' <logPath>` - Search for specific text: `grep 'search_term' <logPath>` - Get last N lines: `tail -n 50 <logPath>` - Count events by type: `jq -r '.type' <logPath> | sort | uniq -c | sort -rn` - Extract specific fields: `jq -c '{type, name, status, dur}' <logPath>` - Filter by type and show details: `jq -c 'select(.type == "discovery")' <logPath>` - Find user messages: `jq -c 'select(.type == "user_message") | .attrs.content' <logPath>` ### Windows (PowerShell) Use `run_in_terminal` with PowerShell commands: - Find errors: `Select-String '"status":"error"' <logPath>` - Find discovery events: `Select-String '"type":"discovery"' <logPath>` - Find tool calls: `Select-String '"type":"tool_call"' <logPath>` - Search for specific text: `Select-String 'search_term' <logPath>` - Get last N lines: `Get-Content <logPath> -Tail 50` - Parse and filter with Node.js (always available): `node -e "require('fs').readFileSync('<logPath>','utf8').split('\n').filter(Boolean).map(JSON.parse).filter(e => e.dur > 5000).forEach(e => console.log(JSON.stringify(e)))"`