Set up and configure Arize tracing for GitHub Copilot sessions. Use when users want to set up tracing, configure Arize AX or Phoenix for Copilot, enable/disable tracing, or troubleshoot tracing issues. Triggers on "set up copilot tracing", "configure Arize for Copilot", "configure Phoenix for Copilot", "enable copilot tracing", "setup-copilot-tracing", or any request about connecting GitHub Copilot to Arize or Phoenix for observability.
Set up and configure Arize tracing for GitHub Copilot sessions. Use when users want to set up tracing, configure Arize AX or Phoenix for Copilot, enable/disable tracing, or troubleshoot tracing issues. Triggers on "set up copilot tracing", "configure Arize for Copilot", "configure Phoenix for Copilot", "enable copilot tracing", "setup-copilot-tracing", or any request about connecting GitHub Copilot to Arize or Phoenix for observability.
Setup Copilot Tracing
Configure OpenInference tracing for GitHub Copilot in VS Code Copilot. Spans are sent directly to the backend from hooks -- no background process or backend-specific dependencies are needed in the user's environment.
How to Use This Skill
This skill follows a decision tree workflow. Start by asking the user where they are in the setup process:
Arize AX is available as a SaaS platform or as an on-prem deployment. Users need an account, a space, and an API key.
First, ask the user: "Are you using the Arize SaaS platform or an on-prem instance?"
SaaS -> Uses the default endpoint (otlp.arize.com:443). Continue below.
On-prem -> The user will need to provide their custom OTLP endpoint (e.g., otlp.mycompany.arize.com:443). Ask for it and note it for the Configure Settings step.
On-prem: Contact their administrator for access to the on-prem instance
2. Get Space ID and API key
Walk the user through finding their credentials:
Log in to their Arize instance (https://app.arize.com for SaaS, or their on-prem URL)
Click Settings (gear icon) in the left sidebar
The Space ID is shown on the Space Settings page
Go to the API Keys tab
Click Create API Key or copy an existing one
Both api_key and space_id are required for the shared config.
No Python dependencies are needed. Both Phoenix and Arize AX use HTTP/JSON -- no additional Python dependencies are needed.
Then proceed to Configure Settings. If the user is on an on-prem instance, remind them to provide their custom endpoint.
Configure Settings
Important: Users must run this setup before tracing will work. The send_span() function requires ~/.arize/harness/config.json to exist for backend credential resolution.
Ask the user for:
Backend choice: Phoenix or Arize AX
Credentials (only if no existing config):
Phoenix: endpoint URL (default: http://localhost:6006), optional API key
Arize AX: API key and Space ID
OTLP Endpoint (Arize AX only, optional): For hosted Arize instances using a custom endpoint. Defaults to otlp.arize.com:443.
Project name (optional): defaults to "copilot", stored under harnesses.copilot.project_name
User ID (optional): Set ARIZE_USER_ID env var to identify spans by user (useful for teams)
Write the config
The config file at ~/.arize/harness/config.json is the single source of truth for backend credentials and per-harness settings. Create the directory structure if needed: mkdir -p ~/.arize/harness/{bin,run,logs,state/copilot}
Important: read-merge-write. If ~/.arize/harness/config.json already exists, read it first, then merge in the new or updated fields (e.g., add/update the harnesses.copilot entry) while preserving existing backend credentials. Only prompt for backend credentials if no existing config is found.
Copilot hooks activated via .github/hooks/hooks.json
Spans are sent directly to the backend from hooks -- no background process needed
After saving, open a new Copilot session and traces will appear in their Phoenix UI or Arize AX dashboard under the project name
Mention ARIZE_DRY_RUN=true to test without sending data (set as env var before launching Copilot)
Mention ARIZE_VERBOSE=true for debug output
Errors are always written to ~/.arize/harness/logs/copilot.log; set ARIZE_VERBOSE=true in the shell before launching VS Code / Copilot CLI to also capture routine hook activity
Hook Events
Copilot fires 6 hook events. Each event maps to a span:
Event
Span Name
Kind
Description
SessionStart
Session Start
CHAIN
Session initialization
UserPromptSubmit
User Prompt
CHAIN
User prompt text
PreToolUse
Tool: {name}
TOOL
Tool start; must print permission response to stdout
PostToolUse
Tool: {name}
TOOL
Tool result
Stop
Agent Stop
LLM
Per-turn completion; transcript at ~/.copilot/session-state/<session_id>/events.jsonl is parsed for model name, prompt, and tool-call count
SubagentStop
Subagent: {id}
CHAIN
Subagent completion
PreToolUse permission response
The pre-tool handler must print a permission response to stdout:
