| name | copilot-log-analysis |
| description | Analyzing GitHub Copilot session log files to extract token usage, model information, and interaction data. Use when working with session files, understanding the extension's log analysis methods, or debugging token tracking issues. |
Copilot Log Analysis Skill
This skill documents the methods and approaches used by the GitHub Copilot Token Tracker extension to analyze Copilot session log files. These files contain chat sessions, token usage, and model information.
Overview
The extension analyzes two types of log files:
.json files: Standard VS Code Copilot Chat session files.jsonl files: Copilot CLI/Agent mode sessions, JetBrains IDE Copilot Chat sessions, Claude Code sessions, Gemini CLI sessions, Antigravity sessions, and other ecosystem adapters (one JSON event per line)
Session File Discovery
Key Method: getCopilotSessionFiles()
Location: src/extension.ts (lines 975-1073)
Helper Methods: getVSCodeUserPaths() (lines 934-972), scanDirectoryForSessionFiles() (lines 1078-1110)
This method discovers session files across all VS Code variants and locations:
Supported VS Code Variants:
- VS Code (Stable)
- VS Code Insiders
- VS Code Exploration
- VSCodium
- Cursor
- VS Code Server/Remote
File Locations Checked:
- Workspace Storage:
{VSCode User Path}/workspaceStorage/{workspace-id}/chatSessions/*.json
- Global Storage (Legacy):
{VSCode User Path}/globalStorage/emptyWindowChatSessions/*.json
- Copilot Chat Extension Storage:
{VSCode User Path}/globalStorage/github.copilot-chat/**/*.json
- Copilot CLI Sessions:
~/.copilot/session-state/*.jsonl
- JetBrains IDE Copilot Sessions:
~/.copilot/jb/{conversationId}/partition-{n}.jsonl
- Gemini CLI Sessions:
~/.gemini/tmp/{project}/chats/session-*.jsonl
- Antigravity Sessions:
~/.gemini/antigravity/brain/{session-uuid}/.system_generated/logs/transcript.jsonl
Platform-Specific Paths:
- Windows:
%APPDATA%/{Variant}/User
- macOS:
~/Library/Application Support/{Variant}/User
- Linux:
~/.config/{Variant}/User (respects XDG_CONFIG_HOME)
- Remote/Server:
~/.vscode-server/data/User, ~/.vscode-server-insiders/data/User
Helper Method: getVSCodeUserPaths()
Location: src/extension.ts (lines 934-972)
Returns all possible VS Code user data paths for different variants and platforms.
Helper Method: scanDirectoryForSessionFiles()
Location: src/extension.ts (lines 1078-1110)
Recursively scans directories for .json and .jsonl session files.
Field Extraction Methods
Parsing and Token Accounting: parseSessionFileContent()
Location: src/sessionParser.ts (lines 184-347)
Purpose: Parses session files and returns tokens, interactions, model usage, and editor type-safe model IDs.
How it works:
- Accepts raw file content along with callbacks for token estimation and model detection.
- Supports both
.json (Copilot Chat) and .jsonl (CLI/agent) formats, including delta-based JSONL streams.
- Counts interactions (user messages), input tokens, and output tokens while grouping by model.
- Uses
estimateTokensFromText() (lines 1139-1155 in src/extension.ts) for character-to-token estimation.
Model Detection Logic: getModelFromRequest()
Location: src/extension.ts (lines 1102-1134)
- Primary:
request.result.metadata.modelId
- Fallback: parses
request.result.details for known model patterns
- Detected patterns: GPT-3.5-Turbo, GPT-4 family (4, 4.1, 4o, 4o-mini, 5, o3-mini, o4-mini), Claude Sonnet (3.5, 3.7, 4), Gemini (2.5 Pro, 3 Pro, 3 Pro Preview); defaults to
gpt-4
- Display name mapping in
getModelDisplayName() (lines 1778-1811) adds variants such as GPT-5 family, Claude Haiku, Claude Opus, Gemini 3 Flash, Grok, and Raptor when present in metadata.modelId.
Editor Type Detection: getEditorTypeFromPath()
Location: src/extension.ts (lines 111-143)
Purpose: Determines which VS Code variant created the session file.
Detection patterns:
- Contains
/.copilot/jb/ → 'JetBrains' (must be checked BEFORE the Copilot CLI rule below since both live under ~/.copilot/)
- Contains
/.copilot/session-state/ → 'Copilot CLI'
- Contains
/code - insiders/ → 'VS Code Insiders'
- Contains
/code - exploration/ → 'VS Code Exploration'
- Contains
/vscodium/ → 'VSCodium'
- Contains
/cursor/ → 'Cursor'
- Contains
.vscode-server-insiders/ → 'VS Code Server (Insiders)'
- Contains
.vscode-server/ → 'VS Code Server'
- Contains
/code/ → 'VS Code'
- Default →
'Unknown'
Token Estimation Algorithm
Character-to-Token Conversion: estimateTokensFromText()
Location: src/extension.ts (lines 1139-1155)
Approach: Uses model-specific character-to-token ratios
- Default ratio: 0.25 (4 characters per token)
- Model-specific ratios loaded from
src/tokenEstimators.json
- Formula:
Math.ceil(text.length * tokensPerChar)
Model matching:
- Checks if model name includes the key from tokenEstimators
- Example:
gpt-4o matches key gpt-4o
Caching Strategy
Cache Structure: SessionFileCache
Location: src/extension.ts (lines 72-77)
Stores pre-calculated tokens, interactions, model usage, and file mtime to avoid re-processing unchanged files.
Cache Methods:
isCacheValid() (lines 227-230): Validates cached entry by mtimegetCachedSessionData() (lines 232-234): Retrieves cached datasetCachedSessionData() (lines 236-254): Stores data with FIFO eviction after 1000 filesclearExpiredCache() (lines 250-264): Drops cache entries for missing filesgetSessionFileDataCached() (lines 811-845): Reads session content, parses via parseSessionFileContent(), and caches results
Schema Documentation
Schema Files Location
Directory: docs/logFilesSchema/
Key files:
session-file-schema.json: Manual curated schema with descriptionssession-file-schema-analysis.json: Auto-generated field discovery (generated by PowerShell script)README.md: Complete guide for schema analysisSCHEMA-ANALYSIS.md: Quick reference guideVSCODE-VARIANTS.md: VS Code variant detection documentation
Note: The analysis JSON file is auto-generated and may not exist in fresh clones. It is created by running the schema analysis script documented below.
Schema Analysis
See the Executable Scripts section for available utilities:
get-session-files.js - Quick session file discoverydiagnose-session-files.js - Detailed diagnosticsanalyze-session-schema.ps1 - PowerShell schema analysis
JSON File Structure (VS Code Sessions)
Primary fields used by extension:
{
"requests": [
{
"message": {
"parts": [
{ "text": "user message content" }
]
},
"response": [
{ "value": "assistant response content" }
],
"result": {
"metadata": {
"modelId": "gpt-4o"
},
"details": "Used GPT-4o model"
}
}
]
}
Key paths:
- Input tokens:
requests[].message.parts[].text
- Output tokens:
requests[].response[].value
- Model ID:
requests[].result.metadata.modelId
- Model details:
requests[].result.details
- Interaction count:
requests.length
JSONL File Structure (Copilot CLI)
Event types:
{"type": "user.message", "data": {"content": "..."}, "model": "gpt-4o"}
{"type": "assistant.message", "data": {"content": "..."}}
{"type": "tool.execution_start", "data": {"toolCallId": "...", "toolName": "...", "arguments": {}}}
{"type": "tool.execution_complete", "data": {"toolCallId": "...", "success": true, "result": {"content": "...", "detailedContent": "..."}}}
{"type": "session.shutdown", "data": {"modelMetrics": {"claude-opus-4.6": {"usage": {"inputTokens": 0, "outputTokens": 0, "cacheReadTokens": 0, "cacheWriteTokens": 0}}}}}
Key fields:
- Event type:
type
- User input:
data.content (when type: 'user.message')
- Assistant output:
data.content (when type: 'assistant.message')
- Tool output:
data.result.content or data.result.detailedContent (when type: 'tool.execution_complete')
- Tool name for counting:
data.toolName (when type: 'tool.execution_start')
- API-accurate token counts:
data.modelMetrics[model].usage (when type: 'session.shutdown') — always prefer these over estimates when available
- Model:
model (optional, defaults to gpt-4o)
JSONL File Structure (JetBrains IDE)
Location: ~/.copilot/jb/{conversationId}/partition-{n}.jsonl — one UUID-named directory per conversation, one or more partition files per conversation. Empty partition files are skipped.
Full schema documentation: docs/logFilesSchema/jetbrains-session-schema.json
Common envelope — every line is { type, data, id, timestamp, parentId }.
Event types:
{"type":"partition.created","data":{"conversationId":"...","partitionId":1,"source":"panel","createdAt":1777552130660}}
{"type":"user.message","data":{"content":"...","turnId":"..."}}
{"type":"user.message_rendered","data":{"turnId":"...","renderedMessage":"<context>...</context><reminderInstructions>You are an agent...</reminderInstructions><userRequest>...</userRequest>"}}
{"type":"assistant.turn_start","data":{"turnId":"..."}}
{"type":"assistant.message","data":{"text":"...","thinking":{"text":"..."},"iterationNumber":1,"messageId":"..."}}
{"type":"tool.execution_start","data":{"toolCallId":"toolu_bdrk_...","toolName":"read_file","arguments":{...}}}
{"type":"tool.execution_complete","data":{"toolCallId":"...","success":true,"result":{"result":[{"type":"text","value":"..."}]}}}
{"type":"assistant.turn_end","data":{"turnId":"...","status":"success"}}
Key extraction rules (also implemented by parseJetBrainsPartition in src/jetbrains.ts):
- Interactions: count of
user.message events
- Input tokens: estimated from
user.message_rendered.data.renderedMessage (falls back to user.message.data.content)
- Output tokens: estimated from
assistant.message.data.text
- Thinking tokens: estimated from
assistant.message.data.thinking.text
- Actual tokens: always 0 — JetBrains does not record API-side usage counts in the session file
- Mode: presence of any
tool.execution_start event ⇒ agent, otherwise ask (no edit/plan/customAgent)
- Model: not in the file. Best-effort heuristic from
toolCallId prefix (toolu_* ⇒ Anthropic Claude, call_* ⇒ OpenAI), otherwise unknown
- First/last interaction: timestamps of the first
user.message and the last assistant.turn_end (or assistant.message) event
JSONL File Structure (Antigravity)
What: Google's closed-source successor to Gemini CLI, released May 2026. An Electron-based desktop IDE backed by cloudcode-pa.googleapis.com.
Location: %USERPROFILE%\.gemini\antigravity\brain\{session-uuid}\.system_generated\logs\transcript.jsonl
Full schema documentation: docs/logFilesSchema/antigravity-session-format.md
Entry types (each line is one entry):
{"step_index":0,"source":"USER_EXPLICIT","type":"USER_INPUT","status":"DONE","created_at":"2026-05-22T21:48:22Z","content":"<USER_REQUEST>\nuser message here\n</USER_REQUEST>\n<ADDITIONAL_METADATA>...</ADDITIONAL_METADATA>"}
{"step_index":1,"source":"SYSTEM","type":"CONVERSATION_HISTORY","status":"DONE","created_at":"2026-05-22T21:48:22Z"}
{"step_index":2,"source":"MODEL","type":"PLANNER_RESPONSE","status":"DONE","created_at":"2026-05-22T21:48:22Z","tool_calls":[{"name":"search_web","args":{"query":"..."}}]}
{"step_index":3,"source":"MODEL","type":"SEARCH_WEB","status":"DONE","created_at":"2026-05-22T21:48:23Z","content":"Search result text..."}
{"step_index":11,"source":"MODEL","type":"PLANNER_RESPONSE","status":"DONE","created_at":"2026-05-22T21:48:44Z","content":"Final answer text...","thinking":"Chain of thought..."}
Key extraction rules (implemented by AntigravityDataAccess in src/antigravity.ts):
- Interactions: count of
USER_INPUT entries (source: "USER_EXPLICIT")
- Session title:
content of first USER_INPUT, strip <USER_REQUEST> XML wrapper
- User message: strip the
<USER_REQUEST>...</USER_REQUEST> wrapper; discard everything after </USER_REQUEST> (metadata blocks)
- Model response:
content field of the last PLANNER_RESPONSE with non-empty content
- Thinking:
thinking field of any PLANNER_RESPONSE
- Tool calls:
tool_calls[].name on PLANNER_RESPONSE entries (e.g. search_web)
- Tool results:
content of SEARCH_WEB entries (and future tool result entry types)
- Actual tokens: always 0 — no token counts in Antigravity transcripts
- Model: not available — not stored in the transcript format
- Session ID: the UUID directory name under
brain/
- Timestamps:
created_at of first and last entries
Pricing Data
Location: src/modelPricing.json
Contains per-million-token costs for input and output:
{
"pricing": {
"gpt-4o": {
"inputCostPerMillion": 1.75,
"outputCostPerMillion": 14.0,
"category": "gpt-4"
}
}
}
Cost Calculation: calculateEstimatedCost()
Location: src/extension.ts (lines 776-802)
Formula:
- Input cost =
(inputTokens / 1_000_000) * inputCostPerMillion
- Output cost =
(outputTokens / 1_000_000) * outputCostPerMillion
- Total cost = input cost + output cost
- Fallback to
gpt-4o-mini pricing for unknown models
Executable Scripts
This skill includes three executable scripts that can be run directly to analyze session files. Always run scripts with their appropriate command first before attempting to read or modify them.
Script 1: Quick Session File Discovery
Purpose: Quickly discover all Copilot session files on your system with summary statistics.
Location: .github/skills/copilot-log-analysis/get-session-files.js
When to use:
- Need a quick overview of session file locations
- Want to know how many session files exist
- Need sample paths for manual inspection
- Troubleshooting why session files aren't being found
Usage:
node .github/skills/copilot-log-analysis/get-session-files.js
node .github/skills/copilot-log-analysis/get-session-files.js --verbose
node .github/skills/copilot-log-analysis/get-session-files.js --json
What it does:
- Scans all VS Code variants (Stable, Insiders, Cursor, VSCodium, etc.)
- Finds files in workspace storage, global storage, and Copilot CLI locations
- Categorizes files by location and editor type
- Shows total counts and sample file paths
Example output:
Platform: win32
Home directory: C:\Users\YourName
VS Code installations found:
C:\Users\YourName\AppData\Roaming\Code\User
C:\Users\YourName\AppData\Roaming\Code - Insiders\User
Total session files found: 274
Session files by location:
Workspace Storage: 192 files
Global Storage (Legacy): 67 files
Copilot Chat Extension: 6 files
Copilot CLI: 9 files
Session files by editor:
VS Code: 265 files
VS Code Insiders: 9 files
Script 2: Detailed Session File Diagnostics
Purpose: Comprehensive diagnostic tool that analyzes session file structure, content, and provides debugging information.
Location: .github/skills/copilot-log-analysis/diagnose-session-files.js
When to use:
- Debugging session file discovery issues
- Need detailed information about session file structure
- Investigating token counting discrepancies
- Troubleshooting parser failures
- Understanding session file metadata and format variations
Usage:
node .github/skills/copilot-log-analysis/diagnose-session-files.js
node .github/skills/copilot-log-analysis/diagnose-session-files.js --verbose
What it does:
- Discovers all session files across VS Code variants
- Reports file locations, counts, and metadata
- Analyzes file structure (JSON vs JSONL format)
- Validates session file integrity
- Provides diagnostic information for troubleshooting
- Shows file modification times and sizes
Script 3: Schema Analysis and Field Discovery
Purpose: PowerShell script that analyzes session files to discover field structures and generate schema documentation.
Location: .github/skills/copilot-log-analysis/analyze-session-schema.ps1
When to use:
- Need to understand the complete structure of session files
- Discovering new fields added by VS Code updates
- Generating schema documentation
- Understanding field variations across different VS Code versions
- Creating or updating schema reference files
Usage:
# Analyze session files and generate schema
pwsh .github/skills/copilot-log-analysis/analyze-session-schema.ps1
# Specify custom output directory
pwsh .github/skills/copilot-log-analysis/analyze-session-schema.ps1 -OutputPath ./output
What it does:
- Scans all discovered session files
- Extracts and catalogs all field names and structures
- Generates JSON schema documentation
- Creates field analysis reports
- Outputs to
docs/logFilesSchema/session-file-schema-analysis.json
- Documents field types, occurrences, and variations
Note: This script generates the session-file-schema-analysis.json file referenced in the Schema Documentation section below.
Usage Examples
Example 1: Finding all session files
const sessionFiles = await getCopilotSessionFiles();
console.log(`Found ${sessionFiles.length} session files`);
Example 2: Analyzing a specific session file
const filePath = '/path/to/session.json';
const stats = fs.statSync(filePath);
const mtime = stats.mtime.getTime();
const content = await fs.promises.readFile(filePath, 'utf8');
const estimate = (text: string, model = 'gpt-4o') => Math.ceil(text.length * 0.25);
const detectModel = (req: any) => req?.result?.metadata?.modelId ?? 'gpt-4o';
const parsed = parseSessionFileContent(filePath, content, estimate, detectModel);
const editorType = getEditorTypeFromPath(filePath);
console.log(`Tokens: ${parsed.tokens}`);
console.log(`Interactions: ${parsed.interactions}`);
console.log(`Editor: ${editorType}`);
console.log(`Models:`, parsed.modelUsage);
Example 3: Processing daily statistics
const now = new Date();
const todayStart = new Date(now.getFullYear(), now.getMonth(), now.getDate());
const sessionFiles = await getCopilotSessionFiles();
const estimate = (text: string, model = 'gpt-4o') => Math.ceil(text.length * 0.25);
const detectModel = (req: any) => req?.result?.metadata?.modelId ?? 'gpt-4o';
let todayTokens = 0;
for (const file of sessionFiles) {
const stats = fs.statSync(file);
if (stats.mtime >= todayStart) {
const content = await fs.promises.readFile(file, 'utf8');
const parsed = parseSessionFileContent(file, content, estimate, detectModel);
todayTokens += parsed.tokens;
}
}
Diagnostic Tools
Output Channel Logging
Location: Throughout src/extension.ts
Methods available:
log(message) (line 146): Info-level loggingwarn(message) (line 151): Warning-level loggingerror(message, error?) (line 156): Error-level logging
All logs go to "GitHub Copilot Token Tracker" output channel.
Diagnostic Report Generation
Method: generateDiagnosticReport()
Location: src/extension.ts (lines 1813-2019)
Creates comprehensive report including:
- System information (OS, Node version, environment)
- GitHub Copilot extension status
- Session file discovery results
- Token usage statistics
- No sensitive data (code/conversations excluded)
Access via:
- Command Palette: "Generate Diagnostic Report"
- Details panel: "Diagnostics" button
File References
When working with log analysis, refer to these files:
-
Main implementation: src/extension.ts
- All field extraction methods
- Session file discovery logic
- Caching implementation
-
Configuration files:
src/tokenEstimators.json - Token estimation ratiossrc/modelPricing.json - Model pricing datasrc/README.md - Data files documentation
-
Schema documentation: docs/logFilesSchema/
- Complete schema reference
- Field analysis tools
- VS Code variant information
-
Skill resources: .github/skills/copilot-log-analysis/
get-session-files.js - Quick session file discovery scriptdiagnose-session-files.js - Detailed diagnostic toolanalyze-session-schema.ps1 - PowerShell schema analysis scriptSKILL.md - This documentation
-
Project instructions: .github/copilot-instructions.md
- Architecture overview
- Development guidelines
Common Issues and Solutions
Issue: No session files found
Solution:
- Run diagnostic script:
node .github/skills/copilot-log-analysis/diagnose-session-files.js
- Check if Copilot Chat extension is active
- Verify user has started at least one Copilot Chat session
- Check OS-specific paths are correct
Issue: Token counts seem incorrect
Solution:
- Verify
tokenEstimators.json has correct ratios for models
- Check if new models need to be added
- Review session file content to verify expected structure
- Check cache hasn't become stale (cache uses mtime)
Issue: Model not detected properly
Solution:
- Check
getModelFromRequest() detection logic
- Review
request.result.details string patterns
- Add new model pattern if needed
- Update
modelPricing.json with new model
Notes
- All file paths must be absolute
- Token estimation is approximate (character-based)
- Caching significantly improves performance
- Session files grow over time as conversations continue
- JSONL format is newer (Copilot CLI/Agent mode)
- The extension processes files sequentially with progress callbacks
