session-analysis

name	session-analysis
description	Analyze Foreman session logs to extract costs, errors, conversation flow, token usage, and agent behavior patterns. Use when asked about session costs, debugging failed sessions, auditing agent behavior, generating usage reports, or reviewing what happened in past sessions. Session logs are JSONL files at ~/foreman-logs/YYYY/MM/DD/session-name.jsonl.

Session Analysis

Analyze Foreman agent session logs (JSONL) without blowing up the context window.

Log Files Are Live

Session log files are append-only and grow in real time while a session is active. If a log file has no terminal event (session_end, timeout, phonehome_logs, or agent_end_session), the session is likely still running — do not report it as dead or crashed. The agent may simply be blocked on a long-running operation. Re-check the file if you need the latest state.

Critical Rule: Protect Your Context Window

Session logs can be very large (100KB–1MB+). NEVER read raw log files directly into context. Always use the bundled scripts to extract only what you need.

Analysis is well-suited to be delegated to subagents, and when the requirement is simple you should run cheap and fast subagents (that can work on session files in parallel!).

Bundled Scripts

All scripts are in the skill's scripts/ directory. Run them with python3.

session_summary.py — Quick overview (start here)

python3 scripts/session_summary.py ~/foreman-logs/2026/02/12/*.jsonl

Shows per-session: name, duration, cost, tokens, tool call counts, errors, user messages. Accepts one or many files. Use this first to identify interesting sessions.

extract_conversation.py — Readable conversation timeline

python3 scripts/extract_conversation.py <session.jsonl>
python3 scripts/extract_conversation.py <session.jsonl> --with-results           # include tool outputs
python3 scripts/extract_conversation.py <session.jsonl> --intermediate-messages   # include intermediate agent messages

Shows the flow: user messages → tool calls → agent responses. Omits bulky tool results by default. Use --with-results only when debugging specific tool failures.

extract_errors.py — Find errors and retry storms

python3 scripts/extract_errors.py <session.jsonl>

Shows all tool errors, session crashes, and detects retry storms (≥3 consecutive errors on the same tool — a sign the agent is stuck in a loop).

cost_report.py — Cost and token usage across sessions

python3 scripts/cost_report.py ~/foreman-logs/2026/02/12/*.jsonl
python3 scripts/cost_report.py ~/foreman-logs/2026/02/12/*.jsonl --by-user
python3 scripts/cost_report.py ~/foreman-logs/2026/02/12/*.jsonl --sort tokens

Table of sessions sorted by cost (default) or token count. --by-user breaks down cost by user.

Workflow

1. Start with session_summary.py on the target files to get the lay of the land.
2. Drill into specific sessions with extract_conversation.py to understand agent behavior.
3. Use extract_errors.py to diagnose failures, retry storms, or wasted tokens.
4. Use cost_report.py for aggregate cost/token analysis.

When analyzing many sessions (e.g. "audit all of today's sessions"):

• Run session_summary.py on all files first — its output is compact.
• Only drill into sessions that look problematic (high cost, many errors, unusual tool counts).
• If you need to delegate per-session analysis to subagents, keep summaries short and do NOT pass raw log content through the parent context.

Log Format

For the complete JSONL event schema, see references/log-format.md.