agentobservability

name

AgentObservability

description

Real-time observability dashboard for multi-agent Claude Code sessions. Visualize agent interactions, tool usage, and session flows in real-time through a web dashboard. Track multiple agents running in parallel with swim lane visualization, event filtering, and live charts. USE WHEN user says 'start observability', 'agent dashboard', 'monitor agents', 'watch agent activity', 'multi-agent monitoring', 'track subagents', or needs to debug multi-agent workflows. **Key Features:** - 🔴 Real-time event streaming via WebSocket - 📊 Agent swim lanes showing parallel execution - 🔍 Event filtering by agent, session, event type - 📈 Live charts for tool usage patterns - 💾 Filesystem-based (no database required) **Inspired by [@indydevdan](https://github.com/indydevdan)**'s work on multi-agent observability. **Our approach:** Filesystem + in-memory streaming vs. indydevdan's SQLite database approach.

Agent Observability Skill

When to Activate This Skill

• User wants to monitor agent activity
• Need to debug multi-agent workflows
• Track parallel subagent execution
• Visualize tool usage patterns
• Analyze session flows in real-time

Prerequisites

• Bun runtime installed
• Claude Code with hooks configured
• PAI_DIR environment variable set

Installation

See SETUP.md for complete installation instructions.

Quick Setup:

# 1. Set environment variable
export PAI_DIR="$HOME/.claude"  # Add to ~/.zshrc or ~/.bashrc

# 2. Configure hooks (merge into ~/.claude/settings.json)
cat settings.json.example

# 3. Create directory structure
mkdir -p ~/.claude/history/raw-outputs

# 4. Install dependencies
cd apps/server && bun install
cd ../client && bun install

Usage

Start the Observability Dashboard

Terminal 1 - Server:

cd ~/Projects/PAI/skills/agent-observability/apps/server
bun run dev

Terminal 2 - Client:

cd ~/Projects/PAI/skills/agent-observability/apps/client
bun run dev

Open browser: http://localhost:5173

Using Claude Code

Once the dashboard is running, any Claude Code activity will appear in real-time:

1. Open Claude Code
2. Use any tool (Read, Write, Bash, etc.)
3. Launch subagents with Task tool
4. Watch events appear in the dashboard

Event Types Captured

• SessionStart - New Claude Code session begins
• UserPromptSubmit - User sends a message
• PreToolUse - Before a tool is executed
• PostToolUse - After a tool completes
• Stop - Main agent task completes
• SubagentStop - Subagent task completes
• SessionEnd - Session ends

Features

Real-Time Visualization

• Agent Swim Lanes: See multiple agents (kai, designer, engineer, etc.) running in parallel
• Event Timeline: Chronological view of all events
• Tool Usage Charts: Visualize which tools are being used most
• Session Tracking: Track individual sessions and their lifecycles

Filtering & Search

• Filter by agent name (kai, designer, engineer, pentester, etc.)
• Filter by event type (PreToolUse, PostToolUse, etc.)
• Filter by session ID
• Search event payloads

Data Storage

Events are stored in JSONL (JSON Lines) format:

~/.claude/history/raw-outputs/YYYY-MM/YYYY-MM-DD_all-events.jsonl

Each line is a complete JSON object:

{"source_app":"kai","session_id":"abc123","hook_event_type":"PreToolUse","payload":{...},"timestamp":1234567890,"timestamp_pst":"2025-01-28 14:30:00 PST"}

In-Memory Streaming

• Server keeps last 1000 events in memory
• Low memory footprint
• Fast real-time updates via WebSocket
• No database overhead

Architecture

┌─────────────────┐
│  Claude Code    │  Executes hooks on events
│   (with hooks)  │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│ capture-all-    │  Appends events to JSONL
│ events.ts hook  │
└────────┬────────┘
         │
         ▼
┌─────────────────────────────────────┐
│ ~/.claude/history/raw-outputs/      │  Daily JSONL files
│ 2025-01/2025-01-28_all-events.jsonl │
└────────┬────────────────────────────┘
         │
         ▼
┌─────────────────┐
│ file-ingest.ts  │  Watches files, streams to memory
│  (Bun server)   │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│  Vue 3 Client   │  Real-time dashboard visualization
│  (Vite + Tail)  │
└─────────────────┘

Configuration

Environment Variables

PAI_DIR: Path to your PAI directory (defaults to ~/.claude/)

export PAI_DIR="/Users/yourname/.claude"

Hooks Configuration

Add to ~/.claude/settings.json (see settings.json.example for full template):

{
  "hooks": {
    "PreToolUse": [{
      "matcher": "*",
      "hooks": [{
        "type": "command",
        "command": "${PAI_DIR}/skills/agent-observability/hooks/capture-all-events.ts --event-type PreToolUse"
      }]
    }],
    // ... other hooks
  }
}

Troubleshooting

No events appearing

1. Check PAI_DIR is set: echo $PAI_DIR
2. Verify directory exists: ls ~/.claude/history/raw-outputs/
3. Check hook is executable: ls -l hooks/capture-all-events.ts
4. Look for today's events file: ls ~/.claude/history/raw-outputs/$(date +%Y-%m)/

Server won't start

1. Check Bun is installed: bun --version
2. Verify dependencies: cd apps/server && bun install
3. Check port 3001 isn't in use: lsof -i :3001

Client won't connect

1. Ensure server is running first
2. Check WebSocket connection in browser console
3. Verify no firewall blocking localhost:3001

Credits

Inspired by @indydevdan's pioneering work on multi-agent observability for Claude Code.

Our implementation differs by using filesystem-based event capture and in-memory streaming instead of SQLite database persistence. Both approaches have their merits! Check out indydevdan's work for a database-backed solution with full historical persistence.

Development

Running in Development

# Server (hot reload)
cd apps/server
bun --watch src/index.ts

# Client (Vite dev server)
cd apps/client
bun run dev

Building for Production

# Client build
cd apps/client
bun run build
bun run preview

Adding New Event Types

1. Update capture-all-events.ts hook if needed
2. Add hook configuration to settings.json
3. Client will automatically display new event types

Documentation

• README.md - Complete documentation
• SETUP.md - Installation guide
• history-structure/ - Data storage structure
• settings.json.example - Hook configuration template

License

Part of the PAI (Personal AI Infrastructure) project.

Contributing

Contributions welcome! Areas for improvement:

• Historical data persistence options
• Export functionality (CSV, JSON)
• Alert/notification system
• Advanced filtering and search
• Session replay capability
• Integration with other PAI skills