Implement AI chatbot analytics and conversation monitoring. Use when adding conversation metrics, tracking AI usage, measuring user engagement with chat, or building conversation dashboards. Activates for AI analytics, token tracking, conversation categorization, and chat performance.
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
A direct command skips the review prompt. Inspect the source before running it.
Implement AI chatbot analytics and conversation monitoring. Use when adding conversation metrics, tracking AI usage, measuring user engagement with chat, or building conversation dashboards. Activates for AI analytics, token tracking, conversation categorization, and chat performance.
allowed-tools
Read,Write,Edit,Bash(npm:*,npx:*)
metadata
{"category":"Data & Analytics","tags":["analytics","chatbot","ai-metrics"],"pairs-with":[{"skill":"llm-streaming-response-handler","reason":"Streaming response metrics (TTFT, tokens/sec) are key chatbot analytics dimensions"},{"skill":"data-pipeline-engineer","reason":"Conversation data pipelines feed analytics dashboards and reporting systems"},{"skill":"prompt-engineer","reason":"Analytics on prompt effectiveness drive iterative prompt optimization"}]}
AI Chatbot Analytics
This skill helps you implement analytics for the AI coaching chat feature while maintaining HIPAA compliance.
// NEVER store these in analyticsinterface PROHIBITED {
messageContent: string; // PHIuserQuery: string; // PHIaiResponse: string; // PHIspecificTopics: string[]; // Could reveal health infoexactSentiment: 'sad'; // Could reveal mental state
}
Detect conversation categories WITHOUT reading content:
// Categories based on metadata flags from AI responseinterfaceAIResponseMetadata {
usedCopingStrategies: boolean;
usedCrisisProtocol: boolean;
usedCheckInSupport: boolean;
usedGeneralChat: boolean;
requestedClarification: boolean;
}
functionderiveCategory(metadata: AIResponseMetadata): string {
if (metadata.usedCrisisProtocol) return'crisis_support';
if (metadata.usedCopingStrategies) return'coping_strategies';
if (metadata.usedCheckInSupport) return'checkin_support';
if (metadata.requestedClarification) return'clarification';
return'general_chat';
}
Dashboard Aggregations
Session Metrics
// Get aggregated session stats (HIPAA safe - no individual data)asyncfunctiongetSessionStats(days: number = 30) {
const since = subDays(newDate(), days);
return db
.select({
totalSessions: count(),
avgMessages: avg(conversationAnalytics.messageCount),
avgDuration: avg(
sql`JULIANDAY(ended_at) - JULIANDAY(started_at)) * 24 * 60`
),
completionRate: sql`
CAST(SUM(CASE WHEN outcome = 'completed' THEN 1 ELSE 0 END) AS FLOAT) /
CAST(COUNT(*) AS FLOAT)
`,
crisisEscalations: sql`
SUM(CASE WHEN outcome = 'crisis_escalated' THEN 1 ELSE 0 END)
`
})
.from(conversationAnalytics)
.where(gte(conversationAnalytics.startedAt, since));
}
CREATE TABLE conversation_analytics (
id TEXT PRIMARY KEY,
conversation_id TEXT NOT NULL,
user_id TEXT NOT NULL,
started_at TEXT NOT NULL,
ended_at TEXT,
message_count INTEGERDEFAULT0,
user_message_count INTEGERDEFAULT0,
ai_message_count INTEGERDEFAULT0,
total_tokens INTEGERDEFAULT0,
input_tokens INTEGERDEFAULT0,
output_tokens INTEGERDEFAULT0,
category TEXT DEFAULT'unknown',
outcome TEXT DEFAULT'in_progress',
avg_response_time REALDEFAULT0,
had_fallback INTEGERDEFAULT0,
FOREIGN KEY (conversation_id) REFERENCES conversations(id),
FOREIGN KEY (user_id) REFERENCES users(id)
);
CREATE INDEX idx_conv_analytics_started ON conversation_analytics(started_at);
CREATE INDEX idx_conv_analytics_user ON conversation_analytics(user_id);
CREATE INDEX idx_conv_analytics_outcome ON conversation_analytics(outcome);
