exploring-ai-failures

Find where an AI/LLM application is failing in production and surface the failure patterns, working from real traces. Use when someone wants to understand what's going wrong with an AI feature, find and categorize failure modes, triage errors, or investigate quality issues (wrong answers, ignored instructions, hallucinations, tool misuse) — "what's failing in my agent", "surface error patterns", "why are the responses bad", "find the common failure modes", "what should I fix next". Covers scoping to one use case, finding failing traces by whichever signal fits the context (code errors, metric outliers, trace-type slices, manual review, existing-eval spikes, clustering), and reading them into a ranked failure taxonomy.

Jump to install Review files

Repository: PostHog/ai-plugin
Last source activity: November 4, 2026 at 12:05
Detected SKILL.md language: English
Stars: 100
Forks: 21

Install options

The review-first prompt is selected by default. You can switch to a direct command or download a local copy.

Install with an AI assistant

Copy this prompt into the AI assistant you're using.

Follow https://skillsmp.com/skill-install/prompt.md to install only the Agent Skill at https://github.com/PostHog/ai-plugin/tree/main/skills/exploring-ai-failures.

A direct command skips the review prompt. Inspect the source before running it.

npx skills add https://github.com/PostHog/ai-plugin --skill exploring-ai-failures

The command stays on one line. Scroll horizontally to inspect it before copying.

Prefer a local copy? Download the files currently available to SkillsMP.

Review the source files

Read SKILL.md and any companion files shown by SkillsMP before deciding whether to install.

File Explorer

2 files

SKILL.md

Source instructions · Read-only preview

name

exploring-ai-failures

description

Back to search Back to installation

authoring-insight-alerts

Author insight alerts that notify when a trends, SQL, or funnel insight crosses a bound, changes sharply, or looks anomalous. Use when the user asks to alert on a metric or insight, get notified when a number drops or spikes, set up anomaly detection, or use the AI anomaly detector (the llm detector type). Covers choosing between threshold, relative change, statistical detector, and AI detector, previewing with alert-simulate before saving, the AI detector's credit cost and limits, the exact create payload, and delivery by email or Slack.

2026-11-08

inbox-exploration

Explore PostHog's Inbox and act on what it surfaces — the place where signal reports cluster into actionable issues and trends. Use when the user asks "what's in my inbox?", "what should I look at?", "which reports are actionable?", "what's PostHog flagged recently?", asks about a specific report by ID or title, wants to act on / fix / implement a report (turn it into a PR), wants to resolve, dismiss, or snooze a report, or wants to see which signal sources are configured. Also use before you investigate an error, bug, metric change, or support ticket in code, to find research the Inbox already did. Covers listing, filtering, drilling into, and acting on reports, plus pointers to the deeper `signals` skill when raw signals or semantic search are needed. For a personal, ranked top N of reports, use `prioritizing-my-inbox`, which starts from the stored personal ranking (`view=for_you`, `sort=personal`).

2026-11-08

prioritizing-my-inbox

Give the current user a personal, ranked shortlist of the PostHog Inbox reports they are most likely to care about right now. Reads their stored personal ranking first, with one `inbox-reports-list` call (`view=for_you`, `sort=personal`), and keeps its order and reasons. Uses `inbox-reports-for-me` for a fresh or focused list, and builds a ranking by hand from the activity summary and the ranking model's lifts only when no stored ranking applies or the user asks for custom weights. Use when the user says "let's look at my inbox", "what's in my inbox for me?", "my top 10 inbox reports", "what should I look at today?", or asks to re-rank, focus, or filter that list. For browsing, filtering, drilling into, or acting on a single report, use `inbox-exploration`.

2026-11-08

View all from this repository

exploring-ai-failures

Source facts

Install options

Install with an AI assistant

Review the source files

More from this repository

Install with an AI assistant

More from this repository