Menu
Debug and analyze LLM eval runs — view traces, compare runs, investigate failures, track costs. Use when debugging @kbn/evals failures, comparing eval runs, or analyzing LLM performance.
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
Based on SOC occupation classification
Start your security session with a personalized briefing — attacks, alerts, cases, rules, threat intel. Use as the first thing when starting security work.
Guide users from zero to a working Elastic cluster — Cloud or on-prem, connection config, first queries, and next steps.
Interactive guide for creating an APM service overview dashboard — discovers service data, presents metrics, and creates a tailored dashboard.
Interactive guide for creating SLOs from discovered APM and metric data — identifies candidates, lets user configure targets, and creates SLOs.
Create, configure, and manage Elasticsearch indices — mappings, settings, templates, data streams, and lifecycle policies.
Interactive workflow for investigating and triaging security alerts — from alert review through enrichment, investigation, and resolution.
| name | llm-eval-observatory |
| description | Debug and analyze LLM eval runs — view traces, compare runs, investigate failures, track costs. Use when debugging @kbn/evals failures, comparing eval runs, or analyzing LLM performance. |
Use eval_observatory to open an interactive 6-tab LLM observability dashboard.
| Tool | Purpose |
|---|---|
eval_observatory | Open the eval observatory dashboard |
In the Failure Investigation tab, click "Debug with Claude" on any failing eval. The full context (input, expected output, trace, evaluator reasoning) is sent to this conversation so I can analyze why it failed and suggest fixes.