monitoring-observability

Monitoring and observability patterns for Prometheus metrics, Grafana dashboards, Langfuse v4 LLM tracing (as_type, score_current_span, should_export_span, LangfuseMedia), and drift detection. Use when adding logging, metrics, distributed tracing, LLM cost tracking, or quality drift monitoring.

Ir para a instalação

Repositório: yonatangross/orchestkit
Última atividade na origem: 5 de agosto de 2026 às 16:40
Idioma detectado do SKILL.md: inglês
Estrelas: 219
Forks: 20

Opções de instalação

Por padrão, está selecionado o prompt que primeiro revisa a origem. Você pode mudar para um comando direto ou baixar uma cópia local.

Instalar com Codex ou Claude Copie este prompt, cole no Codex, Claude ou outro assistente e deixe que ele revise a página da skill e instale para você.

Um comando direto ignora o prompt de revisão. Verifique a origem antes de executá-lo.

npx skills add https://github.com/yonatangross/orchestkit --skill monitoring-observability

O comando permanece em uma só linha. Role horizontalmente para revisá-lo antes de copiar.

Prefere uma cópia local? Baixe os arquivos disponíveis atualmente no SkillsMP.

Revise os arquivos de origem

Leia o SKILL.md e os arquivos complementares exibidos pelo SkillsMP antes de decidir se vai instalar.

Explorador de arquivos

27 arquivos

SKILL.md

Instruções da origem · Visualização somente leitura

Voltar à pesquisa Voltar à instalação

chain-patterns

Chain patterns for multi-phase pipelines: MCP detection, handoff files, checkpoint-resume, worktree agents, CronCreate monitoring. Use when building or debugging a pipeline skill.

2026-08-12

doctor

OrchestKit doctor for health diagnostics across manifest integrity, hook configuration, skill validation, agent frontmatter, MCP server connectivity, CC version compatibility, and permission rules. Reports issues with severity levels and auto-remediation suggestions. Validates component counts, detects orphaned entries, and checks CC version matrix compliance. Use when diagnosing plugin health, troubleshooting configuration issues, or running pre-release checks.

2026-08-12

telemetry-inspect

Inspects the OrchestKit telemetry pipeline for the current project — lists all known telemetry files with write counts, sizes, schema status, growth trend, and orphan detection. Use when verifying the observability pipeline is healthy, debugging a missing writer, or auditing which files have schema locks vs. which are drift-vulnerable. Read-only — never modifies telemetry files.

2026-08-12

Ver todas deste repositório

Monitoring & Observability

A wrap around Prometheus, Grafana, OpenTelemetry and Langfuse, not a re-teaching of them. This skill carries OrchestKit's delta (version floors, house decisions, scars) and points at the vendor for everything else. Start at references/ork-delta.md.

Upstream coverage (do not restate)

These topics are fully covered first-party. Read the source, do not add a local copy.

Topic

First-party source

Prometheus metric types, RED method, cardinality, PromQL

https://prometheus.io/docs/practices/

Alertmanager grouping, inhibition, escalation, runbooks

https://prometheus.io/docs/alerting/latest/configuration/

Grafana dashboards, Loki and LogQL, Promtail

https://grafana.com/docs/

OpenTelemetry spans, sampling, context propagation

https://opentelemetry.io/docs/

Langfuse Python SDK (@observe, as_type, score_current_span, should_export_span, LangfuseMedia)

https://langfuse.com/docs/sdk/python

Langfuse v2 to v4 Python and v3 to v5 JS migration paths

https://langfuse.com/docs/sdk/python/v4-migration

Langfuse self-hosting (ClickHouse, Redis, S3, Helm)

https://langfuse.com/docs/deployment/self-host

Langfuse cost tracking, model pricing, Metrics API v2

https://langfuse.com/docs/model-usage-and-cost

Langfuse scores, online evaluators, annotation queues, prompt management

https://langfuse.com/docs/scores/overview

Langfuse framework integrations (LangChain, LangGraph, CrewAI, Pydantic AI, Bedrock, LiveKit)

https://langfuse.com/docs/integrations

Agent Graphs, observation types, rendered tool calls

Monitoring & Observability

Upstream coverage (do not restate)

These topics are fully covered first-party. Read the source, do not add a local copy.

Topic

First-party source

Prometheus metric types, RED method, cardinality, PromQL

https://prometheus.io/docs/practices/

Alertmanager grouping, inhibition, escalation, runbooks

https://prometheus.io/docs/alerting/latest/configuration/

Grafana dashboards, Loki and LogQL, Promtail

https://grafana.com/docs/

OpenTelemetry spans, sampling, context propagation

https://opentelemetry.io/docs/

Langfuse Python SDK (@observe, as_type, score_current_span, should_export_span, LangfuseMedia)

https://langfuse.com/docs/sdk/python

Langfuse v2 to v4 Python and v3 to v5 JS migration paths

https://langfuse.com/docs/sdk/python/v4-migration

Langfuse self-hosting (ClickHouse, Redis, S3, Helm)

https://langfuse.com/docs/deployment/self-host

Langfuse cost tracking, model pricing, Metrics API v2

https://langfuse.com/docs/model-usage-and-cost

Langfuse scores, online evaluators, annotation queues, prompt management

https://langfuse.com/docs/scores/overview

Langfuse framework integrations (LangChain, LangGraph, CrewAI, Pydantic AI, Bedrock, LiveKit)

https://langfuse.com/docs/integrations

Agent Graphs, observation types, rendered tool calls

name	monitoring-observability
license	MIT
compatibility	Claude Code 2.1.220+.
description	Monitoring and observability patterns for Prometheus metrics, Grafana dashboards, Langfuse v4 LLM tracing (as_type, score_current_span, should_export_span, LangfuseMedia), and drift detection. Use when adding logging, metrics, distributed tracing, LLM cost tracking, or quality drift monitoring.
tags	["monitoring","observability","prometheus","grafana","langfuse","tracing","metrics","drift-detection","logging"]
context	fork
version	3.0.0
author	OrchestKit
user-invocable	false
disable-model-invocation	true
complexity	medium
persuasion-type	reference
targets	[{"library":"langfuse","version":">=4.0.0"}]
upstream-version-tested	4.14.2
metadata	{"category":"document-asset-creation"}
allowed-tools	["Read","Glob","Grep","WebFetch","WebSearch"]
path_patterns	["/metrics/","/tracing/","prometheus.","grafana/*"]

name	monitoring-observability
license	MIT
compatibility	Claude Code 2.1.220+.
description	Monitoring and observability patterns for Prometheus metrics, Grafana dashboards, Langfuse v4 LLM tracing (as_type, score_current_span, should_export_span, LangfuseMedia), and drift detection. Use when adding logging, metrics, distributed tracing, LLM cost tracking, or quality drift monitoring.
tags	["monitoring","observability","prometheus","grafana","langfuse","tracing","metrics","drift-detection","logging"]
context	fork
version	3.0.0
author	OrchestKit
user-invocable	false
disable-model-invocation	true
complexity	medium
persuasion-type	reference
targets	[{"library":"langfuse","version":">=4.0.0"}]
upstream-version-tested	4.14.2
metadata	{"category":"document-asset-creation"}
allowed-tools	["Read","Glob","Grep","WebFetch","WebSearch"]
path_patterns	["/metrics/","/tracing/","prometheus.","grafana/*"]

Category	Rules	Impact	When to Use
Infrastructure Monitoring	1	CRITICAL	Grafana dashboards, Golden Signals, SLO/SLI
LLM Observability	1	HIGH	Langfuse tracing, observation types, agent graphs
Silent Failures	3	HIGH	Tool skipping, quality degradation, loop/token spike alerting

Rule	File	Key Pattern
Tool Skipping	`rules/silent-tool-skipping.md`	Expected vs actual tool calls, Langfuse traces
Quality Degradation	`rules/silent-degraded-quality.md`	Heuristics + LLM-as-judge, z-score baselines
Silent Alerting	`rules/silent-alerting.md`	Loop detection, token spikes, escalation workflow

Decision	Recommendation	Rationale
Metric methodology	RED method (Rate, Errors, Duration)	Industry standard, covers essential service health
Log format	Structured JSON	Machine-parseable, supports log aggregation
Tracing	OpenTelemetry	Vendor-neutral, auto-instrumentation, broad ecosystem
LLM observability	Langfuse (not LangSmith)	Open-source, self-hosted, built-in prompt management
LLM tracing API	`@observe(as_type=...)` + `score_current_span()`	v4: semantic types, inline scoring, span filtering
Langfuse APIs	Observations API v2 + Metrics API v2	v4 (Mar 2026): faster querying, aggregations at scale
Hook telemetry transport	JSONL under `~/.claude/analytics/`, never an SDK in-process	Hooks are per-event processes; SDK init would be paid on every spawn (`references/ork-delta.md`)

monitoring-observability

Opções de instalação

Revise os arquivos de origem