observability-debugging

name	observability-debugging
description	Debug using logs, traces, metrics, and telemetry — runtime behavior analysis, structured logging, and production debugging. Trigger on /observability-debugging, or when debugging from logs, adding tracing, or investigating runtime behavior.
user-invocable	true
allowed-tools	Read, Grep, Glob, Bash

Observability Debugging

Debug using runtime signals — logs, traces, metrics. Different from debug-root-cause which focuses on code path analysis; this skill focuses on production/runtime observability.

Workflow

1. Identify the signal

• What is available? (logs, traces, metrics, dashboards, alerts)
• What is the symptom? (high latency, error rate, memory growth, timeout)
• What time range? Correlate with deployments, config changes, traffic shifts.

2. Log analysis

• Search for error patterns, stack traces, unusual log levels.
• Filter noise — ignore expected warnings, focus on anomalies.
• Correlate timestamps across services.
• Look for logs that changed pattern after a deployment.

3. Structured logging

• Ensure logs include: timestamp, level, service, request ID, error context.
• Recommend structured logging (JSON) over unstructured text.
• Avoid logging sensitive data — PII, tokens, credentials.

4. Tracing

• Trace a single request across service boundaries.
• Identify where time is spent — which span is the bottleneck.
• Check for missing spans, incomplete traces, sampling issues.

5. Metrics

• Correlate metrics (CPU, memory, request rate, error rate) with the timeline.
• Look for step functions (sudden change) vs gradual trends.
• Check if the metric is correctly instrumented.

Rules

• Do not modify production logging without explicit approval.
• Do not expose sensitive data in logs or traces.
• Distinguish between application logs, infra logs, and audit logs.
• Recommend improvements, but implement only when scoped.

Experience Log

This skill learns from experience. Mechanism:

• Read .claude/experience/'"$s"'.md at skill start to benefit from past lessons.
• Write to .claude/experience/'"$s"'.md after use if you learned something new.
• Format: YYYY-MM-DD: <lesson> — one line per entry.
• Evolve: If the same issue repeats 3+ times, update this SKILL.md itself.