| name | observability-and-telemetry |
| description | Designs actionable logs, metrics, traces, dashboards and alerts for reliable operations. Use when instrumenting services, defining SLIs, or improving on-call signal quality. |
| aliases | ["improve-telemetry","telemetry"] |
Observability and Telemetry
When to use
Use this skill when adding or reviewing logs, metrics, traces, dashboards, alerts, telemetry standards, OpenTelemetry instrumentation, service health checks, operational evidence or workflow monitoring.
Objective
Make service behaviour understandable from the outside so operators can detect, diagnose and respond to user-impacting issues quickly.
Procedure
- Identify the user journey or service behaviour to observe.
- Define the key questions operators need to answer.
- Select metrics for symptoms and causes.
- Add structured logs for important decisions and failures.
- Add traces across service boundaries and tool calls.
- Create dashboards around user journeys and SLOs.
- Add alerts only when action is required.
- Include runbook links in alerts.
- Avoid logging secrets or unnecessary sensitive data.
- Validate telemetry in local or test environment.
Telemetry types
| Type | Use for | Anti-pattern |
|---|
| Metrics | trends, rates, thresholds, SLOs | high-cardinality unbounded labels |
| Logs | discrete events, decisions, errors | unstructured noisy dumps |
| Traces | request/workflow path and latency | tracing without useful spans |
| Dashboards | operational review and triage | vanity metrics |
| Alerts | urgent action | non-actionable noise |
Required span attributes for workflows
workflow_id: "uuid"
request_id: "uuid"
source_id: "uuid"
batch_id: "uuid"
tenant_id: "string"
workflow_type: "string"
step_name: "string"
status: "success | failure | partial"
error_type: "string"
Optional overlay
For product-specific DataOps/MCP examples, load skills_docs/overlays/mas-dataops-mcp-overlay.md.
Alert quality test
Before adding an alert, answer what user impact it represents, who owns it, what action the responder should take, what dashboard or runbook helps, what threshold avoids noise and how the alert will be tested.
Rules
- Do not log secrets, tokens, credentials or raw sensitive records.
- Do not alert on every internal cause if one user-impacting symptom alert is enough.
- Do not create dashboards with no decisions attached.
- Do not use unbounded labels such as raw user ID, record ID or SQL text.
- Do not mark a service healthy if dependencies required for readiness are unavailable.
- Do not rely only on logs where metrics or traces are needed.
References
Verification