monitoring-stack-deployer

license	Apache-2.0
name	monitoring-stack-deployer
description	Production monitoring stack deployer with Prometheus, Grafana, and SLO-based alerting. Activate on: monitoring setup, Prometheus configuration, Grafana dashboards, alerting rules, SLO definition, metrics pipeline, observability stack. NOT for: application logging (use log-aggregation-architect), distributed tracing (use logging-observability), incident response (use site-reliability-engineer).
allowed-tools	Read,Write,Edit,Bash(docker:*,kubectl:*,terraform:*,npm:*,npx:*)
category	DevOps & Infrastructure
tags	["monitoring","prometheus","grafana","observability"]
pairs-with	[{"skill":"site-reliability-engineer","reason":"SRE practices depend on monitoring stack for SLOs and incident detection"},{"skill":"log-aggregation-architect","reason":"Logging and metrics pipelines often share infrastructure"}]

Monitoring Stack Deployer

Expert in deploying and configuring production monitoring with Prometheus, Grafana, and SLO-driven alerting.

Activation Triggers

Activate on: "monitoring setup", "Prometheus config", "Grafana dashboard", "alerting rules", "SLO dashboard", "metrics pipeline", "observability stack", "kube-prometheus-stack", "ServiceMonitor"

NOT for: Application logging → log-aggregation-architect | Distributed tracing → logging-observability | Incident response → site-reliability-engineer

Quick Start

1. Deploy kube-prometheus-stack — Prometheus, Grafana, Alertmanager, node-exporter in one Helm chart
2. Define SLOs — availability and latency targets per service
3. Create ServiceMonitors — auto-discover application metrics endpoints
4. Build dashboards — USE method (utilization, saturation, errors) for infrastructure; RED method (rate, errors, duration) for services
5. Configure alerting — SLO burn-rate alerts, not threshold alerts

Core Capabilities

Domain	Technologies
Metrics	Prometheus 3.x, Mimir, Thanos, VictoriaMetrics
Visualization	Grafana 11, Perses (open-source Grafana alternative)
Alerting	Alertmanager, PagerDuty, OpsGenie, Slack integration
SLOs	Sloth, Pyrra, Google SRE workbook burn-rate model
K8s Native	kube-prometheus-stack, ServiceMonitor, PodMonitor, PrometheusRule

Architecture Patterns

SLO-Based Burn-Rate Alerting

Traditional (BAD):  "Alert if error rate > 1% for 5 minutes"
  Problem: Too many false positives, alert fatigue

SLO-Based (GOOD):  "Alert if burning SLO budget too fast"
  SLO: 99.9% availability over 30 days → 43.2 min error budget

  Multi-window burn rate:
  ┌─────────────────────────────────────────────┐
  │ Severity │ Burn Rate │ Long Window │ Short  │
  │ Critical │ 14.4x     │ 1 hour      │ 5 min  │
  │ Warning  │ 6x        │ 6 hours     │ 30 min │
  │ Ticket   │ 1x        │ 3 days      │ 6 hrs  │
  └─────────────────────────────────────────────┘

Prometheus Recording Rules for SLOs