Skip to main content
prometheus-grafana Expert skill for Prometheus metrics and Grafana dashboards. Write and validate PromQL queries, generate Grafana dashboard JSON, create alerting and recording rules, analyze metric cardinality, and debug scrape configurations.
インストールへ移動 Skills Marketplace コミュニティが作成したAIスキルを発見・探索
Codex または Claude でインストール この Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。
直接コマンドでは確認用 Prompt が省略されます。実行前にソースを確認してください。
npx skills add https://github.com/a5c-ai/babysitter --skill prometheus-grafanaコマンドは1行のまま表示されます。コピー前に横へスクロールして全体を確認してください。
ローカルで確認しますか?SkillsMP が現在取得できるファイルをダウンロードできます。
Zipをダウンロード ダウンロード中... このリポジトリの他の Skills Reference for querying the Atlas knowledge graph through its MCP tools — the SECONDARY enrichment/comparison layer that adds best-practice context to systems you have ALREADY scanned from your real sources (`az`, repos, dirs). Use when you need to look up nodes, edges, kinds, clusters, stats, or wiki pages in Atlas to compare against your real inventory. (atlas graph, query atlas, atlas mcp, search the graph, graph neighbors, atlas record, atlas kinds, enrichment layer)
Atlas turns your STATED NEED into a real systems atlas by SCANNING your actual sources (Azure via `az`, git repos, local dirs) and process/data mining them, THEN enriching against the Atlas knowledge graph. Use this skill when asked to inventory/map your real systems, scan your cloud + repos + directories, mine the real processes or data they contain, or collect their real constraints/gotchas. (atlas, scan my systems, inventory our azure account, map my repos, real systems atlas, process mining, data mining, collect nuances, system discovery)
assimilate-popular-workflows This skill should be used when the user asks to "find skills in the wild", "assimilate popular workflows", "discover SKILL.md files in repos", "research external skills", "find workflow patterns", "survey the skill landscape", "what skills exist out there", or wants to investigate public repositories for extractable processes, babysitter plugins, and reusable procedural insights. Searches GitHub for SKILL.md files, classifies repos by archetype, and maintains structured research under docs/reference-repos/.
name prometheus-grafana description Expert skill for Prometheus metrics and Grafana dashboards. Write and validate PromQL queries, generate Grafana dashboard JSON, create alerting and recording rules, analyze metric cardinality, and debug scrape configurations. allowed-tools Bash(*) Read Write Edit Glob Grep WebFetch metadata {"author":"babysitter-sdk","version":"1.0.0","category":"observability","backlog-id":"SK-003"} graph {"domains":["domain:devops"],"specializations":["specialization:devops-sre-platform"],"skillAreas":["skill-area:observability-instrumentation","skill-area:metrics-dashboarding"],"roles":["role:sre","role:devops-engineer"],"topics":["topic:observability-driven-development"]}
prometheus-grafana
You are prometheus-grafana - a specialized skill for Prometheus metrics and Grafana dashboards. This skill provides expert capabilities for building and maintaining observability infrastructure.
Overview
This skill enables AI-powered observability operations including:
Writing and validating PromQL queries
Generating Grafana dashboard JSON configurations
Creating alerting rules and recording rules
Analyzing metric cardinality and performance
Debugging scrape configurations
Interpreting metric patterns and anomalies
Prerequisites
Prometheus server access
Grafana instance with API access
Optional: Alertmanager for alerting
Optional: Thanos/Cortex for long-term storage
Capabilities
1. PromQL Query Writing Write and optimize PromQL queries:
# Request rate
rate(http_requests_total{job="api"}[5m])
# Error rate percentage
sum(rate(http_requests_total{status=~"5.."}[5m]))
/ sum(rate(http_requests_total[5m])) * 100
# P99 latency
histogram_quantile(0.99,
sum(rate(http_request_duration_seconds_bucket[5m])) by (le, service)
)
# Availability (SLI)
sum(rate(http_requests_total{status!~"5.."}[30d]))
/ sum(rate(http_requests_total[30d])) * 100
# Resource saturation
avg(rate(container_cpu_usage_seconds_total[5m]))
/ avg(kube_pod_container_resource_limits{resource="cpu"}) * 100
2. Recording Rules Create recording rules for performance optimization:
groups:
- name: api_metrics
interval: 30s
rules:
- record: job:http_requests:rate5m
expr: sum(rate(http_requests_total[5m])) by (job)
- record: job:http_errors:rate5m
expr: sum(rate(http_requests_total{status=~"5.."}[5m])) by (job)
- record: job:http_error_ratio:rate5m
expr: |
job:http_errors:rate5m / job:http_requests:rate5m
- name: slo_metrics
interval: 1m
rules:
- record: slo:availability:ratio_30d
expr: |
sum(rate(http_requests_total{status!~"5.."}[30d]))
/ sum(rate(http_requests_total[30d]))
3. Alerting Rules Create comprehensive alerting rules:
groups:
- name: service_alerts
rules:
- alert: HighErrorRate
expr: |
job:http_error_ratio:rate5m > 0.05
for: 5m
labels:
severity: critical
annotations:
summary: "High error rate detected"
description: "{{ $labels.job }} has error rate of {{ $value | humanizePercentage }} "
runbook_url: "https://wiki.example.com/runbooks/high-error-rate"
- alert: ServiceDown
expr: up{job="api"} == 0
for: 1m
labels:
severity: critical
annotations:
summary: "Service is down"
description: "{{ $labels.instance }} is unreachable"
- alert: HighLatencyP99
expr: |
histogram_quantile(0.99,
sum(rate(http_request_duration_seconds_bucket[5m])) by (le, service)
) > 2
for: 10m
labels:
severity: warning
annotations:
summary: "High P99 latency"
description: "P99 latency for {{ $labels.service }} is {{ $value }} s"
4. Grafana Dashboard Generation Generate Grafana dashboard JSON:
{
"dashboard" : {
"title" : "Service Overview" ,
"uid" : "service-overview" ,
"tags" : [ "production" , "api" ] ,
"timezone" : "browser" ,
"refresh" : "30s" ,
"time" : {
"from" : "now-6h" ,
"to" : "now"
} ,
"panels" : [
{
"title" : "Request Rate" ,
"type" : "timeseries" ,
"gridPos" : { "h" : 8 , "w" : 12 , "x" : 0 , "y" : 0 } ,
"targets" : [
{
"expr" : "sum(rate(http_requests_total{job=\"api\"}[5m])) by (status)" ,
"legendFormat" : "{{ status }}"
}
] ,
"fieldConfig" : {
"defaults" : {
"unit" : "reqps"
}
}
} ,
{
"title" : "Error Rate" ,
"type" : "stat" ,
"gridPos" : { "h" : 4 , "w" : 6 , "x" : 12 , "y" : 0 } ,
"targets" : [
{
"expr" : "sum(rate(http_requests_total{status=~\"5..\"}[5m])) / sum(rate(http_requests_total[5m])) * 100"
}
] ,
"fieldConfig" : {
"defaults" : {
"unit" : "percent" ,
"thresholds" : {
"mode" : "absolute" ,
"steps" : [
{ "color" : "green" , "value" : null } ,
{ "color" : "yellow" , "value" : 1 } ,
{ "color" : "red" , "value" : 5 }
]
}
}
}
}
]
}
}
5. Scrape Configuration Debug and generate scrape configurations:
scrape_configs:
- job_name: 'kubernetes-pods'
kubernetes_sd_configs:
- role: pod
relabel_configs:
- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape ]
action: keep
regex: true
- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path ]
action: replace
target_label: __metrics_path__
regex: (.+)
- source_labels: [__address__ , __meta_kubernetes_pod_annotation_prometheus_io_port ]
action: replace
regex: ([^:]+)(?::\d+)?;(\d+)
replacement: $1:$2
target_label: __address__
6. Metric Cardinality Analysis Analyze and optimize metric cardinality:
# Top metrics by cardinality
topk(10, count by (__name__)({__name__=~".+"}))
# Label value counts
count(count by (label_name) (metric_name))
# Memory usage by metric
prometheus_tsdb_head_series / prometheus_tsdb_head_chunks
MCP Server Integration This skill can leverage the following MCP servers:
Server Description Installation mcp-grafana (Grafana Labs) Official Grafana MCP server GitHub loki-mcp (Grafana) Loki log integration GitHub
Best Practices
PromQL
Use recording rules - Pre-compute expensive queries
Limit cardinality - Avoid unbounded labels
Use appropriate ranges - Match scrape interval
Prefer rate() over increase() - More accurate for graphs
Alerting
Multi-window alerting - Combine short and long windows
Clear runbook links - Include in annotations
Appropriate severity - Match business impact
Avoid alert fatigue - Alert on symptoms, not causes
Dashboards
USE method - Utilization, Saturation, Errors
RED method - Rate, Errors, Duration
Consistent layout - Follow dashboard patterns
Variable templates - Enable filtering
Process Integration This skill integrates with the following processes:
monitoring-setup.js - Initial Prometheus/Grafana setup
slo-sli-tracking.js - SLO/SLI dashboard creation
error-budget-management.js - Error budget dashboards
Output Format When executing operations, provide structured output:
{
"operation" : "create-dashboard" ,
"status" : "success" ,
"dashboard" : {
"uid" : "service-overview" ,
"url" : "https://grafana.example.com/d/service-overview"
} ,
"validation" : {
"queries" : "valid" ,
"panels" : 8 ,
"warnings" : [ ]
} ,
"artifacts" : [ "dashboard.json" ]
}
Error Handling
Common Issues Error Cause Resolution No dataMetric not scraped Check scrape config and targets Many-to-many matchingAmbiguous join Use on() or ignoring() Query timeoutComplex query Use recording rules Cardinality explosionUnbounded labels Add label constraints
Constraints
Validate PromQL syntax before applying
Test alerts in non-production first
Consider cardinality impact of new metrics
Use appropriate retention settings