monitoring

name	monitoring
description	Master Kubernetes observability, monitoring with Prometheus, logging, metrics, and distributed tracing. Learn to implement comprehensive monitoring strategies.
sasmp_version	1.3.0
eqhm_enabled	true
bonded_agent	06-monitoring-observability
bond_type	PRIMARY_BOND
capabilities	["Prometheus metrics","Grafana dashboards","Loki logging","OpenTelemetry tracing","Alerting","SLO management","Resource monitoring","Application performance"]
input_schema	{"type":"object","properties":{"action":{"type":"string","enum":["query","alert","dashboard","trace","analyze"]},"metric_type":{"type":"string","enum":["cpu","memory","network","custom","slo"]}}}
output_schema	{"type":"object","properties":{"metrics":{"type":"array"},"alerts":{"type":"array"},"recommendations":{"type":"array"}}}

Kubernetes Monitoring & Observability

Executive Summary

Production-grade Kubernetes observability covering the complete stack from infrastructure metrics to application tracing. This skill provides deep expertise in implementing SLO-based monitoring, multi-signal observability, and proactive alerting for enterprise environments.

Core Competencies

1. Metrics with Prometheus

Prometheus Stack Installation

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm install prometheus prometheus-community/kube-prometheus-stack \
  -n monitoring --create-namespace \
  --set grafana.adminPassword=secure-password \
  --set prometheus.prometheusSpec.retention=30d

Essential PromQL Queries

# Pod CPU usage
sum(rate(container_cpu_usage_seconds_total{namespace="production"}[5m])) by (pod)

# Memory utilization
sum(container_memory_working_set_bytes{namespace="production"}) by (pod)
  / sum(container_spec_memory_limit_bytes{namespace="production"}) by (pod) * 100

# Request rate
sum(rate(http_requests_total[5m])) by (service)

# Error rate (5xx)
sum(rate(http_requests_total{status=~"5.."}[5m]))
  / sum(rate(http_requests_total[5m])) * 100

# P99 latency
histogram_quantile(0.99, sum(rate(http_request_duration_seconds_bucket[5m])) by (le))

ServiceMonitor

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: api-server
  namespace: monitoring
spec:
  selector:
    matchLabels:
      app: api-server
  namespaceSelector:
    matchNames:
    - production
  endpoints:
  - port: metrics
    interval: 15s
    path: