| name | cluster-health |
| description | Full cluster health check for the home-ops Kubernetes cluster. Checks nodes, Flux, pods, storage, certs, database, networking, security, alerts, Gatus, Victoria Logs, events, and upgrade status. Use when user asks to check cluster health, run a health check, or diagnose cluster issues.
|
Run each check below in order. After all checks, print a summary table: area | status (✅/⚠️/❌) | one-line note. Only show ⚠️ or ❌ rows in the summary unless everything is green.
Run independent checks in parallel where possible.
1. Nodes
kubectl get nodes -o wide
Flag: any node not Ready, any pressure condition, version mismatch across nodes.
2. Flux
flux get all -A
Flag: any resource where READY != True. Ignore SUSPENDED=True resources.
3. Pods
kubectl get pods -A --no-headers | grep -vE '\s(Running|Completed|Succeeded)\s'
kubectl get daemonsets -A --no-headers | awk '$2 != $4 {print}'
Flag: CrashLoopBackOff, Error, OOMKilled, stuck Pending. Daemonsets where DESIRED != READY.
4. Rook-Ceph
kubectl get cephcluster -n rook-ceph -o jsonpath='{.items[0].status.ceph.health}'
kubectl get pvc -A --no-headers | grep -v Bound
Flag: anything other than HEALTH_OK. Any unbound PVCs.
5. OpenEBS
kubectl get pods -n openebs-system --no-headers | grep -v Running
Flag: any pod not Running.
6. VolSync Backups
kubectl get replicationsource -A
Flag: any source where last sync is >24h ago or status shows error.
7. Certificates
kubectl get certificate -A
Flag: READY != True, or expiry within 7 days.
8. CloudNative-PG
kubectl get cluster -n database
Flag: status not Cluster in healthy state, READY < INSTANCES.
9. Dragonfly
kubectl get pods -n database -l app.kubernetes.io/name=dragonfly --no-headers
Flag: not Running, any restarts.
10. Networking
kubectl exec -n kube-system ds/cilium -- cilium status --brief
kubectl exec -n kube-system ds/cilium -- cilium bgp peers
kubectl get pods -n network --no-headers | grep -v Running
Flag: Cilium not OK, BGP session not established, any network pod not Running.
11. HTTPRoutes / Gateways
kubectl get gateway -n network
kubectl get httproute -A -o json | jq '[.items[] | select(.status.parents[]?.conditions[]? | select(.type=="Accepted" and .status!="True"))] | length'
Flag: gateways not PROGRAMMED=True, any HTTPRoute not accepted (count > 0).
12. Security
kubectl get pods -n security --no-headers
Flag: Authelia or LLDAP not Running, any restarts > 0.
13. Observability Stack
kubectl get pods -n observability --no-headers | grep -v Running
Flag: any pod not Running.
14. Firing Alerts
curl -s https://alertmanager.diaoul.io/api/v2/alerts \
| jq '[.[] | select(.status.silencedBy == [] and (.labels.alertname | test("InfoInhibitor|Watchdog") | not))] | .[] | {alert: .labels.alertname, severity: .labels.severity, namespace: .labels.namespace}'
Flag: any critical alerts. Warning alerts note but don't fail.
15. Gatus — Per-Service Status
curl -s https://status.diaoul.io/api/v1/endpoints/statuses \
| jq '[.[] | select(.results[-1].success == false) | .name]'
Flag: any service in the list (non-empty = failing probes).
16. Victoria Logs — Error/Warning Scan
curl -s 'https://victoria-logs.diaoul.io/select/logsql/query' \
--data-urlencode 'query=(level:ERROR OR level:WARNING) | stats by (kubernetes.pod_namespace) count() as cnt' \
--data-urlencode 'start=1h'
Flag: namespaces with unusually high error counts. Use judgment — kube-system and openebs-system have background noise; flag counts that look anomalous (e.g. >1000 errors/hour from an app namespace).
17. Kubernetes Warning Events
kubectl get events -A --field-selector=type=Warning --sort-by='.lastTimestamp' | tail -20
Flag: OOMKilled, FailedScheduling, recurring BackOff.
18. Resource Pressure
kubectl top nodes
kubectl top pods -A --sort-by=memory | head -20
Flag: any node >85% memory, any node >90% CPU sustained.
19. System Upgrades
kubectl get talosupgrade,kubernetesupgrade -A
Flag: Phase not Completed — upgrade in progress or failed.
