clickhouse-observability

name	clickhouse-observability
description	Monitor ClickHouse with Prometheus metrics, Grafana dashboards, system table queries, and alerting for query performance, merge health, and resource usage. Use when setting up ClickHouse monitoring, building Grafana dashboards, or configuring alerts for production ClickHouse deployments. Trigger: "clickhouse monitoring", "clickhouse metrics", "clickhouse Grafana", "clickhouse observability", "monitor clickhouse", "clickhouse Prometheus".
allowed-tools	Read, Write, Edit
version	1.0.0
license	MIT
author	Jeremy Longshore <[e-mail]>
tags	["saas","database","analytics","clickhouse","olap"]
compatible-with	claude-code

ClickHouse Observability

Overview

Set up comprehensive monitoring for ClickHouse using built-in system tables, Prometheus integration, Grafana dashboards, and alerting rules.

Prerequisites

• ClickHouse instance with system.* table access
• Prometheus (or compatible: Grafana Alloy, Victoria Metrics)
• Grafana for dashboards
• AlertManager or PagerDuty for alerts

Instructions

Step 1: Key Metrics from System Tables

-- Real-time server health snapshot
SELECT
    (SELECT count() FROM system.processes) AS running_queries,
    (SELECT value FROM system.metrics WHERE metric = 'MemoryTracking') AS memory_bytes,
    (SELECT value FROM system.metrics WHERE metric = 'Query') AS concurrent_queries,
    (SELECT count() FROM system.merges) AS active_merges,
    (SELECT value FROM system.asynchronous_metrics WHERE metric = 'Uptime') AS uptime_sec;

-- Query throughput (last hour, per minute)
SELECT
    toStartOfMinute(event_time) AS minute,
    count() AS queries,
    countIf(exception_code != 0) AS errors,
    round((query_duration_ms))  avg_ms,
    round(quantile()(query_duration_ms))  p95_ms,
    formatReadableSize((read_bytes))  total_read
 system.query_log
 type  (, )
   event_time  now()    
     ;



    toStartOfMinute(event_time)  ,
    ()  inserts,
    (written_rows)  rows_written,
    formatReadableSize((written_bytes))  bytes_written
 system.query_log
 type    query_kind  
   event_time  now()    
     ;


 database, , ()  parts, ()  ,
       formatReadableSize((bytes_on_disk))  size
 system.parts  active
  database, 
 parts  
  parts ;