logql-generator

name	logql-generator
description	Generate LogQL queries, log stream selectors, metric queries, and alerting rules for Grafana Loki. Use when this capability is needed.
metadata	{"author":"akin-ozer"}

LogQL Query Generator

Overview

Interactive workflow for generating production-ready LogQL queries. LogQL is Grafana Loki's query language with indexed label selection, line filtering, parsing, and metric aggregation.

Trigger Hints

• "Write a LogQL query for error rate by service."
• "Help me build a Loki alert query."
• "Convert this troubleshooting requirement into LogQL."
• "I need step-by-step LogQL query construction."

Use this skill for query generation, dashboard queries, alerting expressions, and troubleshooting with Loki logs.

Execution Flow (Deterministic)

Always run stages in order. Do not skip required stages.

Stage 1 (Required): Capture Intent

Use AskUserQuestion to collect goal and use case.

Template:

• "What is your primary goal: debugging, alerting, dashboard metric, or investigation?"
• "Do you need a log query (raw lines) or a metric query (numeric output)?"
• "What time window should this cover (example: last 15m, 1h, 24h)?"

Fallback if AskUserQuestion is unavailable:

• Ask the same questions in plain text and continue.

Stage 2 (Required): Capture Log Source Details

Collect:

1. Labels for stream selectors (job, namespace, app, service_name, cluster)
2. Log format (JSON, logfmt, plain text, mixed)
3. Known fields to filter/aggregate (status, level, duration, path, trace_id)

Ambiguity and partial-answer handling:

1. If a required field is missing, ask one focused follow-up question.
2. If still missing, proceed with explicit assumptions.
3. Prefix assumptions with Assumptions: in the output so the user can correct them quickly.

Stage 3 (Required): Discover Loki and Grafana Versions

Collect or infer:

• Loki version (example: 2.9.x, 3.0+, unknown)
• Grafana version (example: 10.x, 11.x, unknown)
• Deployment context (self-hosted Loki, Grafana Cloud, unknown)

Version compatibility policy:

1. If versions are known, use the newest compatible syntax only.
2. If versions are unknown, use compatibility-first syntax and avoid 3.x-only features by default.
3. For unknown versions, provide an optional "3.x optimized variant" separately.

Avoid by default when version is unknown:

• Pattern match operators |> and !>
• approx_topk
• Structured metadata specific behavior (detected_level, accelerated metadata filtering assumptions)

Stage 4 (Required): Plan Confirmation and Output Mode

Present a plain-English plan, then ask the user to choose output mode.

Plan template:

LogQL Query Plan
Goal: <goal>
Query type: <log or metric>
Streams: <selector>
Filters/parsing: <filters + parser>
Aggregation window: <function and [range]>
Compatibility mode: <version-aware or compatibility-first>

Mode selection template:

• "Do you want final query only (default) or incremental build (step-by-step)?"

If user does not choose, default to final query only.

Stage 5 (Conditional, Blocking): Reference Checkpoint for Complex Queries

Complex query triggers:

• Nested aggregations (topk(sum by(...)), multiple sum by, percentiles)
• Performance-sensitive queries (high volume streams, long ranges)
• Alerting expressions
• Template functions (line_format, label_format)
• Regex-heavy extraction, IP matching, pattern parsing
• Loki 3.x feature usage

Blocking checkpoint rule:

1. Read relevant files before generation using explicit file-open/read actions.
2. Minimum file set:
   • examples/common_queries.logql for syntax and query patterns
   • references/best_practices.md for performance and alerting guidance
3. Do not generate the final query until this checkpoint is complete.

Fallback when file-read tools are unavailable:

1. State that reference files could not be read in this environment.
2. Generate a conservative query (compatibility-first, simpler operators).
3. Mark result as Unverified against local references.

Stage 6 (Conditional): External Docs Lookup Policy (Context7 Before WebSearch)

Use external lookup only for version-specific behavior, unclear syntax, or advanced features not covered in local references.

Decision order:

1. Context7 first:
   • mcp__context7__resolve-library-id with libraryName="grafana loki"
   • mcp__context7__query-docs for the exact topic
2. WebSearch second (fallback only) when:
   • Context7 is unavailable
   • Context7 does not provide required version-specific detail
   • You need latest release/deprecation confirmation

WebSearch fallback constraints:

• Prefer official Grafana/Loki docs and release notes.
• Note which statement came from fallback search.

Stage 7 (Required): Generate Query

Stage 7A (Default): Final Query Only

Return one production-ready query plus short explanation.

Stage 7B (Optional): Incremental Build Mode

Use this when requested or when debugging complex pipelines.

Step-by-step template:

1. Stream selector
2. Line filter
3. Parser
4. Parsed-field filter
5. Aggregation/window

Stage 8 (Required): Deliver Usage and Checks

Always include:

1. Final query or incremental sequence
2. How to run it (Grafana Explore/panel or logcli)
3. Tunables (labels, thresholds, range)
4. Any assumptions and compatibility notes

AskUserQuestion Templates

Intake Template

• "What system/service should this query target?"
• "Which labels are reliable for stream selection?"
• "What defines a match (error text, status code, latency threshold, user path)?"
• "Should output be raw logs or a metric for alert/dashboard?"

Version Template

• "What Loki version are you running?"
• "What Grafana version are you using?"
• "If unknown, should I generate a compatibility-first query and add an optional 3.x variant?"

Ambiguity Follow-up Template

• "I am missing <field>. Should I assume <default> so I can continue?"

Core Patterns

Stream Selection and Filtering

{job="app"} |= "error" |= "timeout"
{job="app"} |~ "error|fatal|critical"
{job="app"} != "debug"

Parsing

{app="api"} | json | level="error" | status_code >= 500
{app="api"} | logfmt | caller="database.go"
{job="nginx"} | pattern "<ip> - - [<_>] \"<method> <path>\" <status> <size>"

Metric Aggregation

rate({job="app"} | json | level="error" [5m])
sum by (app) (count_over_time({namespace="prod"} | json [5m]))
sum(rate({app="api"} | json | level="error" [5m])) / sum(rate({app="api"}[5m])) * 100
quantile_over_time(0.95, {app="api"} | json | unwrap duration [5m])
topk(10, sum by (error_type) (count_over_time({job="app"} | json | level="error" [1h])))

Formatting and IP Matching

{job="app"} | json | line_format "{{.level}}: {{.message}}"
{job="app"} | json | label_format env=`{{.environment}}`
{job="nginx"} | logfmt | remote_addr = ip("192.168.4.0/24")

Query Construction Rules

1. Use specific stream selectors (indexed labels first).
2. Prefer filter order: line filter -> parse -> parsed-field filter.
3. Prefer parser cost order: pattern > logfmt > json > regexp.
4. For unknown Loki version, stay on compatibility-first syntax.
5. For complex/critical queries, complete Stage 5 checkpoint before final output.

Advanced Techniques

Multiple Parsers

{app="api"} | json | regexp "user_(?P<user_id>\\d+)"

Unwrap for Numeric Metrics

sum(sum_over_time({app="api"} | json | unwrap duration [5m]))

Pattern Match Operators (Loki 3.0+, 10x faster than regex)

{service_name=`app`} |> "<_> level=debug <_>"

Logical Operators

{app="api"} | json | (status_code >= 400 and status_code < 500) or level="error"

Offset Modifier

sum(rate({app="api"} | json | level="error" [5m])) - sum(rate({app="api"} | json | level="error" [5m] offset 1d))

Label Operations

{app="api"} | json | keep namespace, pod, level
{app="api"} | json | drop pod, instance

Note: LogQL has no dedup or distinct operators. Use metric aggregations like sum by (field) for programmatic deduplication.

Loki 3.x Key Features

Structured Metadata

High-cardinality data without indexing (trace_id, user_id, request_id):

# Filter AFTER stream selector, NOT in it
{app="api"} | trace_id="abc123" | json | level="error"

Query Acceleration (Bloom Filters)

Place structured metadata filters BEFORE parsers:

# ACCELERATED
{cluster="prod"} | detected_level="error" | logfmt | json
# NOT ACCELERATED
{cluster="prod"} | logfmt | json | detected_level="error"

approx_topk (Probabilistic)

approx_topk(10, sum by (endpoint) (rate({app="api"}[5m])))

vector() for Alerting

sum(count_over_time({app="api"} | json | level="error" [5m])) or vector(0)

Automatic Labels

• service_name: Auto-populated from container name
• detected_level: Auto-detected when discover_log_levels: true (stored as structured metadata)

Function Reference

Log Range Aggregations

Function	Description
rate(log-range)	Entries per second
count_over_time(log-range)	Count entries
bytes_rate(log-range)	Bytes per second
bytes_over_time(log-range)	Total bytes in time range
absent_over_time(log-range)	Returns 1 if no logs

Rule:

• Use bytes_over_time(<log-range>) for raw log-byte volume.
• Use | unwrap bytes(field) with unwrapped range aggregations for numeric byte fields extracted from log content.

Unwrapped Range Aggregations

Function	Description
sum_over_time, avg_over_time, max_over_time, min_over_time	Aggregate numeric values
quantile_over_time(φ, range)	φ-quantile (0 ≤ φ ≤ 1)
first_over_time, last_over_time	First/last value in interval
stddev_over_time	Population standard deviation of unwrapped values
stdvar_over_time	Population variance of unwrapped values
rate_counter	Per-second rate treating values as a monotonically increasing counter

Aggregation Operators

sum, avg, min, max, count, stddev, topk, bottomk, approx_topk, sort, sort_desc

With grouping: sum by (label1, label2) or sum without (label1)

Conversion Functions

Function	Description
duration_seconds(label)	Convert duration string
bytes(label)	Convert byte string (KB, MB)

label_replace()

label_replace(rate({job="api"} |= "err" [1m]), "foo", "$1", "service", "(.*):.*")

Parser Reference

logfmt

| logfmt [--strict] [--keep-empty]

• --strict: Error on malformed entries
• --keep-empty: Keep standalone keys

JSON

| json                                           # All fields
| json method="request.method", status="response.status"  # Specific fields
| json servers[0], headers="request.headers[\"User-Agent\"]"  # Nested/array

pattern

| pattern "<ip> - - [<timestamp>] \"<method> <path> <_>\" <status> <size>"

Named placeholders become extracted labels; <_> discards a field.

regexp