kusto

name	kusto
description	Query Azure Data Explorer (Kusto) for deep telemetry. USE WHEN needing workflow run data, database/MySQL/Vitess metrics, or Actions-specific telemetry not in Datadog.
metadata	{"triggers":["kusto","kql","azure data explorer","adx","dataexplorer.azure.com","workflow runs","mysql","vitess","database statements","slow queries","lock contention","deadlock","semi-sync","query-silo"],"provides":["kql-query","telemetry-search"],"requires":["az-cli"],"auto_load":false}

Azure Kusto / KQL Investigation

Query Azure Data Explorer for deep telemetry analysis via REST API.

⚠️ YOU CAN EXECUTE KUSTO QUERIES

DO NOT say "requires manual access". EXECUTE using kusto-query.py.

---

⛔ User-Provided ADX URLs — Run First, Rewrite Later

When the user provides a dataexplorer.azure.com URL, decode and execute the embedded KQL BEFORE writing your own queries. The user's query is the ground truth for what they're seeing.

# Decode and execute the user's ADX URL in one step
python3 $HOME/.pi/agent/skills/kusto/tools/decode-adx-url.py "<ADX_URL>" --execute --json

# Or decode first to inspect the KQL
python3 $HOME/.pi/agent/skills/kusto/tools/decode-adx-url.py "<ADX_URL>"

Rules:

1. Always decode the URL — ADX URLs use gzip+base64 encoding for the ?query= param. Do NOT guess what the query is.
2. Execute the decoded KQL as-is before modifying it. The user's query may use ago() (relative time) which always returns current data — do NOT replace it with hardcoded datetime() values.
3. If 0 rows returned: check whether your rewritten query narrowed the time range beyond the table's retention. Use .show table <T> policy retention to check. 0 rows + exit code 0 = data issue, NOT auth/network failure.

---

Tool: kusto-query.py

All Kusto queries go through the REST API wrapper:

# Execute a KQL query
python3 $HOME/.pi/agent/skills/kusto/tools/kusto-query.py \
  --cluster-uri "<from clusters.json>" \
  --database "<from clusters.json>" \
  --query "<your KQL>"

# Output as JSON
python3 $HOME/.pi/agent/skills/kusto/tools/kusto-query.py \
  --cluster-uri "<from clusters.json>" \
  --database "<from clusters.json>" \
  --query "<your KQL>" --json

# Query from file
python3 $HOME/.pi/agent/skills/kusto/tools/kusto-query.py \
  --cluster-uri "<from clusters.json>" \
  --database "<from clusters.json>" \
  --file /tmp/query.kql --json

# List tables
python3 $HOME/.pi/agent/skills/kusto/tools/kusto-query.py \
  --cluster-uri "<from clusters.json>" \
  --database "<from clusters.json>" \
  --list-tables

# Get table schema
python3 $HOME/.pi/agent/skills/kusto/tools/kusto-query.py \
  --cluster-uri "<from clusters.json>" \
  --database "<from clusters.json>" \
  --table-schema <TABLE>

# List databases
python3 $HOME/.pi/agent/skills/kusto/tools/kusto-query.py \
  --cluster-uri "<from clusters.json>" \
  --list-databases

Auth chain (automatic — no setup needed if az login is done):

1. SPN env vars (AZURE_TENANT_ID, AZURE_CLIENT_ID, AZURE_CLIENT_SECRET)
2. az account get-access-token fallback (uses existing az login session)

Retries: Built-in 503/429 retry with backoff (up to 4 attempts).

---

Cluster Selection

⛔ ALWAYS read $HOME/.pi/agent/skills/kusto/clusters.json to get the endpoint and defaultDatabase for any cluster.

Do NOT construct cluster URIs from the name — some clusters (e.g., ghactions-read) are Fabric-based and their URIs cannot be derived from the name.

Use Case	Cluster Name
Dotcom workflows	ghactions-read
Four-nines (glb, istio)	octokus
Proxima (prod-cus-01)	actions-cus
MySQL/Vitess/Database	query-silo

Common Cluster Details

Use Case	Cluster	Default DB
MySQL/Vitess/Database	query-silo	rac
Dotcom workflows	ghactions-read (Fabric)	Actions
Four-nines (GLB, istio)	octokus	Prod
Dotcom monolith logs	dotcomro	Dotcom
GitHub analytics/BI	gh-analytics	canonical
Actions 3-nines	githubactions	actions
Proxima (prod-cus-01)	actions-cus	Actions

---

⛔ DATABASE ISSUES: USE KUSTO

For MySQL, Vitess, or database replication issues — USE query-silo cluster.

Do NOT skip Kusto for database issues. The rac database has:

Table	Use Case
stmt_history	Recently completed MySQL statements
stmt_live	Currently running statements
trans_history	Completed transactions
trans_live	Transactions holding/waiting for locks

Per-table traffic analysis: Group stmt_history by schema (= table/shard in Vitess) to identify tables with anomalous traffic vs baseline.

See references/kql-examples.md for detailed KQL query patterns (traffic anomaly detection, replication lag, per-table analysis).

Filter by: cluster_name (MySQL cluster), host_name (specific host)

Dotcom Monolith (dotcomro cluster)

Table	Use Case
dotcom	Monolith logs — request tracing, HTTP status analysis
exceptions	Unredacted exceptions — join to dotcom on ['gh.request_id']

// 500 errors by route in last hour
dotcom
| where timestamp > ago(1h) and ['http.status_code'] == "500"
| summarize count() by ['http.target']
| order by count_ desc

GitHub Analytics (gh-analytics cluster)

Table	Use Case
accounts_all	Account data (use accounts_current() for latest)
repositories_all	Repo metadata (use repositories_current())
copilot_metrics_cube	Copilot usage analytics
user_daily_activity_per_product	User engagement by product

---

Workflows

Workflow	Team	Purpose
/kusto:query	—	Execute KQL queries
/kusto:schema	—	Discover tables and schemas
/kusto:actions-workflow-run	actions	Investigate Actions workflow runs
/kusto:actions-global	actions	Cross-stamp queries for Actions

Team Workflows

Team-specific workflows use the naming convention <team>-<workflow>.md and include a team: field in YAML frontmatter.

Discovery: To find workflows for a team:

ls workflows/ | grep "^<team>-"
# or check frontmatter: grep -l "^team: <team>" workflows/*.md

Adding team workflows:

1. Create workflows/<team>-<name>.md
2. Add team: <team> to YAML frontmatter
3. Update this table

---

⛔ RETRY ON 503 ERRORS

kusto-query.py has built-in retry (4 attempts with 5s/15s/30s backoff). If all 4 fail, the script exits non-zero. Only then may you claim unavailable — include the error output as proof.

---

FORBIDDEN

• ❌ npx @azure/mcp kusto query ... (removed — use kusto-query.py)
• ❌ mcp_azure_mcp_kusto tool calls (removed — use kusto-query.py)
• ❌ az kusto query ... (doesn't exist)
• ❌ Saying "manual access required"
• ❌ Linking to cluster without query text
• ❌ Citing Kusto with an empty URL — always generate a deep link with build-kusto-link.py
• ❌ Claiming "unavailable" after a single 503 (kusto-query.py retries automatically)
• ❌ Fetching raw rows without aggregation — always use summarize, top N, or take N in KQL to keep results small

---

Reporting Kusto Results

When reporting Kusto findings (Slack or report), MUST include:

✅ CORRECT:
Kusto: `stmt_history` analysis (47 results)
   Cluster: query-silo.eastus | Database: rac
   [Open in ADX Web](<generated deep link>)
   ```kql
   stmt_history
   | where timestamp > ago(1h)
   | summarize count() by query_type

❌ WRONG: Kusto query: stmt_history analysis

(missing query - humans can't verify) (no deep link - humans can't click to re-run) ```

See: report/SKILL.md (if available locally) → "Output Guardrails" for full link hygiene rules.

⛔ ALWAYS generate deep links using build-kusto-link.py. See references/kql-examples.md for full usage patterns and citation examples.