Use when the user wants to diagnose why a Synthetic Monitoring check is failing — triage probe failures, classify failure scope, run per-probe breakdown, and identify root cause. Trigger on phrases like "why is my check failing", "investigate synthetic check", "probe failures", "check is down". For check status overview use synth-check-status. For creating or managing checks use synth-manage-checks.
Use when the user wants to diagnose why a Synthetic Monitoring check is failing — triage probe failures, classify failure scope, run per-probe breakdown, and identify root cause. Trigger on phrases like "why is my check failing", "investigate synthetic check", "probe failures", "check is down". For check status overview use synth-check-status. For creating or managing checks use synth-manage-checks.
allowed-tools
["gcx","Bash"]
Synthetic Check Investigator
Investigate Synthetic Monitoring check failures by triaging probe data, classifying failure scope, and identifying root cause. Experienced operators need actionable diagnosis, not hand-holding.
Core Principles
Use gcx commands — do not call Grafana APIs directly
Trust the user's expertise — skip background explanations
Use -o json for agent processing, default format for user display
Show timeline graphs for time-series data — they communicate trends faster than text
Collect errors; report them at the end, not interleaved in workflow steps
Prerequisites
gcx configured with an active context and appropriate permissions.
Investigation Workflow
Step 1: Get Check Status (with early exit)
gcx synthetic-monitoring checks status <ID>
If the user provided a name instead of ID, list first:
Early exit — OK: Check success rate >= 50% across all probes.
Report: "Check is healthy. Success rate: %. <probe_count> probes up."
Stop unless the user asks for more.
<job>
Early exit — NODATA: No Prometheus metrics available.
Get check config to verify enabled: true (gcx synthetic-monitoring checks get <ID> -o json | jq .spec.enabled)
If disabled: report "Check is disabled — no metrics will appear until it is re-enabled."
If enabled: report "No metrics found. Check datasource config or whether the SM stack is healthy."
Stop after reporting.