// Identify flaky tests by comparing failures across multiple CI runs. Use when the user suspects flaky tests, sees intermittent failures, or wants to analyze test reliability.
Detect breaking changes by analyzing git diff against dev branch. Checks for state access changes in call/genesis handlers, state item ordering violations, and demo-rollup-schema.json changes.
Analyze compilation time and test durations from CI logs. Use when the user asks about slow builds, slow tests, or wants to optimize CI time.
Debug failed CI tests by fetching workflow logs from GitHub Actions and analyzing failures. Use when the user mentions CI failures, test failures, or wants to understand why their PR's CI is failing.
| name | detect-flaky-tests |
| description | Identify flaky tests by comparing failures across multiple CI runs. Use when the user suspects flaky tests, sees intermittent failures, or wants to analyze test reliability. |
Identify flaky tests by comparing failures across multiple CI runs.
gh CLI installed and authenticated (gh auth login)Get last N runs from the specified branch (default: dev):
gh run list --workflow=Rust --branch=dev --limit=10 --json databaseId,conclusion,headSha,createdAt
For each run, get failed test jobs only (nextest, nextest_all_features, coverage):
gh run view <run-id> --json jobs --jq '.jobs[] | select((.name == "nextest" or .name == "nextest_all_features" or .name == "coverage") and .conclusion == "failure") | {id: .databaseId, name: .name}'
Download logs to ci-logs/flaky-analysis/run-<run-id>/:
gh run view --job <job-id> --log | perl -pe 's/\e\[[0-9;]*m//g' > ci-logs/flaky-analysis/run-<run-id>/<job_name>.log
Parse each log for failed test names. Look for patterns:
FAILED followed by test pathtest result: FAILEDBuild a matrix: test name vs run ID (pass/fail).
Identify:
Provide summary: