| name | triage-rule-release |
| description | Triage CCX rule release pipeline failures — identifies failed stage/job, classifies root cause, and suggests a fix. |
| allowed-tools | Bash(curl:*), Bash(python3:*), Bash(gh pr:*), Bash(gh search:*), Bash(gh api:*), Bash(podman manifest inspect:*), AskUserQuestion |
Triage Rule Release
Triage failures in the CCX rule release pipeline (ccx/ccx-rules-releaser).
Prerequisites
- VPN: Red Hat VPN required —
gitlab.cee.redhat.com is internal. If the script can't connect, check VPN.
- Token:
GITLAB_TOKEN or GL_TOKEN env var — personal access token with read_api scope.
- macOS SSL: Python can't verify the Red Hat internal CA from the Keychain. One-time setup:
security find-certificate -a -p > ~/.ssl/certs.pem then export SSL_CERT_FILE=~/.ssl/certs.pem. On Linux this is usually not needed.
- Script:
triage_rule_release.py next to this file. Outputs JSON.
Invocation
/triage-rule-release # Latest pipeline
/triage-rule-release <pipeline-id> # Specific pipeline
/triage-rule-release <gitlab-url> # Parse from URL
Execution
- Run
python3 triage_rule_release.py (with optional pipeline ID or URL)
- Show pipeline status and all jobs
- For failed/stuck jobs, check downstream repos and images
- For Jenkins failures, fetch the console log and find the actual error
Pipeline
The pipeline triggers on a new ccx-rules-ocp tag (or manually with CCX_RULES_TAG variable). Notifications go to #ccx-dev-notifications.
Pipeline stages and jobs are defined in the release.yaml file in the releaser repo. Refer to that file for the current list of jobs and their configuration.
Images
After PRs merge in content-service and data-pipeline, Konflux push builds produce images. The rules-containers-private image is built by the rules-containers pipeline triggered earlier. Verify images exist with podman manifest inspect <image>.
| Image | Tag |
|---|
quay.io/redhat-services-prod/obsint-processing-tenant/content-service/content-service | First 7 chars of content-service merge commit SHA |
quay.io/redhat-services-prod/obsint-processing-tenant/data-pipeline/data-pipeline | First 7 chars of data-pipeline merge commit SHA |
quay.io/redhat-services-prod/obsint-processing-tenant/rules-containers/rules-containers-private | CCX_RULES_TAG date (e.g. 2026.06.16) |
Find merge commit SHA: gh pr view <number> --repo RedHatInsights/<repo> --json mergeCommit --jq '.mergeCommit.oid[:7]'
Check Konflux build: gh api repos/RedHatInsights/<repo>/commits/<sha>/check-runs --jq '.check_runs[] | select(.name | test("on-push")) | {name, status, conclusion, completed_at}'
Checking downstream PRs
GitHub (content-service, data-pipeline):
gh pr list --repo RedHatInsights/<repo> --state all --search "ccx-rules-ocp" --limit 3
gh pr view <number> --repo RedHatInsights/<repo> --json state,mergeable,statusCheckRollup
GitLab (ccx-rules-metrics, rules-containers, molodec, iqe-ccx-plugin):
curl -s -H "PRIVATE-TOKEN: ${GITLAB_TOKEN:-$GL_TOKEN}" \
"https://gitlab.cee.redhat.com/api/v4/projects/<url-encoded-path>/merge_requests?state=all&per_page=3&order_by=created_at&sort=desc"
URL-encoded paths: ccx%2Fccx-rules-metrics, ccx%2Frules-containers, ccx%2Fmolodec, insights-qe%2Fiqe-ccx-plugin
Jenkins logs (for wait-rules-in-stage-internal, wait-inter-prod, jenkins-exter-pipe-test-prod):
curl -s "https://jenkins-csb-ccx-dev-main.dno.corp.redhat.com/job/<job-name>/<build-num>/consoleText" | grep -B5 "FAILED\|AssertionError\|short test summary"