Menu
ObsInt Processing team reference โ repos, services, deployment, clusters, shared coding standards, PR rules, and language conventions. Shared guidance for team repositories; repo-specific details live in each repo's AGENTS.md.
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
Generate a report of what you have worked on in a given quarter. The Quarterly Connection is something all Red Hat users must complete after a quarter has ended. There you talk what you have accomplished, the goals you had and the goals you want to set up for the next quarter. Most of this information is already in Jira, so we can use that to generate the report and help the user remember what they have done in those months. This should be used when the user asks for help writing the quarterly connection on Workday.
Collect GitHub PRs and GitLab MRs authored by a user during a given quarter. Produces a themed report of code contributions across team repositories. Use alongside the quarterly-connection-jiras skill to complement Jira-based reports with actual code work, or standalone when the user asks for a PR/MR summary.
Triage CCX rule release pipeline failures โ identifies failed stage/job, classifies root cause, and suggests a fix.
Triage Glitchtip-sourced Jira tickets in the CCXDEV project โ detect duplicates, investigate root causes via stack traces and source code, and post "[AI suggestion]" comments. Use this skill whenever the user mentions triaging Glitchtip errors, reviewing ccx-processing/obsint-processing Jira tickets, deduplicating error reports, or investigating production/staging errors tracked in Jira with the glitchtip label.
Generate the weekly Processing team Scrum of Scrums report from Jira epics. Compares progress over the last 7 days (report day is usually Wednesday): child-task movement, epic completion %, and epic status updates. Use when the user asks for a scrum of scrums, SoS report, weekly epic status, processing epic report, or Wednesday status report.
Triage CCX Jenkins CI failures โ classifies failure patterns (isolated blip, consecutive, flapping), analyzes logs, matches against known issues, and recommends action. Covers all active CCX jobs.
| name | team-info |
| description | ObsInt Processing team reference โ repos, services, deployment, clusters, shared coding standards, PR rules, and language conventions. Shared guidance for team repositories; repo-specific details live in each repo's AGENTS.md. |
Insights Operator archives arrive on the platform.upload.announce Kafka topic. archive-sync (an insights-ccx-messaging consumer) downloads the archive to S3 (rh-openshift-obs-stage / rh-openshift-obs-prod-secure) and publishes to ccx.archive.synced. rules-processing (a data-pipeline consumer) picks up from ccx.archive.synced, downloads the archive from S3, runs OCP rules, and publishes results to ccx.insights.rules.results. rules-uploader writes each result back to S3. parquet-factory also reads from ccx.insights.rules.results and aggregates rule-hit and cluster-info data into Parquet files on S3 (hourly).
ccx-data-pipeline (data-pipeline repo) consumes new archive events from Kafka, reads the archive from S3, runs OCP rules, and publishes results to ccx.ocp.results. insights-results-db-writer (same codebase as aggregator, different deployment params) consumes from ccx.ocp.results and writes to PostgreSQL. insights-results-aggregator reads from that DB and serves results via REST API. insights-results-smart-proxy is the API gateway โ external clients (UI, integrations, notifications) go through it to reach the aggregator and content-service (rule metadata: titles, remediations, tags). ocp-advisor-frontend is the React UI on console.redhat.com.
ccx-notification-writer also consumes from ccx.ocp.results and writes new/changed hits into a notification DB. ccx-notification-service reads that DB and sends email/webhook notifications.
ccx-upgrades-data-eng queries RHOBS for cluster metrics and calls ccx-upgrades-inference (ML model) to predict upgrade failure risk.
AWS MSK cluster managed by AppSRE, shared across tenants. Topics are configured via PRs to platform-mq. Used topics: platform.upload.announce, ccx.archive.synced, ccx.insights.rules.results, ccx.ocp.results.
| Area | Technologies |
|---|---|
| Backend services | Go (aggregator, smart-proxy, notification-*, content-service, cleaner, exporter, gathering-conditions-service), Python (data-pipeline, parquet-factory, insights-ccx-messaging, upgrades-data-eng, upgrades-inference, template-renderer) |
| Frontend | React, PatternFly (ocp-advisor-frontend) |
| Messaging | Kafka (all inter-service communication) |
| Storage | PostgreSQL (aggregator DB, notification DB), S3 (Parquet files) |
| Deployment | Clowder/ClowdApp on OpenShift, managed via app-interface SaaS files |
| CI/CD | Konflux (Tekton), GitHub Actions, Jenkins |
| Testing | BDD with Behave (insights-behavioral-spec), IQE (iqe-ccx-plugin), pytest (Python services), Go standard testing |
All new code requires tests and coverage. See each repo's AGENTS.md (when present) or README for repo-specific setup and how to run locally / in ephemeral.
app-interface controls service definitions, deployments, namespaces, cluster config, Grafana dashboards, and SaaS promotions. Our services live under data/services/insights/ccx-data-pipeline/.
App-interface GraphQL API: https://app-interface.apps.rosa.appsrep09ue1.03r5.p3.openshiftapps.com/graphql
Visual app-interface: https://visual-app-interface.devshift.net
Available app-interface skills.
| Repo | What it does | Related Repos | AGENTS.md |
|---|---|---|---|
| data-pipeline | Consumes archives from Kafka, runs OCP rules, publishes results. | insights-ccx-messaging, parquet-factory | AGENTS.md |
| insights-results-aggregator | Stores OCP rule hits in PostgreSQL, serves them via REST API to smart-proxy. Also deployed as insights-results-db-writer (same code, different params โ writer consumes Kafka, aggregator serves RESTAPI). | insights-results-smart-proxy, aggregator-cleaner, aggregator-exporter | AGENTS.md |
| insights-results-smart-proxy | API gateway between external clients (UI, integrations) and backend pipeline services. | insights-results-aggregator, content-service, ocp-advisor-frontend | AGENTS.md |
| ccx-notification-service | Reads new / changed rule hits and sends notifications to customers. | ccx-notification-writer, insights-results-aggregator | โ |
| ccx-notification-writer | Consumes rule-hit from Kafka and writes them into the notification DB. | ccx-notification-service | โ |
| ccx-upgrades-data-eng | Queries RHOBS for cluster metrics and calls the inference model to predict upgrade risk. | ccx-upgrades-inference, obsint-mocks | โ |
| ccx-upgrades-inference | REST API serving the ML model that predicts upgrade failure likelihood from cluster metrics. | ccx-upgrades-data-eng | โ |
| content-service | Serves rule metadata (titles, descriptions, remediations, tags, groups) to smart-proxy. | insights-results-smart-proxy, ocp-advisor-frontend | โ |
| insights-content-template-renderer | Renders report messages from DoT.js templates using content data and report details. | content-service | โ |
| insights-results-aggregator-cleaner | Periodic job that deletes old/obsolete records from the aggregator database. | insights-results-aggregator | โ |
| insights-results-aggregator-exporter | Exports aggregator PostgreSQL data to S3 as CSV for offline analysis. | insights-results-aggregator | โ |
| insights-operator-gathering-conditions | Remote configuration data (conditional gathering rules, rapid recommendations) for Insights Operator. | insights-operator-gathering-conditions-service | โ |
| insights-operator-gathering-conditions-service | Serves gathering conditions and rapid recommendations to Insights Operator on clusters. | insights-operator-gathering-conditions | โ |
| ocp-advisor-frontend | React UI for OCP Advisor on console.redhat.com โ displays rule hits per cluster. | insights-results-smart-proxy, content-service | โ |
| Repo | What it does | Related Repos | AGENTS.md |
|---|---|---|---|
| insights-ccx-messaging | Python framework (on top of insights-core-messaging) that all CCX Kafka consumers inherit from. | data-pipeline, parquet-factory | AGENTS.md |
| parquet-factory | Aggregates rule-hit events from Kafka into Parquet files and uploads them to S3. | data-pipeline, insights-ccx-messaging | โ |
| Repo | What it does | Related Repos | AGENTS.md |
|---|---|---|---|
| insights-on-prem | Deployment files and code for running Insights on Prem. | all EDP ones | AGENTS.md |
| Repo | What it does | Related Repos | AGENTS.md |
|---|---|---|---|
| processing-tools | Scripts, CI workflows, skills, and automations shared across all team repos. | all repos | โ |
| ccx-docs | Internal documentation for the team. | all repos | AGENTS.md |
| insights-operator-utils | Shared Go library used by multiple insights-operator-* and pipeline services. | insights-results-aggregator, smart-proxy, gathering-conditions-service | โ |
| insights-behavioral-spec | BDD test suite (Gherkin + Python) covering the full pipeline, OCM, notifications, and ServiceLog. | all pipeline services | AGENTS.md | | iqe-ccx-plugin | IQE test plugin for CCX integration tests in ephemeral environments. | insights-behavioral-spec | AGENTS.md | | obsint-mocks | Mock AMS and RHOBS endpoints used for local dev and BDD testing. | ccx-upgrades-data-eng | โ | | insights-results-aggregator-mock | Mock aggregator/smart-proxy API for local frontend and BDD development. | insights-results-aggregator, ocp-advisor-frontend | โ | | insights-results-mcp | MCP server exposing Insights results to LLMs via the Model Context Protocol. | - | โ | | insights-results-aggregator-utils | Utility scripts (Python/Go) for testing and interacting with pipeline REST APIs. | insights-results-aggregator, insights-results-smart-proxy | โ |
All services are deployed via app-interface SaaS files:
ref: master (or main). Merging to the default branch auto-deploys to stage.ref: <pinned-sha> with promotion: auto: false. Promoting requires an app-interface MR that updates the sha (use /promote-hypershift in app-interface).Each SaaS file publishes/subscribes to promotion channels (e.g. commit-lag-exporter-*).
Under data/services/insights/ccx-data-pipeline/:
| What | Path |
|---|---|
| External SaaS files | external-data-pipeline/<service>.yml |
| Internal SaaS files | internal-data-pipeline/<service>.yml |
| Stage namespace | namespaces/stage-ccx-data-pipeline-stage.yml |
| Prod namespace | namespaces/ccx-data-pipeline-prod.yml |
| Pipelines namespace | namespaces/ccx-data-pipeline-pipelines.appsrep09ue1.yaml |
| Grafana dashboards | provider: directory resourceTemplates in each SaaS file, path /dashboards |
| SLO document | slo-documents/ccx-data-pipeline.yml |
| Post-deploy jobs | jobs/post-deploy-jobs.yml |
| Environment | Cluster | Namespace |
|---|---|---|
| Stage | crcs02ue1 | ccx-data-pipeline-stage |
| Production | crcp01ue1 | ccx-data-pipeline-prod |
| Pipelines (Konflux) | appsrep09ue1 | ccx-data-pipeline-pipelines |
It may be relevant to know who is currently on watch duty. The rotation: data/teams/insights/schedules/ccx-processing-ic.yml in app-interface โ browse
The processing-tools repo hosts shared skills that can be installed into any agent:
| Skill | What it does |
|---|---|
konflux-dep-bumps | Triage and fix failing Konflux/MintMaker dependency bump PRs. |
resolve-cve | Resolve CVE vulnerability issues from Jira โ assess, bump, or mark not-affected. |
update-refs | Update refs of deployments to the latest commit SHA |
To install a skill from processing-tools (Claude-specific):
# Via npx (works with any agent โ replace claude-code with agent name)
npx skills add RedHatInsights/processing-tools --skill <skill-name> -g -a claude-code -y
# Via the plugin system (Claude Code only)
claude plugin marketplace add https://github.com/RedHatInsights/processing-tools
claude plugin install <skill-name>
# Or via curl into your skills directory
curl -sL https://raw.githubusercontent.com/RedHatInsights/processing-tools/master/skills/<skill-name>/SKILL.md \
-o ~/.claude/skills/<skill-name>/SKILL.md --create-dirs
Conventions for all team repositories. Individual repos' AGENTS.md files should cover only what's specific to that repo.
When working on tasks in any team repository, create a TODO list to track progress and ensure all steps are completed systematically.
Branch protection and merge controls (CODEOWNERS, status checks, bot bypass) are documented in ProdSec and enforced via github-rulesets. Team policy is stricter than the org floor:
CODEOWNERS file)[CCXDEV-12345] Summary[WIP]Human or agentic reviewers should understand the PR from its description, then verify:
Go repos have a make before_commit (or before-commit) target that runs the full suite of checks โ run it before pushing. Python repos use pre-commit hooks instead (pre-commit run --all-files). Check the repo's Makefile for the exact target.
All shell scripts must pass shellcheck. Go repos expose make shellcheck.
/health, /ready) are standard for all deployed services.pre-commit-config.yaml covering both Go and Python (golangci-lint, abcgo, ruff, shellcheck, hadolint). It also exports custom hooks (abcgo, go-version-consistency) that repos can use via repo: https://github.com/RedHatInsights/processing-tools.github/sync.ymlgofmtgolangci-lint (specific linter set configured per repo)rs/zerolog with structured context and request IDsgo test; check each repo's Makefile for integration test targetsrufflogging.getLogger(__name__)pytest; check each repo's Makefile for test targets and configurationNormative rules for PRs; strategy and test-type detail live in Quality.
AGENTS.md or Makefile; frameworks are listed under Tech Stack above