osdu

name	osdu
description	GitLab CI/CD test job reliability analysis for OSDU projects. Tracks test job (unit/integration/acceptance) pass/fail status across pipeline runs. Use for test job status, flaky test job detection, test reliability/quality metrics, cloud provider analytics. Wraps osdu-quality CLI.
version	2.0.0
brief_description	OSDU GitLab CI/CD test reliability analysis
triggers	{"keywords":["osdu","gitlab","quality","ci","cd","pipeline","test","job","reliability","flaky","acceptance","integration","unit","azure","aws","gcp","cloud","provider"],"verbs":["analyze","track","monitor","test","check"],"patterns":["test.*(?:reliability|status|job)","pipeline.*(?:analysis|status)","flaky.*test","ci.*cd","gitlab.*(?:pipeline|job)"]}
allowed-tools	Bash

Analyze GitLab CI/CD test job reliability for OSDU platform projects, tracking test job pass/fail status across pipeline runs to identify flaky tests and provide quality metrics. OSDU project test status queries ("how is {project} looking", "partition test quality") Flaky test detection ("are there flaky tests in {project}") Pipeline health monitoring ("recent pipeline failures") Cloud provider comparison ("azure vs aws test reliability") Stage-specific analysis ("unit test status", "integration test failures")

<skip-when>
  <condition>Individual test case tracking (we track job-level, not test-level)</condition>
  <condition>Non-test jobs (build, deploy, lint, security scans)</condition>
  <condition>Non-OSDU projects or non-GitLab CI systems</condition>
  <condition>Real-time monitoring (data is from completed pipelines only)</condition>
</skip-when>

Pipeline Run → Test Stage (unit/integration/acceptance) → Test Job → Test Suite (many tests)

<capabilities>
  <supported>Test job pass/fail status across multiple pipeline runs</supported>
  <supported>Flaky test job detection (jobs that intermittently fail)</supported>
  <supported>Stage-level metrics (unit/integration/acceptance)</supported>
  <supported>Cloud provider breakdown (azure, aws, gcp, ibm, cimpl)</supported>
  <unsupported>Individual test results (not tracked)</unsupported>
  <unsupported>Non-test jobs like build, deploy, lint</unsupported>
</capabilities>

<example>
  Pipeline #1: job "unit-tests-azure" → PASS (100/100 tests passed)
  Pipeline #2: job "unit-tests-azure" → FAIL (99/100 tests passed)
  Pipeline #3: job "unit-tests-azure" → PASS (100/100 tests passed)
  Result: This job is FLAKY (unreliable across runs)
</example>

<progressive-approach mandatory="true">
  <step number="1" name="start-light">
    <action>Use status.py for quick overview</action>
    <command>script_run osdu status.py --format json --pipelines 3 --project {name}</command>
    <rationale>Lightweight, fast, safe token usage</rationale>
  </step>

  <step number="2" name="deep-dive" condition="only-if-needed">
    <action>Use analyze.py with strict filters</action>
    <command>script_run osdu analyze.py --format json --pipelines 5 --project {name} --stage unit</command>
    <rationale>Heavy query, use only when status insufficient</rationale>
  </step>

  <step number="3" name="never-query-all">
    <action>ALWAYS specify --project to avoid 30-project scan</action>
    <rationale>Prevents token limit exceeded error</rationale>
  </step>
</progressive-approach>

<format-selection>
  <format type="json">
    <use-when>Extracting specific metrics or calculating statistics</use-when>
    <use-when>Building summaries or comparisons</use-when>
    <use-when>Parsing structured data programmatically</use-when>
    <use-when importance="critical">ALWAYS for status.py (lightweight, parseable)</use-when>
  </format>

  <format type="markdown">
    <use-when>Analyze.py queries (10x smaller than JSON, still readable)</use-when>
    <use-when>Creating reports for sharing</use-when>
    <use-when>Need human-readable tables without parsing</use-when>
    <use-when>Token budget is tight</use-when>
  </format>

  <format type="terminal" status="never-use">
    <avoid-because>Includes ANSI codes and colors, hard to parse</avoid-because>
    <avoid-because>Only for direct human terminal viewing</avoid-because>
  </format>
</format-selection>

<domain-services>
  <project name="wellbore-domain-services" description="Wellbore data"/>
  <project name="well-delivery" description="Well delivery workflows"/>
  <project name="seismic-store-service" description="Seismic data storage"/>
  <project name="dataset" description="Dataset management"/>
  <project name="register" description="Data registration"/>
  <project name="unit-service" description="Unit conversion"/>
</domain-services>

<reference-services>
  <project name="crs-catalog-service" description="Coordinate reference systems"/>
  <project name="crs-conversion-service" description="CRS conversion"/>
</reference-services>

<ddms-services>
  <project name="rafs-ddms-services" description="R&D data management"/>
  <project name="eds-dms" description="Engineering data management"/>
</ddms-services>

<workflow-processing>
  <project name="ingestion-workflow" description="Data ingestion pipelines"/>
  <project name="indexer-queue" description="Indexing queue management"/>
  <project name="notification" description="Event notifications"/>
  <project name="segy-to-mdio-conversion-dag" description="Seismic format conversion"/>
</workflow-processing>

<infrastructure>
  <project name="infra-azure-provisioning" description="Azure infra provisioning"/>
  <project name="os-core-common" description="Shared core libraries"/>
  <project name="os-core-lib-azure" description="Azure-specific libs"/>
</infrastructure>

<other-services>
  <project name="geospatial" description="Geospatial services"/>
  <project name="policy" description="Policy engine"/>
  <project name="secret" description="Secret management"/>
  <project name="open-etp-client" description="ETP protocol client"/>
  <project name="schema-upgrade" description="Schema migration tools"/>
</other-services>

<cloud-providers>
  <provider code="azure" name="Microsoft Azure"/>
  <provider code="aws" name="Amazon Web Services"/>
  <provider code="gcp" name="Google Cloud Platform"/>
  <provider code="ibm" name="IBM Cloud"/>
  <provider code="cimpl" name="CIMPL (Venus) provider"/>
</cloud-providers>

osdu-quality CLI installed: uv tool install git+https://community.opengroup.org/danielscholl/osdu-quality.git GitLab authentication (choose one): - GITLAB_TOKEN environment variable, OR - glab CLI authenticated (glab auth login) Access to OSDU GitLab projects Best approach: Start with status.py script_run osdu status.py --format json --pipelines 3 --project partition

<pattern name="flaky-test-detection">
  <step number="1">Check status</step>
  <command>script_run osdu status.py --format json --pipelines 5 --project partition</command>
  <step number="2">If issues found, deep dive with analyze.py</step>
  <command>script_run osdu analyze.py --format markdown --pipelines 5 --project partition --stage unit</command>
</pattern>

<pattern name="provider-comparison">
  <description>Compare Azure vs AWS for specific project/stage</description>
  <command>script_run osdu analyze.py --format markdown --pipelines 5 --project storage --stage integration --provider azure</command>
  <command>script_run osdu analyze.py --format markdown --pipelines 5 --project storage --stage integration --provider aws</command>
</pattern>

<pattern name="stage-specific-analysis">
  <description>Focus on unit tests only</description>
  <command>script_run osdu analyze.py --format markdown --pipelines 5 --project entitlements --stage unit</command>
</pattern>

script_run osdu analyze.py --format json --pipelines 10 Will exceed 200K token limit!

<dont-do description="Use high pipeline counts without project filter">
  <bad-example>script_run osdu analyze.py --format json --pipelines 20</bad-example>
  <reason>Takes 3+ minutes, huge output</reason>
</dont-do>

<dont-do description="Use terminal format in agent context">
  <bad-example>script_run osdu status.py --format terminal --project partition</bad-example>
  <reason>Includes ANSI codes, hard to parse</reason>
</dont-do>

<dont-do description="Jump straight to analyze.py">
  <bad-example>script_run osdu analyze.py --format json --pipelines 10 --project partition</bad-example>
  <reason>Heavy query when status.py would suffice</reason>
</dont-do>

Always start with status.py, only use analyze.py if needed Always specify --format json or --format markdown Always include --project {name} to avoid all-30-projects scan Start with --pipelines 3-5, increase only if necessary Use --stage or --provider to narrow scope Prefer --format markdown for analyze.py (10x token savings) Structured parsing, metrics extraction Reports, sharing, analyze.py queries Human viewing in terminal with colors Clear message with installation command Message about authentication requirements API error details List of valid options Suggestion to reduce --pipelines or add filters Always follow progressive query approach Never query without --project filter Start with minimal pipeline counts Use markdown format for analyze.py to save tokens Apply stage or provider filters when possible