| name | check-test-coverage |
| description | Analyze oc-mirror CLI feature coverage across integration and e2e tests, identifying untested features and gaps |
| user-invocable | true |
Check Test Coverage
Analyze which oc-mirror CLI features are tested at the integration and e2e levels, and which have gaps.
Step-by-step process
1. Discover CLI features
Run these commands to enumerate all subcommands and flags:
oc-mirror --v2 --help
oc-mirror --v2 delete --help
oc-mirror --v2 list operators --help
oc-mirror --v2 list releases --help
oc-mirror --v2 version --help
If oc-mirror is not in PATH, build it first with make build and use ./bin/oc-mirror.
From the output, extract:
- Workflows: mirrorToMirror (m2m), mirrorToDisk (m2d), diskToMirror (d2m)
- Subcommands: delete, list operators, list releases, version
- Flags: every flag from each subcommand's help output (skip hidden/deprecated flags unless they appear in tests)
2. Parse integration tests
Read all *_test.go files in tests/integration/. For each file, extract:
- Ginkgo
Describe, Context, and It block labels
- Flags and workflows referenced in the test body (e.g.,
--dry-run, --delete-id, --from)
- Helper function calls that exercise specific features
Build a map of feature -> test file + test label.
3. Parse e2e tests
Read test files in tests/e2e/test/e2e/ (primarily oc_mirror_v2.go). Extract:
- Test case descriptions and case IDs
- Workflows and flags exercised in each case
- Test data configs referenced from
tests/e2e/test/e2e/testdata/
4. Cross-reference and classify
For each CLI feature (subcommand, flag, workflow), classify coverage:
| Level | Meaning |
|---|
| Covered | Tested at integration and/or e2e level with meaningful assertions |
| Partially covered | Only happy path, or tested at only one level when both would be appropriate |
| Not covered | No test exercises this feature |
Consider a feature "partially covered" if:
- It's only tested in the happy path but has error handling worth verifying
- It's a flag that modifies behavior but is only tested implicitly (e.g., the flag is set but its effect isn't asserted)
5. Produce the report
Present results in this format:
Summary table
| Feature / Flag | Integration | E2E | Status |
|----------------------|-------------|------|------------------|
| m2m workflow | m2m_test.go | 73359| Covered |
| --dry-run | dry_run_... | - | Partially covered|
| --secure-policy | - | - | Not covered |
| ... | | | |
Priority gaps
List the top uncovered or partially covered features, ordered by importance:
- Features that affect data correctness (e.g., signature verification, archive integrity)
- Features that affect user-facing behavior (e.g., filtering, error codes)
- Features that are operational concerns (e.g., parallelism, profiling)
For each gap, recommend:
- Which test level is appropriate (integration vs. e2e)
- A one-line description of what the test should verify
6. Optional: compare with previous run
If a previous coverage report exists in the conversation history, highlight what changed: new tests added, features that moved from "not covered" to "covered", and any regressions.
