open-testing-coverage

name

open-testing-coverage

description

Test coverage analysis, gap detection, coverage reporting, and coverage optimization using the COVERED agent pipeline. Use this skill whenever the user mentions test coverage, coverage analysis, coverage gaps, traceability matrix, coverage heat map, coverage dashboard, coverage metrics, coverage trends, test intensity, risk-based coverage, requirement coverage, code coverage mapping, or wants to know if their testing is sufficient. Also trigger when users ask about uncovered requirements, orphaned tests, coverage regression, or coverage targets. MANDATORY TRIGGERS: coverage, traceability, gap analysis, RTM, heat map, coverage dashboard, coverage trend, test intensity evaluation.

Open Testing Coverage Analysis (COVERED)

Analyze, measure, visualize, and optimize test coverage using TMap coverage groups, ISO 25010 quality mappings, and risk-based intensity levels. The COVERED pipeline takes test inventories and requirements as input and produces coverage dashboards, gap reports, and optimization recommendations.

Agent Pipeline

Seven agents process coverage data sequentially. Each agent enriches the coverage model before passing to the next.

Agent 1: C_classify — Coverage Classifier

Purpose: Classify every test case into TMap coverage groups and coverage types.

Procedure:

1. Read the test case inventory (xlsx/CSV input from user)
2. Load coverage group definitions from knowledge base
3. For each test case, classify by:
   • Coverage Group: Process-oriented, Condition-oriented, Data-oriented, or Appearance-oriented
   • Coverage Type: Functional, boundary, decision, state, path, equivalence, condition, or experiential
   • Technique Used: Map to one of 106 technique patterns
4. Output a classified test inventory with coverage metadata columns added

TMap Coverage Groups:

Group	Focus	Primary Techniques	Typical Artifacts
Process-oriented	Business flows, end-to-end scenarios	Process cycle test, use case testing, scenario testing	Business process models, use cases
Condition-oriented	Logical decisions, rules, conditions	Decision table, cause-effect graphing, MC/DC, condition testing	Decision rules, business rules
Data-oriented	Input/output values, data combinations	Equivalence partitioning, BVA, classification tree, pairwise	Data specifications, field constraints
Appearance-oriented	UI, UX, look-and-feel, accessibility	Exploratory testing, usability testing, checklist-based testing	UI mockups, style guides, WCAG

KB Files:

• terminology.json — Coverage term definitions
• technique-patterns.json — 106 technique patterns with coverage criteria
• quality-to-technique.json — Quality characteristic to technique mappings

Agent 2: O_overlay — Requirements Overlay

Purpose: Overlay classified test coverage against the Requirements Traceability Matrix (RTM) to identify gaps.

Procedure:

1. Read requirements list (from user input or generated RTM)
2. Map each requirement to test cases that cover it
3. Identify:
   • Uncovered requirements: Requirements with zero linked test cases
   • Partially covered requirements: Requirements with some but insufficient test cases based on risk class
   • Orphaned test cases: Test cases not linked to any requirement
   • Over-tested areas: Requirements with excessive test cases relative to risk
4. Calculate coverage percentage per requirement, per feature, and overall

Output Format:

| Requirement ID | Description | Risk Class | Required Intensity | Test Cases | Coverage % | Status |
|---|---|---|---|---|---|---|
| REQ-001 | Login validation | A (High) | Thorough (●●●) | TC-01,TC-02,TC-15 | 85% | Gap |
| REQ-002 | Search function | C (Low) | Light (●) | TC-05 | 100% | Covered |

Agent 3: V_visualize — Coverage Visualizer

Purpose: Generate visual coverage reports — heat maps, sunburst diagrams, and trend charts.

Visualizations Generated:

1. Coverage Heat Map (HTML/React): Color-coded grid showing coverage density per requirement/feature area

   • Green (>90%): Well covered
   • Yellow (70-90%): Partially covered
   • Red (<70%): Critically under-covered

2. Coverage Sunburst Diagram (HTML/React): Hierarchical view showing coverage by:

   • Outer ring: Individual requirements
   • Middle ring: Features/modules
   • Inner ring: Coverage groups (Process/Condition/Data/Appearance)

3. Coverage Distribution Chart (xlsx): Bar chart showing test case count per coverage group and intensity level

4. Coverage Radar (HTML/React): Spider chart comparing actual vs. target coverage across quality characteristics (ISO 25010)

Agent 4: E_evaluate — Coverage Evaluator

Purpose: Evaluate whether coverage meets the targets defined in the Test Intensity Table and risk-based thresholds.

Evaluation Algorithm:

1. Load technique-to-intensity.json for intensity definitions
2. For each risk class (A/B/C), determine required intensity level:
   • Risk Class A (High): Thorough (●●●) — 95%+ coverage target
   • Risk Class B (Medium): Standard (●●) — 80%+ coverage target
   • Risk Class C (Low): Light (●) — 60%+ coverage target
3. Compare actual coverage per requirement against its risk-based target
4. Flag all requirements below threshold as "coverage gap"
5. Calculate overall coverage adequacy score (0-100)

Coverage Adequacy Formula:

Adequacy = (Σ min(actual_coverage[i], target_coverage[i]) / Σ target_coverage[i]) × 100

KB Files:

• technique-to-intensity.json — 20 techniques with light/standard/thorough definitions
• quality-to-technique.json — Technique recommendations per quality characteristic

Agent 5: R_recommend — Gap Recommender

Purpose: For each coverage gap, recommend specific techniques and test cases to close it.

Recommendation Logic:

1. For each uncovered/partially-covered requirement: a. Identify its quality characteristics (functional, performance, security, etc.) b. Look up applicable techniques from quality-to-technique.json c. Select techniques matching the required intensity level d. Estimate additional test cases needed per technique
2. Prioritize recommendations by:
   • Risk class (A > B > C)
   • Coverage gap size (largest gaps first)
   • Effort-to-close (prefer low-effort high-impact recommendations)

Output: Ranked list of recommendations with technique, expected test case count, effort estimate, and expected coverage improvement.

Agent 6: E_export_cov — Coverage Exporter

Purpose: Package all coverage analysis into deliverable formats.

Export Formats:

• xlsx Dashboard: Multi-sheet workbook with Summary, Classified Tests, RTM Overlay, Gap Analysis, Recommendations, Trends
• HTML Interactive Report: Self-contained HTML with charts and drill-down capability
• JSON Metrics: Machine-readable coverage data for CI/CD pipeline integration
• docx Executive Summary: 2-3 page coverage summary for stakeholders

Read the xlsx skill at /sessions/trusting-pensive-ptolemy/mnt/.claude/skills/xlsx/SKILL.md before generating spreadsheets. Read the docx skill at /sessions/trusting-pensive-ptolemy/mnt/.claude/skills/docx/SKILL.md before generating documents.

Agent 7: D_delta — Coverage Delta Analyzer

Purpose: Compare coverage between test runs, releases, or sprints to detect coverage regression.

Procedure:

1. Load current coverage analysis and previous coverage baseline
2. For each requirement, calculate coverage delta (current - previous)
3. Classify changes:
   • Improved: Coverage increased ≥5%
   • Stable: Coverage changed <5%
   • Regressed: Coverage decreased ≥5%
   • New: Requirement added since last baseline
   • Removed: Requirement removed since last baseline
4. Generate delta report with drill-down to specific test cases added/removed

Input Requirements

Users should provide one or more of:

• Test case inventory (xlsx/CSV): Columns include Test ID, Title, Steps, Expected Result, Linked Requirement(s), Priority
• Requirements list (xlsx/CSV): Columns include Requirement ID, Description, Priority, Risk Class
• Previous coverage report (xlsx/JSON): For delta analysis
• Quality risk analysis output: Risk classes per requirement (from open-testing-design skill)

Example Invocations

"Analyze coverage for our login module" → Run C_classify on provided test cases → O_overlay against requirements → E_evaluate against intensity targets → R_recommend gap-closing techniques → E_export as xlsx dashboard

"Compare coverage between Sprint 12 and Sprint 13" → Load both sprint coverage data → D_delta comparison → V_visualize trend chart → E_export delta report

"Are we ready to release? Check coverage against our risk targets" → E_evaluate full coverage against risk-class intensity targets → R_recommend if gaps found → O_outcome with go/no-go assessment → E_export executive summary

Integration

• open-testing-design: Receives risk classes and intensity targets from STRATEGY pipeline
• open-testing-create: Can trigger CREATES pipeline to generate test cases for gaps
• open-testing-reporting: Feeds coverage metrics into REPORTS pipeline
• CI/CD: JSON export integrates with Jenkins, GitHub Actions, Azure DevOps quality gates