test-coverage-expansion

name	test-coverage-expansion
description	A systematic workflow to analyze, expand, and enforce robust test coverage across any Python project.

Test Coverage Expansion Workflow

This skill provides a structured methodology for expanding test coverage systematically, uncovering hidden edge cases, and ensuring that tests add real value rather than just ticking boxes.

1. Baseline Measurement & Target Identification

Before writing any tests, establish a clear baseline and identify the most critical areas lacking coverage.

1. Run Initial Coverage: Use your language's standard coverage tool (e.g., pytest --cov=src --cov-report=term-missing in Python).
2. Save the Baseline: Export the initial output to a file (e.g., pytest_cov_baseline.txt) so you can compare progress later.
3. Identify High-Value Targets:
   • Prioritize core business logic, risk management, and state reconciliation modules over UI or config files.
   • Look for modules with 0% coverage or high complexity but low coverage (< 50%).

2. Test Design & Mocking Strategy

When adding new tests to untested legacy or complex code, isolation is key.

1. Mock External Dependencies: Use testing libraries (like unittest.mock.MagicMock or pytest-mock in Python) to fake API calls, databases, and file I/O.
2. Setup Fixtures: Create reusable fixtures for common data structures (e.g., dummy DataFrames, default config dictionaries, stub classes).
3. Target the Missing Lines: Look at the exact line numbers reported as missing in the coverage report. Design specific inputs that force the code to traverse those conditional branches (if, elif, except).

3. Execution: The Probing & Testing Loop

Do not just write happy-path tests. Actively hunt for hidden bugs.

1. Write Happy-Path Tests First: Ensure the function behaves correctly under normal conditions.
2. Write Boundary & Edge-Case Tests:
   • Zero values, empty strings, empty lists/DataFrames.
   • None, NaN, Infinity.
   • Extreme negative or positive numbers.
3. Use Probe Scripts for Discovery: If you suspect a bug, write a quick throwaway probe script to call the function with extreme edge cases.
4. Report Bugs, Do Not Fix: If your test uncovers a real bug (e.g., a ZeroDivisionError or unhandled exception), DO NOT fix the application code. Your role is strictly QA. Mark the test as an expected failure (e.g., using @pytest.mark.xfail(strict=True) in Python), document the bug clearly in your report or PROGRESS.md, and assign the fix to the Dev team.

4. Verification & Iteration

Validate that your tests actually increased coverage without breaking the existing suite.

1. Re-run the Suite: Run the exact same coverage command from Step 1.
2. Compare Metrics:
   • Did the total number of passing tests increase?
   • Did the percentage coverage of the target module increase?
   • Did total project coverage increase?
3. Ensure Clean Run: Ensure all tests pass or are explicitly marked as expected failures (xfail). A coverage expansion task should not leave the CI pipeline red.

5. Bookkeeping & Handoff

Document the coverage expansion clearly for the next agent or human developer.

1. Save the Latest Snapshot: Pipe the final coverage output to pytest_cov_latest.txt (or equivalent).
2. Update Changelogs: Record the specific modules touched, the number of tests added, and the before/after coverage percentage in PROGRESS.md or the project's changelog.
3. Commit with Context: Use a clear commit message, e.g., test: expand coverage for risk module (0% -> 100%), added 25 tests.

6. CI Coverage Enforcement

Coverage expansion has no value if it can regress silently.

1. Set a Minimum Threshold: Add a --cov-fail-under=N flag to the CI coverage command. The pipeline fails if total coverage drops below N%.
   • Example: pytest --cov=src --cov-fail-under=80
2. Ratchet, Don't Plateau: After each expansion session, raise the threshold to the new achieved level. Never lower it.
3. Per-Module Thresholds (Advanced): Use .coveragerc to enforce stricter limits on critical modules (e.g., risk_manager.py must stay at 100%).

7. Mutation Testing (Quality Gate)

Coverage % measures which lines run, not whether tests catch bugs. Mutation testing fills this gap.

1. When to Use: After achieving high line coverage (>80%) on a critical module. Run mutation testing to verify tests are actually catching logic errors.
2. Tool: mutmut (Python)

   pip install mutmut
   mutmut run --paths-to-mutate src/trading/risk_manager.py
   mutmut results
   

3. Interpret Results: A surviving mutant = a real bug that your tests missed. Write a test that kills it.
4. Scope: Do not run on the entire codebase — it is slow. Target high-risk modules: risk management, signal generation, kill switch logic.

8. Property-Based Testing

For ML/trading code, manual edge cases miss the combinatorial space. Property-based tests generate hundreds of random inputs automatically.

1. Tool: hypothesis (Python)

   pip install hypothesis
   

2. Pattern:

   from hypothesis import given, strategies as st
   
   @given(st.floats(min_value=0.01, max_value=10.0))
   def test_kelly_sizing_never_exceeds_max_lot(risk_fraction):
       result = compute_kelly_lot(risk_fraction, balance=10000)
       assert result <= MAX_LOT_SIZE
   

3. High-Value Targets in This Project: Kelly sizing, confidence threshold clamping, drawdown calculations, feature normalisation bounds.
4. Avoid Unbounded Floats: Always constrain st.floats() with min_value/max_value and allow_nan=False unless testing NaN handling explicitly.

Specific Language Tips (Python / Pytest)

• Coverage Command: python -m pytest tests/ --cov=src --cov-report=term-missing -q
• Coverage with CI gate: python -m pytest tests/ --cov=src --cov-fail-under=80 -q
• Testing Exceptions: Use with pytest.raises(ExpectedException):
• Approximations for Floats: Use assert value == pytest.approx(expected_value)
• Strict Warnings: Run pytest -W error to catch deprecation and runtime warnings. Caution: this flag causes false positives on legacy codebases — apply it to new modules only, or filter specific warning categories via filterwarnings in pytest.ini.