sam-create-test-coverage

name	sam-create-test-coverage
description	Design, implement, and validate risk-based regression coverage across unit, component, integration, API/contract, and browser E2E layers, selecting the smallest reliable proof for each changed behavior. Use when asked to add tests, prove a bug fix, increase confidence or coverage, map acceptance criteria, or close backend or frontend test gaps.

Sam Create Test Coverage

Create the smallest reliable set of tests that proves the changed contract. Remain stack-, host-, provider-, tool-, and model-agnostic.

Non-Negotiable Contract

• Honor the exact repository, path, branch, commit, range, and criteria supplied.
• Preserve existing work. Do not reset, checkout, stash, clean, rebase, or rewrite history.
• Freeze base SHA, head SHA, bundle fingerprint, intent, no-go scope, environment identity, and cleanup ledger before editing.
• Inspect changed scripts, hooks, test runners, package commands, containers, and CI definitions before executing them.
• Fail closed when a real-data E2E target is unknown or is not a verified local, test, or development environment.
• Keep artifacts local by default. Publish only when explicitly requested and the exact remote target is resolved.
• Never expose secrets, credentials, private data, or sensitive paths in bundles, commands, artifacts, reports, or returned evidence.
• Reject .only, .skip, retries, broad timeouts, snapshot refreshes, assertion weakening, and mocks that remove the contract under test.
• Limit production changes to the in-scope correction or smallest test seam required by an accepted scenario.
• Record and clean every process, container, port, record, override, and artifact.

Resource Routing

• Read references/layer-selection.md when mapping scenarios to test layers.
• Read references/scenario-and-risk-policy.md while building the coverage ledger.
• Read references/environment-and-data-safety.md before starting services or using persistent data.
• Read references/regression-proof.md before claiming a new test protects a regression.
• Read references/output-contract.md before drafting and validating the report.

1. Resolve and Freeze the Change

Set the skill directory to the directory containing this file. Build a local bundle without fetching or modifying refs:

SAM_COVERAGE_DIR="<absolute directory containing this SKILL.md>"
WORK_TMP="$(mktemp -d)"
python3 "$SAM_COVERAGE_DIR/scripts/build_test_impact.py" \
  --repo "$PWD" --environment-kind unknown \
  --environment-id "unverified" > "$WORK_TMP/baseline-bundle.json"

Pass --base, --head, and repeated --path arguments when specified. Rebuild after verifying a real-data environment.

Freeze:

• Target mode, base/head refs and SHAs, bundle fingerprint, and changed files.
• Intended behavior, invariants, acceptance criteria, and explicit no-go scope.
• Owning boundaries, affected contracts, and command definitions.
• Environment kind, identity, endpoints, database/tenant, and proof.
• Cleanup ledger initialized for all resources the run may create.

Ask one concise question only when the target or safety-critical environment cannot be discovered. Never infer a safe database or tenant from a name alone.

2. Build the Behavior and Risk Ledger

Use stable IDs:

• AC-###: acceptance criterion.
• B-###: changed behavior.
• R-###: reachable risk.
• S-###: scenario.
• T-###: test.
• CMD-###: validation command and result.
• ART-###: local or explicitly published evidence.
• CL-###: cleanup resource.

Link criteria to behaviors, risks, scenarios, tests, commands, results, and artifacts. Cover applicable success, negative, boundary, permission, validation, state-transition, persistence, cache, concurrency, error, recovery, compatibility, and accessibility cases. Omit inapplicable classes with a reason.

Assign each scenario PLANNED, AUTOMATED, MANUAL_PROOF, REDUNDANT, or NOT_COVERED. Link REDUNDANT to an equivalent scenario. Give NOT_COVERED an exact blocker, residual risk, and next action.

3. Select the Smallest Reliable Layer

Apply references/layer-selection.md. Prefer:

• Unit for pure rules, mapping, parsing, validation, and state transitions.
• Component for isolated rendering, interaction, and accessibility state.
• Integration for module, storage, cache, queue, or service coordination.
• API/contract for method, route, auth, payload, status, headers, and response.
• E2E for critical real-browser journeys and frontend/backend wiring.

Do not default every case to E2E. Use multiple layers only when each proves a different boundary. Record why the selected layer is sufficient.

4. Plan Counterfactual Regression Proof

Give every new or changed test one proof status:

• RED_GREEN: safely observed failure before correction and pass after it.
• MUTATION: focused reversible mutation made the test fail.
• CONTRACT: authoritative boundary plus targeted assertion proves discrimination.
• NOT_PROVEN: proof was unsafe or unavailable, with reason and residual risk.

Never mutate the user's checkout solely to manufacture proof. Use an isolated temporary copy when safe. Do not claim full confidence with required high-risk proof marked NOT_PROVEN.

5. Implement Without Gaming Coverage

Use repository frameworks, fixtures, factories, helpers, selectors, and style. Test observable contracts instead of internal calls when practical. Avoid shared mutable state, sleeps, execution-order dependencies, hardcoded-literal assertions, and mocks that bypass ownership boundaries.

Before running the changed suite, rerun the exact builder command with the same target arguments and verified environment into $WORK_TMP/bundle.json. Preserve baseline-bundle.json; the final bundle must include the newly changed tests. Then audit that final patch:

python3 "$SAM_COVERAGE_DIR/scripts/audit_test_diff.py" \
  "$WORK_TMP/bundle.json" > "$WORK_TMP/test-diff-audit.json"

Treat audit findings as blocking until disproven from the exact diff. Inspect all changed command definitions before execution.

6. Prove the Real System When Required

For browser-facing behavior, start the real UI linked to the intended backend using safe repository-supported workflows. Confirm the browser uses the frozen environment. Use isolated deterministic data.

Treat mocked pages, request-only checks, and component shells as fallback proof only after recording serious direct, container, port/config, and linking attempts. State the exact blocker and residual risk. Never label fallback proof as real E2E.

7. Run and Classify Validation

Run new targeted tests, affected suites, relevant type/lint checks, broader suites proportional to risk, then required real-system proof. Record every command once as PASS, FAIL, or NOT_RUN, classified TARGET, BASELINE, ENVIRONMENT, or EXTERNAL.

Do not hide a product defect by changing expectations. Fix only in-scope product behavior at its owning boundary. Record unrelated failures separately.

8. Validate, Clean, and Return

Draft the structured report from references/output-contract.md, then run:

python3 "$SAM_COVERAGE_DIR/scripts/validate_coverage_report.py" \
  --baseline "$WORK_TMP/baseline-bundle.json" \
  --bundle "$WORK_TMP/bundle.json" "$WORK_TMP/report.json"

Stop only resources created by this run. Remove temporary data, overrides, logs, and artifacts unless explicitly retained. Update the CL-### ledger, revalidate, and remove WORK_TMP.

Return FULL only when all required scenarios and commands pass, required counterfactual proof exists, real-system proof is honest, the audit passes, and cleanup succeeds. Return PARTIAL for residual gaps. Return BLOCKED for unsafe environment, scope, authorization, or execution conditions.