| name | noyalib-coverage-campaign |
| description | Executable, decision-gated playbook for the noyalib coverage-hardening campaign — driving line/branch/function coverage from the live workspace gate (96 fn / 94 lines / 93 regions per .github/workflows/ci.yml and shared-coverage.yml defaults; doc/TESTING.md still says 95/93/92 and is stale drift) toward the 98 % target declared in PLAN.md. Load this when the ask is "raise coverage", "close the llvm-cov gaps", "what should I work on next", "make coverage-gap says X is under threshold", "ratchet the fail-under numbers", "we're at 96, get us to 98", or any variant of coverage triage on this repo. Coordinates the scripts/coverage-gap-report.sh punchlist, the tests/coverage_*.rs house-pattern, and the shared-coverage.yml gate. Owns the end-to-end workflow: baseline → target selection → gap analysis → test authoring → verification → promotion. Explicitly fences off wrong turns (chasing 100 %, weakening thresholds, assert-free coverage tests, testing invariant_violated defensive paths). |
noyalib-coverage-campaign
Purpose. Run the coverage-hardening campaign to specification, unsupervised,
on a Sonnet-class session. This skill is the decision procedure, not a
tutorial: at each phase there is a command, an expected observation, and a
gate that says whether to proceed, retry, or escalate.
Scope guard. This skill covers only the coverage campaign. For the wider
"how do we test noyalib at all" question (test layers, proptest, fuzz,
Miri, spec suite) use noyalib-validation-and-qa. For interpreting a
specific tool's output (llvm-cov flags, cargo-vet, deny) use
noyalib-diagnostics-and-tooling. Do not duplicate those skills here.
Progressive disclosure. This SKILL.md is the campaign spine. Extended
material — bucket-classification walkthroughs, verbatim house-pattern
snippets, "why not" rationale for each fenced wrong-path, expanded
solution-menu prefer-when notes, and the invariant_violated
file:line inventory — is in
reference.md. Load reference.md when the compact
guidance below is insufficient; each section flags its
cross-reference.
0. Baseline numbers (VOLATILE — re-verify at campaign start)
Captured 2026-07-08 from .github/workflows/ci.yml:160-162,
.github/workflows/shared-coverage.yml reusable defaults,
doc/TESTING.md §"Coverage gate", and PLAN.md §1 "Coverage baseline":
| axis | live workspace gate (ci.yml) | shared-coverage.yml default | doc/TESTING.md (stale) | target |
|---|
| functions | 96 % | 96 % | 95 % | 98 % |
| lines | 94 % | 94 % | 93 % | 98 % |
| regions | 93 % | 93 % | 92 % | 98 % |
Two threshold layers exist and both must be understood:
- Live workspace gate (
.github/workflows/ci.yml:160-162, 96/94/93) —
the values a PR is actually measured against today. This matches the
shared-coverage.yml reusable-workflow defaults; the workspace caller
does not override them.
doc/TESTING.md documented axis (95/93/92) — documentation drift.
The doc lags the workflow files by one ratchet step. Trust the workflows;
fix the doc in the same PR that next moves the gate.
These numbers drift. Before touching anything, run Phase 0 and treat
the axis printed there as ground truth. If Phase 0 disagrees with this
table by more than 1 pp on any axis, update this section in the same PR
that closes the gap that caused the drift.
Phase 0 — Baseline: what do we actually cover today?
Command (canonical):
make coverage-gap
Under the hood this runs (verbatim from scripts/coverage-gap-report.sh):
NOYALIB_COVERAGE=1 cargo +nightly llvm-cov \
--workspace --all-features --no-fail-fast \
--ignore-filename-regex '' \
--summary-only
Expected observation. A TSV block, header file regions lines functions,
one row per file below the threshold, and on stderr a one-liner
<N> files below <T> %. The full raw summary is teed to
/tmp/noyalib-coverage.log.
Gate.
- Zero files below threshold → campaign done for this ratchet step;
advance to Phase 5 to ratchet the gate numbers.
- N > 0 → continue to Phase 1 with the punchlist.
If the tool errors: toolchain mismatch → cargo +nightly;
missing nightly → rustup toolchain install nightly --component llvm-tools-preview. Numbers wildly off → confirm --all-features.
CI cache poisoning symptom (cargo check under 2 s + drifting
numbers) → coverage jobs use CARGO_TARGET_DIR=target-coverage
(memory feedback_ci_cache_poisoning); locally cargo clean and
rerun. Extended error triage in
reference.md §6.
Phase 1 — Target selection: pick the ≤ 3 files that matter most
Rule of thumb. Rank Phase 0 rows by
(uncovered_lines × load_bearingness) where load-bearingness is:
| tier | modules | weight |
|---|
| S | parser/, scanner/, loader.rs, de/, ser/, error.rs | 3 |
| A | value.rs, borrowed.rs, cst/, document.rs | 2 |
| B | fmt.rs, path.rs, public re-exports | 1 |
| C | convenience wrappers, Display shims, example-only helpers | 0.5 |
Parser / loader / scanner / error paths outrank convenience code, always.
A file at 90 % in parser/events.rs beats a file at 85 % in a Display
shim (memory feedback_security_first_posture: correctness on
adversarial input IS a security control).
PR discipline gate. Pick ≤ 3 files per PR. One logical change
per PR (memories project_main_branch_protection +
feedback_no_premature_phasing). Bundling five files into one "coverage
sweep" makes the ratchet PR hard to revert per-file if one addition
regresses.
Entangled files — e.g. parser/events.rs gaps only close when
scanner.rs gaps close — may live in the same PR as a single logical
change. Say so in the description.
Phase 2 — Gap analysis: what kind of gap is each uncovered region?
Command. For a specific file:
NOYALIB_COVERAGE=1 cargo +nightly llvm-cov show \
--workspace --all-features \
--show-line-counts-or-regions \
--ignore-filename-regex '' \
-- crates/noyalib/src/parser/events.rs
Or the whole workspace with per-file annotated HTML:
NOYALIB_COVERAGE=1 cargo +nightly llvm-cov --html --workspace --all-features
open target/llvm-cov/html/index.html
Classification. Every uncovered region falls into one of four
buckets. This decision drives the rest of the campaign:
| bucket | signal | action |
|---|
| (a) | Reachable error path (an Err(Error::…) return, a bail on bad input) | Write a test that triggers exactly that input; assert the ErrorKind variant. |
| (b) | Feature-gated code (#[cfg(feature = "…")], #[cfg(target_arch = …)]) | Feature-matrix test leg — the workspace's --all-features still misses cfg-arch. |
| (c) | Defensive-unreachable — a _ => arm that calls invariant_violated | NOT a test target. Confirm the annotation, document exclusion in PR. |
| (d) | Dead code — genuinely unreachable, not defensive | Do not test. Open a change-control PR to remove it. Coverage rises by subtraction. |
How to spot (c) vs (d). (c) is guarded by
#[cfg_attr(noyalib_coverage, coverage(off))] and calls
crate::error::invariant_violated(...). If an arm is NOT annotated,
that's the bug — annotate it. (d) is code you can prove no input can
reach; if unsure, write the test first — failure to trigger is
empirical evidence for (d).
Extended walkthroughs (signal / action / common trap per bucket) and
the current invariant_violated file:line inventory are in
reference.md §1 and §3.
Phase 3 — Test authoring: the house pattern
Location. crates/noyalib/tests/coverage_<slug>.rs. The existing
family is ~20 files including coverage_scanner.rs,
coverage_error.rs, coverage_loader{,_full}.rs,
coverage_borrowed.rs, coverage_ser.rs, coverage_de.rs,
coverage_fmt.rs, coverage_value.rs,
coverage_spanned_anchors.rs, plus historical sweep files
(coverage_final*.rs, coverage_100.rs, coverage_remaining.rs).
Naming. Prefer topic-scoped names (coverage_scanner) over sweep
names (coverage_final_sweep4). Sweep names are historical; don't
add more.
Every file must:
- Start with the SPDX header and a module doc naming the source file
whose gaps it closes.
- Assert semantics — every test asserts a specific behaviour, not
"some output happened". Wrong-but-green is the declared enemy.
- On error-path tests,
matches! on the ErrorKind variant — not
err.to_string().
Canonical style references (mirror these three verbatim):
coverage_scanner.rs, coverage_error.rs, coverage_loader_full.rs.
Verbatim snippets (SPDX header, error-path shape, anti-pattern,
"any outcome is fine" exception) are in
reference.md §2.
Doctest option. If the file being covered has a public API without
a # Examples section, a doctest closes the coverage gap AND satisfies
the rustdoc-coverage goal in PLAN.md §1 (missing_docs = warn → deny,
≥ 98 % items). Double win; prefer this on the public surface.
Phase 4 — Verification gate
Command. Re-run Phase 0:
make coverage-gap
Expected. Targeted files no longer appear in the punchlist, or
appear with all three axes ≥ 98 %. Workspace totals rise by the
uncovered-region contribution of the files closed.
Gate.
- Targeted files ≥ 98 on all three axes → proceed to Phase 5.
- Improved but still below 98 → back to Phase 2 for the residual
regions. Common cause: an error path with three variants, test hit
two.
- Unchanged → the test isn't actually executing the intended path.
dbg! on entry, run cargo test --test coverage_<slug> -- <name> --nocapture under NOYALIB_COVERAGE=1, confirm the region enters.
If a file refuses to close the last percent: check for
#[cfg(target_arch = …)] (cross-target is out of scope per PLAN.md
§7.8), macro-expanded lines (llvm-cov show quirk on macro_rules!;
document as at-ceiling), or #[cfg]-conditional match arms needing
a different feature subset (--no-default-features --features alloc
for the no_std leg — only worth it for tier-S). Extended notes in
reference.md §6.
Principled exclusion. If a region is genuinely (c) or (d) but
cannot be excluded via #[cfg_attr(noyalib_coverage, coverage(off))],
document location + reason in the PR — change-control question, see
noyalib-change-control.
Phase 5 — Promotion: normal change control
Standard flow, per memory project_main_branch_protection:
- Branch from
main, feature branch.
- Local run of the full pre-push gate (see noyalib-build-and-env
or noyalib-ci-and-release for the exact command list).
- PR into
main — main rejects direct pushes.
- Wait for CI green on all shared workflows including the coverage
gate (memory
feedback_ci_must_be_green).
gh pr merge --squash --admin once green.- If the campaign has fully closed a threshold step (e.g. all files
≥ 98 % lines), a separate PR ratchets the numbers in both:
doc/TESTING.md §"Coverage gate"- the workspace's caller of
shared-coverage.yml (its
fail-under-* inputs)
- and, if this is the reference for satellites, the defaults in
.github/workflows/shared-coverage.yml
Never bundle a threshold ratchet with a test-authoring PR — two
separate logical changes, different risk profiles.
Success is measured, not eyeballed. The number of record is the
coverage-gate job (currently 96 fn / 94 lines / 93 regions per
ci.yml:160-162 + shared-coverage.yml defaults), not a local
terminal readout. If CI says "failed under 94 lines" and local says
"94.02", CI wins — check CARGO_TARGET_DIR isolation and re-run.
Known wrong paths (fenced — terse list)
Per-item "why not" rationale is in reference.md §4.
- Chasing 100 %. Unreachable by construction; ceiling is ≥ 98 %.
- Weakening thresholds to pass. Never lower a
fail-under-* in
shared-coverage.yml or doc/TESTING.md to green a red PR.
- Assert-free coverage tests. Wrong-but-green is the declared enemy.
- Duplicating existing coverage under a new name. Grep the
coverage_*.rs family first; extend if it exists.
- Testing
invariant_violated arms directly. Annotate them; do
not reach them via unsafe or hand-forged inputs.
- Touching
${1:-98} in scripts/coverage-gap-report.sh silently.
It's contract with the maintainer.
- Bundling with unrelated refactors. Two logical changes = two
PRs. Coverage PRs must be trivially revertible per-file.
The solution menu (ranked — brief)
Given a specific gap, prefer higher-ranked techniques first. Extended
when-to-prefer rationale in reference.md §5.
- Targeted unit / integration test — default. Use for (a)
reachable-error-path gaps and single-function invariants.
- Proptest generator — for combinatorial / shape-family gaps
(scalar shapes × chomping × indent; escape sequences; type tags).
- Doctest on the public API — closes coverage AND moves rustdoc
coverage toward PLAN.md §1 in one commit.
- Fixture-driven spec test — YAML file under
tests/fixtures/
plus expected event stream / value. For YAML-semantic gaps
(anchor/alias corners, merge-key ordering, spec-clause edges).
None replaces the others — a well-covered file uses all four.
Provenance
scripts/coverage-gap-report.sh — exact cargo +nightly llvm-cov
invocation, TSV output, default threshold 98 %..github/workflows/ci.yml:160-162 — live coverage-gate inputs:
96 fn / 94 lines / 93 regions..github/workflows/shared-coverage.yml — reusable workflow.
Defaults 96/94/93 (caller doesn't override).
CARGO_TARGET_DIR: target-coverage isolation is load-bearing
(memory feedback_ci_cache_poisoning).doc/TESTING.md §"Coverage gate" — stale (95/93/92); trust the
workflows. Also lists legitimate exclusions
(noyalib-wasm/src/lib.rs, MCP/LSP protocol.rs).PLAN.md §1 + §7.8 — 98 % target, ratchet plan, cross-target
matrix as future work.crates/noyalib/tests/coverage_*.rs — ~20-file house pattern.
Style refs: coverage_scanner.rs, coverage_error.rs,
coverage_loader_full.rs.crates/noyalib/src/error.rs::invariant_violated (line 1697) —
the #[cfg_attr(noyalib_coverage, coverage(off))] mechanism. Full
call-site inventory in reference.md §3.- Makefile
coverage-gap target — canonical entry point.
Sibling skills (out of scope; delegate):
- noyalib-validation-and-qa — proptest / fuzz / Miri / spec-suite.
- noyalib-diagnostics-and-tooling — llvm-cov flags, cargo-vet / deny.
- noyalib-change-control — ratchet-PR mechanics.
- noyalib-ci-and-release — pre-push gate; shared-workflow
caller-permission unioning (memory
feedback_shared_workflow_caller_perms).
