smoke-test-triage

name	smoke-test-triage
description	Triage and fix arity corpus smoke-test regressions (losslessness, idempotence, format-error, panic) reported by the weekly scan of real R package repos and the GitHub issues it files.

Use this skill when asked to investigate failures reported by the corpus smoke-test scan (.github/workflows/smoke-test.yml) or the CI: corpus regression in <repo> (<category>) issues it files—especially losslessness and idempotence regressions.

How the scan works (read this before triaging)

The workflow shallow-clones every repo in TARGET_REPOS, then runs the Tier 0 corpus test (tests/corpus.rs) once, in-process, over all of them in CI mode (ARITY_CORPUS=<dir> ARITY_CORPUS_REPORT=<tsv>). The test writes one tab-separated record per failure—relative-key \t category \t message—and returns cleanly; the workflow enriches those with the repo + SHA and files issues. Consequences that shape triage:

• The unit of work is a category bucket per repo, not a file. One issue covers every file in that bucket; it lists at most ten samples, and the named sample is often not the most actionable one. Re-run the scan yourself over the whole clone and triage by root cause, not by the sample.
• Failure categories are losslessness, idempotence, format-error, and panic.
• A file arity cannot parse is skipped, not failed. Parse diagnostics are a known limitation, not a regression. So a format-error is never "just parse errors"—it is the formatter refusing input it was able to parse.
• Losslessness is checked on every file, including the skipped-unparseable ones.
• The scan is weekly and issues are deduped by a marker (arity-corpus-key:repo=…;type=…). An issue that stops reproducing gets a "No longer reproducing" comment rather than being closed automatically.

Check the ALLOWLIST in .github/workflows/smoke-test.yml first. It holds repo|path|category entries for failures already triaged as out of scope, and matching failures are suppressed from the report. An issue naming an already-listed file means the scan predates the entry—confirm and close rather than re-triaging.

Goals

1. Reproduce the exact failure from the report.
2. Minimize to a stable local fixture.
3. Add regression coverage in the right test surface.
4. Fix root cause (not symptom).
5. Validate targeted cases, then the whole target repo, then the full suite.

Triage workflow

1. Read the issue. Note the category, the target repo + commit SHA, the arity version/commit used by the scan, the sample files, and the per-file detail messages (the message column—for format-error and panic this is the actual error text and is usually the fastest route to the root cause). The corpus-scan-results artifact holds the full failures.tsv plus corpus.log (the run's totals, including how many files were skipped as unparseable); fetch it with gh run download <run-id> -n corpus-scan-results (the run id is in the issue's workflow-run link).

2. Reproduce. Clone the target repo at the exact SHA from the issue, then re-run the whole scan against it—the bucket, not the sample, is the thing you are triaging:

   git clone https://github.com/<repo>.git /tmp/target
   git -C /tmp/target checkout <repo_sha>
   ARITY_CORPUS=/tmp/target ARITY_CORPUS_REPORT=/tmp/before.tsv \
     cargo test --release --test corpus -- --ignored --nocapture
   cut -f2 /tmp/before.tsv | sort | uniq -c     # bucket sizes
   cut -f2,3 /tmp/before.tsv | sort | uniq -c   # distinct messages per category
   

   Then narrow to one file with the CLI. arity here means the current checkout (cargo run --release -- or target/release/arity), never an installed release:

   arity parse --verify --quiet <file>   # losslessness
   arity format --verify <file>          # idempotence
   arity format < <file> > /tmp/once.R   # pass 1 (stdin form!)
   arity format < /tmp/once.R > /tmp/twice.R
   diff /tmp/once.R /tmp/twice.R         # the idempotence divergence
   

   Two traps:

   • arity format <path> rewrites the file in place. Always use the stdin form (arity format < file) when capturing passes out of a clone.
   • arity parse --verify exits 1 on before it ever runs the losslessness check, so on an unparseable file it tells you nothing about losslessness. The corpus runner checks independently—trust that, or call directly.

Arity-specific guidance

• Formatting is deterministic and rule-based (Tenet 1): the input's line breaks never influence the result, unlike air. Never "fix" an idempotence failure by mirroring the input's layout, and push back on per-construct special cases.
• Comments are trivia the parser preserves losslessly; a losslessness diff that drops or relocates a comment is a CST attachment bug, and a comment that moves between format passes is an idempotence bug in comment relocation (src/formatter/trivia.rs and the rules/ comment paths).
• Roxygen (#') blocks have their own formatting path and their own oracle (roxygen-parity skill, tests/roxygen_format_stability.rs). An idempotence failure whose diff sits inside a #' block belongs there.
• Line endings: the corpus contains CRLF files; keep the original ending when minimizing (tests/line_endings.rs is the home for ending-specific cases).
• The corpus test catches panics per file, so a panic bucket does not stop the scan—but it does mean the run's other results came from a process that hit a bug. Fix panics first.

Report-back format

When done, report:

1. Whether the issue reproduced, and the exact command.
2. Minimal reproducer summary.
3. Fixture(s) added, and where they are registered.
4. Root cause and the code path changed.
5. Validation commands and outcomes, including the before/after failure counts over the whole target repo and explicit confirmation that no file regressed.
6. What you did not fix: the remaining buckets by root cause, which were allowlisted as out of scope (with the reason), which are open gaps still worth work, and any newly-checked files that surfaced pre-existing formatter gaps. Say plainly whether the issue can be closed or should stay open—a scan issue is rarely one bug, and a partial fix reported as a whole one is worse than no fix.