linter-investigation

name

linter-investigation

description

Investigate arity's linter (and, secondarily, its parser) against a real-world R codebase. Clone a target repo, lint it, and triage the diagnostics for false positives, incorrect spans, and unsafe autofixes; parse failures on valid R are caught along the way. Every suspected bug is confirmed against real R via `Rscript` before it is called a bug. Use when asked to stress-test, investigate, or triage the linter (or parser) over an external repo or corpus.

Point arity's linter at a large body of real R code and hunt for linter quality bugs: false positives, incorrect spans, and unsafe fixes. This is the primary goal. Parse failures are a secondary catch—a parse error blocks linting a file, so they surface naturally, and a parse failure on valid R is a real parser bug worth reporting—but the center of gravity is the linter, not a full parser audit.

This is distinct from the smoke-test-triage skill. That one reacts to the weekly automated corpus scan's formatter regressions (losslessness, idempotence, format-error, panic) filed as GitHub issues. This skill is proactive and interactive: you choose a repo and go looking for linter/parser quality problems. Formatter losslessness and idempotence are out of scope here—leave them to smoke-test-triage.

The core principle (read first)

A finding is only a bug once the oracle says arity is wrong. The oracle is real R: Rscript. Real codebases contain intentionally invalid files (bug-repro fixtures like tests/Pkgs/PR*, deliberately-broken regression inputs), so a diagnostic on those is correct, not a bug. Before reporting anything, classify each suspicious finding into exactly one of:

• True positive — arity is right; move on.
• False positive — arity flags legitimate R. The highest-value find.
• Incorrect span — the finding is real but the caret underlines the wrong tokens.
• Unsafe fix — the autofix produces code that doesn't parse, changes semantics, or drops trivia (a comment). Test the fix, don't eyeball it.
• Parser bug — valid R that arity fails to parse or mis-parses (surfaces as syntax-error, or as a wrong CST shape). Confirm validity with Rscript.

Workflow

1. Target. Take the repo from the user's argument: a GitHub owner/name, a full clone URL, or a local path. If none is given, propose a good default (wch/r-source is large and idiomatic; a tidyverse package is smaller) and confirm before cloning.

2. Setup (parallel/background). Build the release binary and shallow-clone the target into the session scratchpad directory (not bare /tmp—honor the global scratchpad convention), running both at once:

   cargo build --release
   git clone --depth 1 https://github.com/<owner>/<name>.git "$SCRATCH/<name>"
   

   Use target/release/arity for speed; a debug build over a big corpus is slow.

3. Lint the tree, capture everything. Findings print to stderr; capture both streams to a file:

   target/release/arity lint "$SCRATCH/<name>" >lint.out 2>lint.err
   

   Gotcha: arity currently aborts the whole run on the first non-UTF-8 file (stream did not contain valid UTF-8). If that happens, find the offender (file reports the encoding), move/rename it aside, and re-run. (This abort is itself a known robustness bug—see arity's TODO.md.)

4. Summarize by rule. Count findings per rule to prioritize the high-volume and high-risk buckets:

   grep -oE '(warning|error): [a-z-]+' lint.err | sort | uniq -c | sort -rn
   

   The semantic rules (undefined-symbol, unused-binding, shadowed-builtin) are the most false-positive-prone; the syntax-error/error: bucket is where parser bugs hide.

5. Triage (the heart of the work). For each priority rule, pull real findings (grep -B1 -A6 'warning: <rule>' lint.err), open the cited source line, and reduce each suspect to a minimal reproducer piped to the tool:

Arity-specific notes

• Findings are on stderr, not stdout. lint exits non-zero when it reports anything.
• undefined-symbol needs a package root. Base R ships DESCRIPTION.in, not DESCRIPTION, so cross-file resolution may not activate and every sibling function reads as undefined—a corpus artifact, not a rule bug. Sanity-check by dropping a real DESCRIPTION next to an R/ dir and re-linting.
• NSE is the FP frontier. Formula ~ terms, quote/substitute/bquote/ expression bodies, and implicit method vars (.Generic/.Method/.Class) are common false-positive sources for undefined-symbol.
• Scope asymmetry drives unused-binding FPs (e.g. a for-body binding read from the enclosing frame). Compare the for vs while shapes when a binding looks wrongly-flagged.
• Autofix correctness is correctness, not layout (Tenet 1): a fix must keep the code parseable and lossless, but need not respect line width. Test a fix by applying it and re-parsing, never by reading it.