testing-linters

name	testing-linters
description	Guidelines and conventions for writing unit tests for linters in the r-lib/lintr package using testthat. Use this skill whenever creating or modifying linter test files under tests/testthat/.

Testing Linters in r-lib/lintr

When writing or updating unit tests in tests/testthat/test-*.R, adhere to the following testing conventions and production-readiness standards:

1. Constructing Package Fixtures (DESCRIPTION & NAMESPACE)

• Use write.dcf() for DESCRIPTION files: When creating temporary package structures (e.g., using withr::local_tempdir()), always construct DESCRIPTION files using base R's write.dcf() rather than manual string concatenation:

  write.dcf(
    list(Package = "testpkg", Version = "1.0.0"),
    file.path(pkg_dir, "DESCRIPTION")
  )
  

• Pass character vectors to writeLines(): Never pass newline-delimited single strings ("line1\nline2\n") to writeLines(), as writeLines() appends its own trailing newline and expects a character vector where each element is a line:

  writeLines(
    c("importFrom(stats, median)", "importFrom(utils, head)"),
    file.path(pkg_dir, "NAMESPACE")
  )
  

2. Fuzz Testing & Syntax Permutations (# nofuzz)

• Mark multi-file package fixtures with # nofuzz: lintr includes an automated test consistency and fuzz-testing harness that mutates and runs expect_lint() expressions across various permutations. Tests that create temporary directories (withr::local_tempdir()) or multi-file package fixtures (DESCRIPTION, NAMESPACE) on disk will fail or behave unpredictably under automated fuzzing. Explicitly mark any such block with # nofuzz on the test definition line:

  test_that("namespace_linter detects functions already imported in the NAMESPACE", { # nofuzz
    pkg_dir <- withr::local_tempdir("testpkg")
    ...
  })
  

• Trust the fuzzing suite for standard syntax permutations: Because the automated fuzzing harness exercises interchangeable R syntax structures across tests (such as mutating function(...) definitions into lambda shorthand \(...)), do not demand separate positive/negative unit tests for interchangeable syntax variations (\() vs function()) unless the individual linter implementation has custom code explicitly handling OP-LAMBDA vs FUNCTION differently.

3. Writing expect_lint() Assertions

• Explicit argument names: When asserting lints against a file path (file = test_file), explicitly name the checks and linters arguments rather than relying on positional ordering:

  expect_lint(
    file = test_file,
    checks = list(
      list("Don't use `::` to access median.*already imported", line_number = 1L),
      list("Don't use `:::` to access head.*already imported", line_number = 2L)
    ),
    linters = namespace_linter()
  )
  

• Avoid rex::rex() around plain strings to reduce visual noise: expect_lint() natively interprets check strings as regular expressions. Do not wrap plain target strings in rex::rex("...") when no regex metacharacters require escaping ((), [], ., $, etc.) or advanced composition. The specific purpose of omitting rex::rex() on clean literal matches is to directly eliminate repetitive visual noise in test suites, but it should be used to reduce the visual noise of '\' used to escape regex specials, especially repeated ones.
• Pass file paths using the file = keyword argument: When testing a file on disk (tmp_file), always pass it using file = tmp_file. Passing a file path as the first positional argument (expect_lint(tmp_file, ...)) passes it as , causing to treat the file path string itself as R code (resulting in false-positive syntax errors like ).

4. Concise File Fixtures (withr::local_tempfile)

• Create and populate tempfiles in one step: When creating single-file test fixtures for expect_lint() or expect_no_lint(), use withr::local_tempfile(fileext = ".Rmd", lines = c(...)) to create and write the file cleanly:

  test_that("chunkless files are fine", {
    tmp <- withr::local_tempfile(fileext = ".Rmd", lines = c(
      "---",
      "some_option: true",
      "---",
      "Some text!"
    ))
    expect_no_lint(file = tmp, linters = assignment_linter())
  })
  

5. Testing Document Structure & Literate Formats

• Keep literate (.Rmd/.qmd) boundary tests out of individual linter suites: It is exceedingly rare for an individual linter to warrant having .Rmd, .qmd, or other literate document format tests within its own dedicated test file (tests/testthat/test-<linter_name>.R). Almost exclusively, tests targeting literate formats, NA_character_ masked line extraction, zero-chunk, or multi-chunk boundary parsing belong in tests/testthat/test-knitr_formats.R (or extraction suites). Do not request or add .Rmd/.qmd unit tests when reviewing individual linter implementations, unless the linter executes custom format-specific logic directly dependent on those extensions.

6. Rules Governing # nocov and Coverage Patches

• Exhaust public reachability before annotating: Never add # nocov or # nocov start/end to unreached lines until you have verified through public interface boundaries (lint(), lint_package(), read_settings()) that no realistic code pattern, malformed file format, or configuration structure can execute that path.
• Eliminate dead parameters and impossible branches: If an unreached branch (op_xpath <- NULL) or function parameter (normalize_path = FALSE across internal utilities) is never triggered in production, remove or simplify the dead logic outright rather than masking it under # nocov.
• Convert impossible states to explicit internal errors if appropriate: If defensive checks guard against violations of foundational R grammatical rules (such as zero-child <expr> AST nodes), raise an intentional internal error (cli_abort_internal("Invalid state encountered...")) rather than silently returning empty outputs.

7. All tests must use the public API

• Test unexported helpers exclusively through public entry points: Even though {testthat} injects package private namespaces (pkgload::load_all()), always route argument validations and error assertions across public functions (lint(), lint_dir(), lint_package()) to prevent unit tests from coupling directly to internal private mechanics. Don't mention implementation details when describing/commenting tests, either.
• Use authentic parsed structures over mock lists: Never generate simplified artificial mock lists (list(full_parsed_content = 1L)) to satisfy internal type verification. Always generate real source_expression objects using get_source_expressions() and extract real configuration files (write.dcf()) to verify production structures accurately.

8. Robust Filesystem Paths & Cache Hashing on Windows

• Normalize paths before hashing or comparison: Whenever writing unit tests that read, corrupt, or assert specific filesystem locations (get_cache_file_path(file, path) or temporary file checking), always pass file strings through normalize_path(file) before calculating SHA1 digests or checking outputs. Un-normalized paths from withr::local_tempfile() retain Windows backslashes (\) and unexpanded short segment names (~1.TMP), which alter SHA1 digests (digest::digest(..., algo = "sha1")) and cause false-positive failures on Windows platforms.

9. Empirical Test Case Synthesis & Modular Grouping

• Distill real-world, empirical usage over abstract toy examples: When verifying complex functions or new options (list.files(), dir(), pipelines), research and synthesize actual pattern combinations used across open-source and real-world codebases. Exercise realistic flags (full.names = TRUE, recursive = TRUE, ignore.case = TRUE), piped invocations (getwd() |> list.files(...)), dynamic combinations (paste0(...)), exact regex anchors ("csv$"), and literal string escapes ("_bmarks\\.csv" vs "_bmarks.csv").
• Group assertions into modular, focused test_that() blocks: Avoid stuffing all variations into one monolithic test block. Separate tests into clearly labeled blocks by testing concern:
  1. Standard keyword invocations (pattern = ...) and empirical configuration options.
  2. Positional parameter lookups and positional pipelines (list.files("path", "pattern") and "dir" |> list.files("pattern")).
  3. Option toggles and behavior overrides (check_file_listing = FALSE).
• Comprehensive test matrix for linter extensions: Whenever extending a linter to check new functions or syntax variations (e.g. 2-argument seq()), systematically test all dimensions of the empirical matrix:
  1. Positional arguments: seq(1, 10)
  2. Standard named arguments: seq(from = 1, to = 10)
  3. Inverted named arguments: seq(to = 10, from = 1)
  4. Mixed positional/named arguments: seq(1, to = 10), seq(to = 10, 1), seq(10, from = 1), seq(from = 1, 10)
  5. Literal types: 1L, 10L, 1
  6. Boundary / single-element calls: seq(1, 1), seq(from = 1, to = 1)
  7. Decreasing sequence calls: seq(10, 1), seq(from = 10, to = 1), seq(to = 1, from = 10), , ,

10. Failing Regression Tests for Reported Bugs & Review Findings

• No failing test = no bug claim: In all but exceptional cases, any claim of a bug, unhandled warning, unhandled edge case, parser crash, or regression in a PR or issue must be substantiated by a concrete, executable regression test (e.g. expect_lint(...), expect_no_lint(...), or a standalone test snippet) that fails on the current code and succeeds with the proposed fix.
• Empirical proof over theoretical hazards: Do not claim a function or chunk option is vulnerable to unhandled conditions (e.g. warnings, invalid vector lengths, type coercion errors) without first writing a reprex that actually triggers the failure in an R session. If no valid or invalid input can trigger the issue, the hazard is theoretical and must not be reported as a defect.
• Prohibit inventing artificial syntax symmetries: When writing or suggesting tests for literate formats or external tool integrations (knitr, quarto), tests must exclusively assert valid, documented, and empirically verified upstream behaviors. Never suggest or write tests based on theoretical symmetry (such as {python, engine = "r"}) without verifying upstream execution (e.g. via knitr::knit()).