test-tripwire

# Tripwire Tests A tripwire test asserts an **assumption the code relies on but does not enforce**. It passes today and checks no behaviour. It fails only when a later change breaks the assumption, and it fails in the PR that makes that change. The usual reason to write one is a deferred fix. An issue says "if the list ever grows past ten, add a count beside it", and the condition is a fact about the code. Filed as an issue, it sits open for months and nobody connects it to the commit that finally triggers it. Written as a test, it fails in front of the one person who needs to read it. Related names: *fitness function* (Ford, Parsons and Kua, *Building Evolutionary Architectures*), *executable TODO*, and `static_assert` for the compile-time form. ## When to Use This Skill | Use a tripwire test when... | Use something else when... | |---|---| | A deferred fix waits on a condition visible in the source: a list length, a constant, an enum's members, a pinned dependency version | The trigger is a date or an outside event (vendor deprecation, contract end) → an issue or a dated reminder | | Code truncates, caps or indexes by a size it assumes, and nothing checks the assumption | The trigger is runtime data (row counts, traffic) → a runtime assertion or monitoring alert | | Two lists or constants must stay in a relation (disjoint, equal, one within the other) and live in different files | The assumption is enforceable at the boundary → a guard clause (`software-design-plugin:design-by-contract`) | | Grooming finds an open issue whose trigger is a code condition | You want to know whether existing tests catch regressions → `testing-plugin:test-strategy-review` / `testing-plugin:mutation-testing` | ## Writing One 1. **Assert the assumption, not the behaviour.** Compare the two facts directly: `expect(CODES.length).toBeLessThanOrEqual(LIST_LIMIT)`. Driving the code path is unnecessary; the assumption is the subject. 2. **Import the real values.** Export the constant if it is private. A copied literal in the test drifts silently and the tripwire stops tracking anything. 3. **Name the reason in the test name**, with the issue or PR that deferred the fix: `"fits within LIST_LIMIT, since the committees list has no dropped count (#123)"`. The failure message is the only documentation the next person is sure to see. 4. **List every resolution in a comment**, and state what each does to the test. Be exact: "raise the limit" makes it pass; "add a dropped count" does not, so that path says to retire the test. A comment claiming a fix makes the test pass when it does not sends the reader in a circle. 5. **Place it beside the tests of the code that relies on the assumption**, not in a catch-all file, so the reader lands with context. ```ts describe("COMMITTEE_CODES", () => { it("fits within LIST_LIMIT, since the run row's committee list has no dropped count (#123)", () => { // The run row caps the per-committee list at LIST_LIMIT with no count // beside it, unlike errors/errorsTotal. Past the limit it would drop // entries silently. Either raise the limit, or add a total beside the // list and retire this test. expect(COMMITTEE_CODES.length).toBeLessThanOrEqual(LIST_LIMIT) }) }) ``` Other shapes the same pattern takes: | Assumption | Assertion | |---|---| | Retired codes must never return to the active list | every retired code is absent from the active list | | A copy kept in another runtime (a migration image, a script) must match the source | the two lists are equal | | An enum handled by a `switch` has exactly N members | the member count, with the `switch` named in the comment | | A workaround is needed only below version X | the installed version is below X; the comment says to delete the workaround | ## Verify It Can Fail A tripwire that cannot fail is an issue nobody reads in a different place. Break the assumption on purpose and watch the test go red. Run the probe through the `mutation-check.sh` helper that `agent-patterns-plugin:parallel-agent-dispatch` ships in its scripts directory (call it by its absolute path in the installed plugin). It snapshots the file, applies one sed expression, runs your command, and restores the file on every exit path, including a failed or interrupted run. A hand-written edit-run-restore sequence leaves the file mutated whenever its restore step is skipped, and an in-place sed edit is blocked by the `sed-inplace` hook. Without that plugin, make the mutation with the Edit tool and revert it with Edit straight after the run, before any other step. 1. Run only the tripwire on the unmutated file and confirm it is green. The helper runs no baseline, so a command that is already red would count as a kill. 2. Mutate the value so the assumption no longer holds (lower the limit below the list length, add a retired code to the active list): ```bash mutation-check.sh <file> '<sed-expr>' -- <tripwire command> mutation-check.sh src/limits.ts 's/LIST_LIMIT = 10/LIST_LIMIT = 4/' -- bunx vitest run src/limits.test.ts -t "fits within LIST_LIMIT" ``` 3. Read the verdict from stdout. `MUTANT=killed` means the tripwire can fail. `MUTANT=survived` means it passed with the assumption broken, so it is inert: fix the assertion and probe again. `STATUS=ERROR` is no verdict (a sed expression that changed nothing, a command that could not start); correct the call and rerun. 4. On a kill, confirm from the test output (the helper sends it to stderr) that the tripwire failed **on the assertion**, with a message that names both values (`expected 5 to be less than or equal to 4`), not on an import or type error. Check `RESTORED=` and re-run the tripwire green. Report the control with the test: the mutation (the sed expression), the failure line, and the restored value. ## Closing the Issue When a tripwire replaces a deferred item, close or narrow the issue in the same PR. Its condition now lives in CI, so the issue only adds a second copy that can drift. Link the test from the closing comment or the PR body. ## Agentic Optimizations | Context | Command | |---|---| | Run only the tripwire (Vitest) | `bunx vitest run <file> -t "<test name fragment>"` | | Run only the tripwire (pytest) | `pytest <file> -k "<name fragment>" -q` | | Find where the assumed size is used | `rg -n '\.slice\(0, LIST_LIMIT\)\|LIST_LIMIT' src` | | Check the issue thread before replacing it | `gh issue view <n> --json title,body,comments` |