| name | control-design |
| description | Use BEFORE writing or changing any check, test, gate, guard or assertion — while the control is being built. Answers ONE question, will this fire when the defect is present. Catches controls that cannot fail, negative controls that planted nothing, probes answered by a guard before they reach the subject, floors that cannot detect a relocation, and checks that enumerate documents while the surface is what those documents need. Triggers on "add a check", "write a test for", "gate this", "assert that", "guard against", or any first run of a new control. Secrets in fixtures, verifications and startup branches moved to credential-custody. |
Control Design — will this control fire when the defect is present?
One question only. What surface does it cover, and what claim may I write from it? is a
different question with a different failure mode — see control-scope.
Reading a verdict you did not write: control-readout.
Claiming your own work is done: proof-discipline.
Anything that has to HANDLE a secret: credential-custody.
Split from a 480-line file on 2026-08-02, and split again on 2026-08-03 when it had grown
back to 495, and a third time on 2026-08-04 at 232 lines, when the three credential-handling
sections left for credential-custody — they had ridden along
since the first split without ever answering this file's question.
The recurring cause is that soundness and scope read as one topic while writing
and as two topics while debugging. The risk being removed is not length — it is that the half
you needed sat inside a document you opened for the other reason.
The one line to carry: a control that cannot fail is worth exactly as much as no control,
and it costs more, because it also buys false confidence. Before a control is finished, make
it go red on purpose and watch it.
Failure-Mode Catalogue (pattern-match fast)
| # | Failure mode | Smell | Detection |
|---|
| 1 | Sub-scale fixture | "tiny fixture, all green" | EXPLAIN (ANALYZE) at fixture size vs a 50k+ row copy — does the plan node change? |
| 2 | Forced-plan artifact | test drops an index / sets seqscan=off / stubs a guard to make the path fire | grep the test for drop index, enable_seqscan, mocks; if the path needs forcing, it is a demo |
| 3 | Silent cap | result set "looks complete", never tested past a limit | load cap+1 matching rows; assert count == cap+1, not cap. Sweep max_scan_tuples, least(count,N), ef_search |
| 4 | Deployed/template drift | "the migration says X" but the live object differs | pg_get_functiondef('schema.fn'::regproc) diff against the .sql source |
| 5 | Observed-not-proven security | "0 leak" seen once, in geometry where a leak could not show | rebuild fixture with tenants interleaved; run via signed JWT + real role; assert exact id-set |
| 6 | Vacuous control | "I broke it and the check caught it" — but the thing you broke was never there | assert the PRECONDITION first (below) |
| 7 | Misaimed instrument | "I searched, found nothing, and proved my search works" | a scope failure → control-scope |
| 8 | Answered by the guard | a real, confident answer from a path that never touched the subject | craft the input to PASS the guard; if the two states now differ, the old probe was measuring the guard |
6 in full — verify the precondition before trusting the control
A negative control proves something only if the thing you removed was present to remove.
Delete a token that was never there and you planted nothing: the suite passes, and the pass
says nothing about whether the check can fail.
Same class as a suite going green because 19 DB-gated files silently skipped, or a grep
whose alternation never matched — a clean result from a check that never ran looks identical
to a clean result from a clean system.
It bites hardest on exemptions. When you write a rule then exempt yourself from it, the
control proving the exemption is still narrow is the only thing between "declared" and
"disabled". A vacuous control there is worse than none: it manufactures confidence.
sed -i 's/disabled/_removed/' target.ts && run_suite
grep -q 'disabled' target.ts || { echo "anchor absent — control would be vacuous"; exit 1; }
sed -i 's/disabled/_removed/' target.ts
grep -c '_removed' target.ts
run_suite
If the anchor is absent, do not silently substitute a different control — say the control
could not be run, and find one whose precondition holds.
Earned 2026-08-01: a per-file exemption was reported "controlled" after an attempt to remove a
disabled token from a file containing none. Nothing was planted; the 22/22 pass was
meaningless. The real control — planting an undeclared construct in the same file — failed
2 tests, which is what the exemption's narrowness actually rests on.
This file was itself failure mode 4. It lived only at ~/.claude/skills/, which is
gitignored and does not travel, so the lesson about verification proving nothing existed on one
machine and nowhere else. Ruling 2026-08-01: the repo is canonical, the machine copy is a
deploy artifact, one-way repo → machine, never the reverse. Editing ~/.claude/skills/ in
place is editing a production server. skills-drift-check fails the build if they diverge.
The first run of a new control is the FAILING one
A new check is not trusted until it has been observed to FAIL — and to fail for the reason
you think. Write it, aim it at a defect you have planted, watch it go red, and only then aim
it at the real system. A control whose first observed state is green has been tested for its
ability to agree with you.
Verify the failure, not just the exit code. "It failed" is not enough — read the message
and confirm it names the planted defect.
Controls fail toward green, and the bias is directional, not random. 2026-08-01/02, four
mis-designs in one session, all four green:
| # | Control | What went wrong | Read as |
|---|
| 1 | review tree-integrity | planted file at *.tmp, gitignored repo-wide | "no mutation" |
| 2 | secrets-scan coverage | planted secret in .md, which is in _SKIP_EXTS | "no secrets" |
| 3 | build-freshness | exit code measured through | head, reporting head's 0 | "control passed" |
| 4 | route-exercise | attached to an orphaned server serving the previous build | "no broken route" |
Plus a fifth in the tool built to catch exactly this: a history scanner whose input paths all
carried a stray \r, so it read 0 blobs and printed "no secrets found" over 6160 paths.
Five of five landed on green. Not chance — you write a test expecting it to pass, so every
accident lands on the side you expected. Assume your control is green because it is broken
until you have seen it red.
The one that went the other way, and why it does not soften the rule. A sixth reported FAIL
against a working scanner: scanner | grep -q, where the scanner exits 1 on a violation, so
under pipefail the pipeline was non-zero even though grep matched. A false RED. It cost ten
minutes and was self-correcting, because a red result gets investigated. A false green is
never investigated, because nobody audits good news. Both are bugs; only one is dangerous.
Three habits that catch all five:
- Plant the defect where the check must look, not merely nearby. Check ignore rules,
extension filters and path prefixes first.
- Never measure an exit code through a pipe.
cmd > file; echo $?, never cmd | head.
- Assert the scan did work: blobs read > 0, files scanned > 0, paths exercised > 0. A
checker that examined nothing must fail, not pass.
A control must REACH the code path that would reveal the defect
Sharper than "the reassuring field cannot fail", and a different fault. Here the control
ran, returned a real answer, and answered a different question — because the
subject sits behind a guard that replies first.
2026-08-04, DR-NRPG. A watcher decided webhook_secret_configured from "HTTP 400 mentioning
signature". The handler validates the signature header before it reads
STRIPE_WEBHOOK_SECRET, so an unsigned request is answered by the guard, not by the thing
being measured — and gets that 400 whether the secret exists or not. The field read true for
every run the watcher had ever made. It was false.
The discriminating fixture is a PAIR, and it belongs on the live endpoint, not in a file:
unsigned -> 400 "Missing Stripe signature" <- the guard talking
signed-but-invalid -> 500 "Missing STRIPE_WEBHOOK_SECRET" <- the subject talking
Same endpoint, same second, opposite verdicts. Craft the input so it passes the guard and
fails at the subject. If both states still produce the same response, the probe cannot see
the subject and owes you undetermined — not a verdict.
Ask of any probe: what is the first thing that could answer this, and is it the thing I am
asking about? Ordering decides who replies — header before secret, auth before handler,
validation before write — and the earliest replier wins.
A page is not a surface — what it NEEDS is classified separately
A control that enumerates documents has not enumerated the surface. Walk the route tree and
you have found pages and endpoints. What each one needs in order to work — its bundle, its
styles, its data calls — is classified by a different rule, usually a path prefix. A change to
paths moves those across a security boundary without touching the document, and the control
stays green because everything it asserts about the document is still true.
It presents as broken, not blocked, which is what makes it expensive. A blocked page sends
you to the auth config. A page that renders unstyled, whose bundle 302s, sends you to the CDN,
the build, the deploy — the investigation starts in the wrong system entirely.
2026-08-04, Pi-Dev-Ops. Setting assetPrefix for a multi-zone move put every /_next/* URL
inside /command-centre, a protected prefix. The public login page's own CSS and JS were
then answered with a redirect to login. Twenty-two auth assertions stayed green; not one probed
an asset path.
Ask of any path-prefix rule: what else did that prefix just capture? Probe one of those, and
probe a near-miss — /_nextdoor, or ?ref=/_next/x — so the boundary is asserted, not assumed.
