Execution Mode Selection (HARD GATE โ must complete before executing ANY task)
Do NOT execute any task before completing this analysis. Analyze independence first, select the execution tier, then begin. Starting sequential execution "because the first tasks feel simple" is a workflow violation โ it forfeits parallelization savings on the remaining tasks.
Before starting the sequential task loop, check for parallelization opportunities:
Step 0: Tier 0 pre-check (Lifecycle Parallelism)
Read the plan. Apply a single judgment: "Does this plan have distinct code and test workstreams that can be assigned to separate agents with non-overlapping file scopes?"
- If yes (interactive mode): offer Tier 0 to the user
- If yes (pipeline mode): auto-select Tier 0 without prompting
- If declined or ineligible: fall through to Step 1 below
Read plugins/soleur/skills/work/references/work-lifecycle-parallel.md now for the full Tier 0 protocol (offer/auto-select, generate contract, spawn 2 agents, collect/commit, test-fix-loop, docs). If Tier 0 executes, proceed directly to Phase 3 after completing Step 06 of the protocol. If declined, fall through to Step 1.
Step 1: Analyze independence
Read the TaskList. Identify tasks that have no blockedBy dependencies and reference
different files or modules (no obvious file overlap). Count the independent tasks.
If fewer than 3 independent tasks exist, skip to Tier C: Sequential below.
If 3+ independent tasks exist, proceed through the tiers in order (A, then B, then C).
Each tier either executes or falls through to the next.
Pipeline mode override: If running in pipeline mode (plan file argument detected in Phase 1), auto-select Tier 0 if eligible (Step 0 above). If Tier 0 is ineligible, skip Tier A entirely and auto-accept Tier B without prompting. Do not present "Run as Agent Team?" or "Run in parallel?" questions -- proceed directly to Step B2 of the Subagent Fan-Out protocol if 3+ independent tasks exist, otherwise fall through to Tier C.
Tier A: Agent Teams (highest capability, ~7x token cost)
Read plugins/soleur/skills/work/references/work-agent-teams.md now for the full Agent Teams protocol (offer, activate, spawn teammates, monitor/commit/shutdown). If declined or failed, fall through to Tier B.
Tier B: Subagent Fan-Out (fire-and-gather, moderate cost)
Read plugins/soleur/skills/work/references/work-subagent-fanout.md now for the full Subagent Fan-Out protocol (offer, group/spawn, collect/integrate). If declined, fall through to Tier C.
Tier C: Sequential (default)
Proceed to the task execution loop below.
Task Execution Loop
Design Artifact Gate (before first UI task): If DESIGN_ARTIFACTS was set in Phase 0.5, spawn the ux-design-lead agent with the artifact paths and ask it to produce an implementation brief (see ux-design-lead "Wireframe-to-Implementation Handoff" workflow). The brief is a structured description of every section, its content, and its layout โ this becomes the binding input for all UI tasks. Do not write any markup until the brief is received.
UX artifact commit checkpoint (after Design Artifact Gate): After the implementation brief is received, commit before proceeding to UI tasks:
- Run
git status --short to discover the implementation brief and any generated design files
- Stage output files:
git add <discovered files>
- Commit:
git commit -m "wip: UX implementation brief for <feature-name>"
This checkpoint ensures the implementation brief survives session crashes.
For each task in priority order:
while (tasks remain):
- Mark task as in_progress in TodoWrite
- Read any referenced files from the plan
- If task creates UI/pages: verify implementation brief exists (HARD GATE)
- TDD GATE: (see below)
- Look for similar patterns in codebase
- RED: Write failing test(s) for this task's acceptance criteria
- GREEN: Write minimum code to make the test(s) pass
- REFACTOR: Improve code while keeping tests green
- Run full test suite after changes
- Mark task as completed in TodoWrite
- Mark off the corresponding checkbox in the plan file ([ ] โ [x])
- Evaluate for incremental commit (see below)
No mid-plan pause gates (HARD GATE). A multi-phase plan
(tasks.md Phase 0 through Phase N) is a SINGLE execution unit.
Do NOT insert "Pause for review or continue?" prompts between
phases. Do NOT end a turn after one phase commits with "Continue
into Phase N+1 next turn?". The skill's Phase 4 handoff is the
only sanctioned stopping point โ until then, chain straight
through every phase the plan defines, including phases the plan
labels "Pre-merge verification" or "Post-merge (operator)" if
they're automatable per the next gate. Why: the founder is a
solo operator; every "continue or pause?" is a context switch
that defeats the entire point of a multi-phase plan. Pipeline
mode (file-path arg in Phase 1) means pipeline mode for the WHOLE
plan, not per-phase.
Operator-step automation gate (HARD GATE). Before treating
any task in tasks.md as "operator-driven" (apply migration,
verify pg_cron, verify Storage bucket, run end-to-end smoke,
gh pr ready, gh pr merge --auto), check whether it is
automatable via a loaded MCP server or CLI:
- Supabase migrations +
cron.job queries + Storage bucket
existence + RLS spot-checks โ mcp__plugin_supabase_supabase__*
with Doppler DATABASE_URL_POOLER fallback when MCP is
unavailable โ see "Supabase fallback chain" below. When a
migration needs a SECURITY DEFINER RPC (e.g. to bypass an RLS /
column-grant restriction), start from
sql-security-definer-rpc-scaffold.sql
โ it encodes the search_path pin + 4-role REVOKE + auth.uid()
authorization pin that test/migration-rpc-grants.test.ts enforces.
gh pr ready / gh pr merge --squash --auto / gh issue close
โ Bash via gh CLI
- End-to-end UI flow โ Playwright MCP (
mcp__playwright__*)
- Cloudflare DNS / WAF / Workers โ
mcp__plugin_soleur_cloudflare__*
- Live Stripe state โ
mcp__plugin_soleur_stripe__*
If automatable, EXECUTE it inline as part of the work pipeline โ
never list it back to the operator. The /ship skill already
handles gh pr ready + auto-merge + migration verification (see
plugins/soleur/skills/ship/SKILL.md); chain to /soleur:ship
at Phase 4 and let it run. For migration apply to dev (vs
verify), invoke mcp__plugin_supabase_supabase__apply_migration
inline at the phase where the migration lands, not as a
post-merge todo. Why: see ship/SKILL.md:1027 ("Every 'please
run this manually' is a context switch") and ship/SKILL.md:1177
(PR #1375 โ migration verification was left as a manual
"post-merge todo" instead of being executed; deployed code
expected the new schema and broke). Same class as the
Playwright-first audit in Phase 4: if a tool exists, use it.
Supabase fallback chain (when MCP OAuth fails). The Supabase
MCP OAuth flow at https://api.supabase.com/v1/oauth/authorize
intermittently rejects valid URLs at the dashboard auth_id
handoff (cause: external โ Supabase-side). When that happens, do
NOT fall back to "paste this SQL into the dashboard SQL editor"
handoff โ that's a manual-step rationalisation that violates
hr-never-label-any-step-as-manual-without. Instead walk down the
hr-exhaust-all-automated-options-before priority chain:
(1) Doppler DATABASE_URL_POOLER โ already provisioned for every
env; the migration apply path. (2) Verify the project ref in the
URL matches the plan's stated dev/prd refs โ Doppler is the
source of truth (plan-quoted project refs are preconditions to
verify, never facts; the plan can drift). (3) Rewrite the URL's
port :6543 โ :5432 so the pooler runs in session mode (multi-
statement DDL works; transaction mode rejects with SQLSTATE 42601
"cannot insert multiple commands into a prepared statement").
(4) Apply via pg (node-pg, bun-installed in /tmp if missing)
wrapped in BEGIN; <migration>; COMMIT;. The direct DB host
db.<ref>.supabase.co:5432 is IPv6-only and typically
unreachable from operator/CI networks; the pooler is IPv4.
(5) Post-apply, verify schema via the same connection โ RLS
enabled, policy_count, trigger names, RPC signatures + SECURITY
DEFINER flag, UNIQUE constraints. Write the verification artifact
to knowledge-base/project/specs/feat-<name>/migration-checklist.md.
Why: PR #3853 / #3205 โ Supabase MCP OAuth was rejecting URLs
at the auth_id handoff; the agent first proposed "paste SQL into
dashboard" (manual-step violation), then pivoted to Playwright-
first audit on dashboard navigation (correct), then discovered
Doppler had the working DATABASE_URL_POOLER and applied via
pg directly โ the path it should have taken at step 1.
Session stickiness: once the MCP OAuth handoff has failed even
once in the current session, treat Doppler DATABASE_URL_POOLER as
the default for ALL subsequent Supabase operations this session โ do
not re-attempt the OAuth flow per-operation. Re-probing a known-flaky
external auth each time is the wasted-cycle trap; the fallback is not
slower once you are already authenticated to Doppler.
Pre-apply collision check (always, even on first attempt).
Before invoking pg apply (or supabase migration up) against any
shared env, run git fetch origin main && git ls-tree origin/main -- apps/web-platform/supabase/migrations/ | awk '{print $4}' | grep -oE '^[0-9]{3}_[^.]+' | sort -u. For each LOCAL migration
file the branch introduces, assert no DIFFERENT filename with the
same 3-digit prefix exists in that list. A collision means a
sibling PR is landing the same number window; renumber FIRST,
then apply under the final filename. Why: PR #4225 โ applied
053โ057 in the morning; PR #4251 landed 054_schema_migrations_ content_sha.sql 10 hours later and main's CI drift probe flagged
the entire branch; the recovery (renumber 054โ058, 055โ059, 056โ060,
057โ061 + reconcile public._schema_migrations on both dev + prd
via git hash-object content_sha) took ~30 min and could have been
zero-cost if the operator had grepped origin/main first.
The collision window extends through /ship, not just work-time. This
check at work-start is necessary but NOT sufficient: a sibling migration can
land on main DURING the (often 30โ90 min) ship phase โ especially under a
fast-moving-main burst where /ship Phase 7 performs repeated git merge origin/main auto-syncs on OPEN BEHIND. Each sync that pulls in a sibling
supabase/migrations/NNN_*.sql sharing your prefix is a silent collision the
BEHIND loop pushes straight to CI (where the migration drift/shape gate fails,
~16 min later). After ANY ship-time sync whose merge output lists
supabase/migrations/, re-run the prefix check above and renumber-during-ship
(git mv both up/down + update every in-repo reference: migration headers,
code comments, plan/tasks/learning) BEFORE the next push. Why: PR #5760 โ
114_disk_io_top_wal_statements (a #5739-sibling) landed mid-ship; my
114_prune_cron_job_run_details collided and surfaced only at CI after ~6
auto-syncs; recovery was a renumber to 115. See
knowledge-base/project/learnings/workflow-patterns/2026-06-30-migration-number-collision-mid-pipeline.md.
Tracking row in the SAME transaction as the migration body.
The project's canonical apps/web-platform/scripts/run-migrations.sh
writes INSERT INTO public._schema_migrations (filename, content_sha) VALUES ('<basename>', '<git-hash-object>') in the same transaction
as the migration SQL. The Doppler+pg fallback MUST mirror this โ
bare BEGIN; <migration>; COMMIT; produces a phantom-applied state
where the schema reflects the migration but _schema_migrations
does not, and the next deploy attempts re-apply (failing on
non-idempotent statements like CREATE TRIGGER). The reconciliation
pattern (UPSERT with ON CONFLICT (filename) DO UPDATE SET content_sha = EXCLUDED.content_sha) is the recovery shape โ but
doing it inline is cheaper.
PostgREST schema cache reload via session-mode pooler does NOT
work. NOTIFY pgrst, 'reload schema' over a :5432 pooler
connection does not reach PostgREST's LISTEN (PgBouncer
multiplexes; LISTEN/NOTIFY channel scope is bound to backend
process identity, not session). 90 attempts over 5 minutes
returned PGRST205. After a direct-pg apply: either wait for the
natural ~10-min schema poll cycle, OR use the Supabase Management
API to restart PostgREST. The direct DB host
(db.<ref>.supabase.co:5432) is IPv6-only and typically
unreachable from operator networks, so the canonical "NOTIFY via
direct connection" workaround documented upstream isn't available.
Storage-bucket migrations: down.sql cannot DELETE storage tables;
column-takeover proof is permissive-vs-restrictive, not name-count.
Supabase installs a platform BEFORE DELETE trigger (protect_objects_delete
โ storage.protect_delete()) that blocks direct DELETE FROM storage.objects
AND storage.buckets ("Direct deletion from storage tables is not allowed").
So a bucket migration's down.sql reverts only SQL-droppable objects
(policies โ function โ column) โ NOT the bucket/objects (Storage-API/operator
teardown; 019/042 precedent ship none; 071's DELETE FROM storage.buckets is
a dormant bug). Runtime cleanup uses service.storage.from(b).remove([...])
(allowed). And when verifying "no client can write column X" (read-proxy
trust), assert no PERMISSIVE INSERT/UPDATE/DELETE/ALL policy (a
RESTRICTIVE FOR ALL like workspaces_jti_not_denied only denies, never
grants) + a behavioral authenticated UPDATE affecting 0 rows. The
pooler also presents a self-signed CA chain โ transient node-pg verify
scripts use ssl:{rejectUnauthorized:false} (dev-only, mirrors
run-migrations.sh sslmode=require; no committed code disables TLS verify).
See knowledge-base/project/learnings/2026-06-04-supabase-bucket-migration-down-and-rls-takeover-proof.md (#4916).
TDD Gate (HARD GATE): Before writing ANY implementation code for a task, determine if the task has testable behavior:
Emit rule-application telemetry (records that the TDD gate was reached โ see AGENTS.md cq-write-failing-tests-before):
source "$(git rev-parse --show-toplevel)/.claude/hooks/lib/incidents.sh" && \
emit_incident cq-write-failing-tests-before applied \
"Write failing tests BEFORE implementation code whe"
- Check: Does the plan have a "Test Scenarios" or "Acceptance Criteria" section that covers this task? If yes, this task requires test-first.
- Exempt: Infrastructure-only tasks (config files, CI workflows, scaffolding directories, dependency installs) are exempt. If the task only creates/modifies config, it skips to Infrastructure Validation below.
- Enforce: For non-exempt tasks, write the failing test file FIRST. The test must:
- Import the component/function/module that will be created (the import will fail โ that is correct)
- Assert the specific behavior from the acceptance criteria
- Be runnable via the project's test command (even if it fails due to missing implementation)
- Verify RED: Run the test. It must fail (missing module, assertion failure, etc.). If it passes, the test is not testing new behavior โ rewrite it. For gating/sequencing primitives (semaphores, locks, queues, ordering guarantees), the test must distinguish gate-absent from gate-present: add an intermediate-state assertion that would fail without the primitive (e.g.,
count === 2 while two slots are held) in addition to the final-state assertion. A test that passes identically with and without the primitive isn't testing the primitive. See knowledge-base/project/learnings/test-failures/2026-04-18-red-verification-must-distinguish-gated-from-ungated.md. Test-environment fidelity: if the SUT's buggy code lives behind a guard (if [[ -d "$X" ]], if (cache.has(key)), etc.), the harness MUST seed the precondition the guard requires โ otherwise both buggy and fixed paths short-circuit identically and any negative-space assertion passes vacuously. See knowledge-base/project/learnings/test-failures/2026-04-22-red-test-must-simulate-suts-preconditions.md. Early-exit shadowing: if the SUT has a guarded fast path (substring strip like replaceAll(arg, ""), cache-hit, env-flag short-circuit) that handles a superset of inputs the slow path under test handles, RED inputs MUST choose identities ONLY the slow path can produce. Sharing a fixture across the fast/slow boundary lets the fast path scrub first and the regex/branch under test never fires โ the assertion passes without testing the fix. Add an invariant guard test asserting the fast/slow fixtures do not collide. See knowledge-base/project/learnings/2026-05-04-vacuous-red-via-shared-fixture-and-toolchain-pinning.md. In-component state machines (RTL): when the gate-under-test is component-local state (useState/useRef/useReducer), drive the SUT through state transitions with result.rerender(<C />) โ never unmount() + fresh render(). Remount resets the in-component bookkeeping that IS the gate, producing vacuous green. See knowledge-base/project/learnings/test-failures/2026-05-11-rerender-not-remount-for-in-component-state-machine-tests.md. Laundered-target resolvability (normalizer/strip/prefix-mangle security fixes): a regression guard for an anchored-strip / path-canonicalization / allowlist-key fix is vacuous unless the fixture makes the LAUNDERED (mis-normalized) target resolvable to an observable effect โ if the downstream gate rejects it for an unrelated reason (nonexistent skill/row/file), the test passes identically with and without the fix. Litmus: under the buggy impl, does this input produce a DIFFERENT output than under the correct one? See knowledge-base/project/learnings/test-failures/2026-07-05-security-fix-regression-guard-must-make-the-laundered-target-resolvable.md. Fixture-space cardinality (ask this per contract sentence, and note that your own passing mutation battery cannot answer it): for each property the test claims, name the SET it quantifies over and count how many distinct members the FIXTURE instantiates โ one member is a sample, not a proof, and code-mutation coverage does not detect a fixture-space gap. Three shapes recur: (a) a temporal contract (a wait/retry/debounce/convergence) sampled only by STATIC fixtures probes t=0 and t=โ but never the transition โ the case the guard exists for โ so deleting the loop's break stays green; drive it with a stateful stub that changes on the Nth invocation and assert the success arm was reached VIA the loop; (b) a bidirectional guard (-w, an ordering, a comparison) must be fixtured in the direction where the weaker implementation gives a FALSE POSITIVE, not the direction that fails either way; (c) a stub that ignores argv, or a sleep/clock stubbed to a no-op, silently voids the call-shape and budget contracts โ validate "$*" in the stub and COUNT the stubbed calls against the design's bound. Why: #6441 โ a 7-mutation battery reported 7/7 RED while nine unimagined mutations (loop-break deleted, grep -qwFโ-qF, addr showโlink show, bound 30โ1, sleep 2โ600) all survived. See knowledge-base/project/learnings/2026-07-19-my-own-mutation-battery-was-the-false-confidence.md.
- Only then: Write the minimum implementation to make the test pass (GREEN).
- Refactor: Improve code while keeping tests green.
Skipping this gate โ writing implementation before tests โ is a workflow violation equivalent to committing directly to main. The rationalization "this is simple enough to not need test-first" is exactly the reasoning TDD is designed to prevent.
-
When adding MCP tools to an existing registration block in agent-runner.ts, verify each tool's prerequisites are independent of the block's guard condition. Write a test that validates the new tool works WITHOUT the existing block's prerequisites (e.g., Plausible tools work without GitHub installation).
-
When adding route handler tests that require vi.mock(), create a separate test file from existing unit tests that import the real module. Vitest hoists all vi.mock() calls to the top of the file, clobbering real imports for the entire file regardless of describe block scope.
-
When creating test files with vi.mock() factories that reference shared variables, use vi.hoisted() from the start -- vitest hoists vi.mock to the top of the file before const/let declarations execute.
-
When a NEW shared module will be imported (directly or transitively) by files that already have test suites mocking a node builtin (vi.mock("node:child_process") with spawn-only factories is the common case), do NOT destructure that builtin's exports at module top level (promisify(execFile) crashes EVERY sibling suite at import). Lazy-import inside the function that uses it. Why: #5091 โ _cron-safe-commit.ts's top-level promisify(execFile) broke 28 cron-bug-fixer tests at module load. See knowledge-base/project/learnings/2026-06-10-bot-cron-safe-commit-substrate-symlink-removal.md.
-
Before adding a vi.mock("<module>") to an EXISTING test file, grep the file for a pre-existing mock of the same module (grep -n 'vi.mock' <file> | grep <module-basename>) and wire your spy into that block instead. Vitest registers one mock per resolved module per file; a duplicate does not error โ it silently picks one, and your hoisted spy captures zero calls. Why: PR #5090 โ a new egress-posture-log spy was added as a second @/server/logger mock while the factory test already mocked it ~150 lines down; cost 4 debug cycles. See knowledge-base/project/learnings/bug-fixes/2026-06-10-sandbox-network-plane-not-token-plane-error-shape-triage.md.
-
A WHOLESALE vi.mock("<module>", () => ({...})) replaces the ENTIRE module, dropping every export the factory omits โ so a module with multiple named exports (@/server/logger exposes default AND createChildLogger; observability, db-helper, supabase wrappers similarly) breaks any REAL sibling in the SUT's import graph that consumes a different export. Default to vi.mock(spec, async (importOriginal) => ({ ...await importOriginal(), <override> })); reserve wholesale factories for modules you fully replace โ or skip the mock entirely if the thing under test already mocks the export's consumer. Detection is free: run the FULL test file (never -t "<new test>" alone) โ the RED run surfaces unexpected sibling failures naming the missing export. Why: #5689 โ a wholesale @/server/logger mock dropped createChildLogger (used by probe-octokit.ts via _cron-shared), breaking 10 unrelated arm-2 tests. See knowledge-base/project/learnings/test-failures/2026-06-29-wholesale-module-mock-drops-named-exports-needed-by-transitive-siblings.md.
-
A partial vi.mock(spec, async (importOriginal) => ({ ...actual, B: spy })) override only changes what importERS see โ it does NOT intercept a call made by a REAL sibling function A (kept via ...actual) to B within the same module; A references B through the module's internal lexical binding, not the export object. Symptom: the spy reports 0 calls even though the path clearly runs B. To observe B while keeping A real, mock the deeper boundary B itself crosses (fetch, the DB client, child_process) and assert there. Decision rule: mock the seam the unit under test does not own. Why: #5728 โ overriding postSentryHeartbeat didn't intercept the real finalizeOutputAwareHeartbeat's internal call; fixed by keeping it real + stubbing fetch + asserting the POST URL. See knowledge-base/project/learnings/test-failures/2026-06-30-partial-module-mock-does-not-intercept-intra-module-calls.md.
-
When mocking child_process.spawn, fetch, or any constructor returning an event-emitter-like object, use mockImplementation(() => factory(...)) rather than mockReturnValue(factory(...)). mockReturnValue evaluates the factory eagerly at test-setup time; any queueMicrotask / setTimeout / setImmediate scheduled inside the factory fires BEFORE the SUT attaches its listeners, producing empty event data or an "uncaught error" test timeout. See knowledge-base/project/learnings/test-failures/2026-04-17-vitest-mockReturnValue-eager-factory-async-event-race.md.
-
A vi.fn(() => value) mock declared with a ZERO-parameter implementation cannot be invoked via a (...args) => mock(...args) forwarder (the standard vi.mock factory shape) โ tsc rejects the spread with TS2556 "A spread argument must either have a tuple type or be passed to a rest parameter", even though the vitest run is GREEN (vitest type-checks test files lazily). Give the impl a rest param: vi.fn((..._args: unknown[]) => value), matching sibling vi.fn() mocks. Only a standalone ./node_modules/.bin/tsc --noEmit catches it. Why: #5817 โ execFileSyncMock = vi.fn(() => Buffer.from("")) passed 36/36 tests but failed tsc. See knowledge-base/project/learnings/test-failures/2026-07-01-vitest-zero-arg-mock-cannot-take-spread-suite-green-tsc-red.md.
-
When the SUT awaits something (mkdtemp, a config read, a lock) BEFORE it calls the mocked spawn/fetch and attaches listeners, emit the child's close/error/data events from INSIDE the spawn mock (spawnMock.mockImplementation(() => { queueMicrotask(emit); return child; })), NOT from a sibling top-level queueMicrotask in the test body. A test-level microtask scheduled right after calling the SUT fires during the pre-spawn await gap โ before listeners exist โ so the settle-once promise never resolves and the test times out (16s). The emit must be scheduled relative to when spawn is actually invoked. Why: PR #4970 โ adding await mkdtemp before spawn in c4-render.ts timed out 6 tests until the emit moved inside the mock; see knowledge-base/project/learnings/best-practices/2026-06-05-external-cli-exit-0-is-not-proof-validate-the-artifact.md.
-
When using vi.doMock("specifier", () => { throw new Error("X") }) to simulate a module-init failure, do NOT assert on the inner error message via the SUT's caller. Vitest wraps factory throws with its own synthetic Error ("[vitest] There was an error when mocking a module...") and the inner string is unobservable. Assert on the SUT's observable contract (return shape, observability mirror call) instead โ the throw is a trigger, not a contract. See knowledge-base/project/learnings/2026-05-07-vitest-domock-factory-throw-wrapped-message.md.
-
To prove a cache-hit skips work (not just that the response status is correct), wrap the real implementation in a spy via vi.importActual rather than stubbing the return value: vi.mock("@/module", async () => { const actual = await vi.importActual(...); return { ...actual, expensiveFn: (...args) => { spy(...args); return actual.expensiveFn(...args); } }). Stubbed returns break any downstream behavior that depends on the real output (hash-match, SQL row shape, etc.); wrapping preserves the contract while exposing call counts for assertions like expect(spy).toHaveBeenCalledTimes(1) across a HEAD+GET sequence. Why: In PR #2515, verifying that HEAD populates shareHashVerdictCache so a follow-up GET skips the SHA-256 drain required counting hashStream calls, not stubbing its return โ a stubbed return would have broken the post-drain hash-equality check and masked the very regression the test was meant to catch.
-
When testing decorative images (alt="") with happy-dom, use container.querySelector instead of screen.getAllByRole("img", { hidden: true }) -- happy-dom excludes presentational elements from role queries even with hidden: true.
-
When asserting against a conditional render branch in a component test, grep the test file's vi.mock(...) factories for the inputs the branch reads and confirm the mock returns values that activate the target branch. Mocks that simplify (e.g., getDisplayName: (id) => id.toUpperCase()) often skip production branches like leader.title.includes(displayName) โ assertions on the skipped branch fail for non-bug reasons. Why: PR #3427 โ see knowledge-base/project/learnings/2026-05-07-test-assertion-must-verify-mock-activates-branch.md.
-
A wait-on-ABSENCE (await vi.waitFor(() => expect(queryByTestId(x)).toBeNull())) is vacuous โ it passes on the FIRST tick, before the async work resolves, so it never proves "absent AFTER the state commit." Anchor the wait on a positive settle signal (e.g., a .finally(() => { settled = true; }) flag on the mocked response body), then assert absence. Also: vitest's vi.waitFor and RTL's waitFor/findBy* have independent 1 s defaults and independent config surfaces โ a global RTL asyncUtilTimeout bump does not touch vi.waitFor call sites. Why: #5113 โ see knowledge-base/project/learnings/test-failures/2026-06-10-parallel-load-flake-two-mechanisms-and-vacuous-absence-waits.md.
-
An intermittent absence-wait that times out at the FULL (explicit) timeout is a component/state RACE, not a timeout-floor problem โ raising the timeout cannot fix it. Discriminator: if the failing vi.waitFor site already carries an explicit { timeout }, the floor is irrelevant; trace the component's effect ordering. A passive effect that resets state on EVERY render where a condition holds (if (cond) setX(false)) rather than on a prevโcurr transition races any user action that should win (React runs passive effects AFTER commit, so it can land after the click and undo it) โ gate such effects on the transition via a prevValue ref. Why: #5796 โ see knowledge-base/project/learnings/test-failures/2026-06-30-vi-waitfor-floor-vs-component-rearm-race.md.
-
When testing a fallback ladder or mode option (primary-then-degrade, retry-then-cache, mergeMode directโarm-auto-merge), assert the FIRST rung was attempted (the primary call fired), not just the fallback's effect โ an effect-only assertion passes identically against an option-ignoring implementation whose default path yields the same end state. Why: PR #5133 โ two mergeMode-direct fallback tests passed against the pre-#5111 helper; see knowledge-base/project/learnings/2026-06-11-pipeline-consolidation-behavior-preserving-migration-traps.md.
-
When adding sessionStorage usage to React components, ensure the component's test file includes sessionStorage.clear() in its beforeEach block. Shared jsdom environments leak sessionStorage between tests, causing ordering-dependent failures.
-
When adding a React-context-dependent hook (useTheme, useRouter, any provider-gated hook) OR a new provider import to a SHARED component, grep test/ for every file that renders that component DIRECTLY (not via a vi.mock of its module) and add the provider stub in the SAME commit. tsc and the component's own test pass; sibling direct-render tests fail at RUNTIME with <hook> must be used inside <Provider>. Why: PR #5217 โ C4Canvas gained useTheme(); c4-fullscreen.test.tsx (the only direct <C4Canvas> renderer) broke 8 tests until stubs for theme-provider + @mantine/core were added. See knowledge-base/project/learnings/2026-06-12-likec4-mantine-color-scheme-seam-and-vendored-theme-preservation.md.
-
To reproduce a provider's SSR-hydration "no-bootstrap" state in jsdom (lazy useState initializer landed on a server fallback like "system" WHILE durable storage holds the real value AND the DOM attribute is absent), do NOT use a Storage.prototype.getItem call-count spy โ it bleeds across tests in the shared jsdom worker (passes in isolation, fails in-suite) and a leftover DOM attribute pollutes later inits. Instead use REAL localStorage (empty at init) and write the stored value from inside the matchMedia.matches getter (fires during the resolved-state initializer โ after both init storage reads, before the first-mount effect); scrub the attribute + clear storage inside the mount helper and cleanup() in afterEach. Pair it with a precondition self-check (if (!released) throw) so a future init refactor that stops touching matchMedia fails as a clear FIXTURE error, not a phantom SUT regression. A naive client-only mount masks the bug (initializer reaches the durable value directly โ vacuous green). Why: PR #5312 โ see knowledge-base/project/learnings/test-failures/2026-06-15-ssr-hydration-no-bootstrap-theme-test-gate.md.
-
When asserting on vi.getTimerCount(), remember that vi.useFakeTimers() mocks every timer-like API by default โ including requestAnimationFrame, setImmediate, queueMicrotask, requestIdleCallback. The count is a SUM across all fake timer types, not just setTimeout. Prefer stability assertions (count before N extra calls === count after) over magnitude assertions (count === 1) so refactors that add a well-behaved rAF or microtask don't falsely read as leaks. See knowledge-base/project/learnings/test-failures/2026-04-17-vitest-getTimerCount-counts-requestAnimationFrame.md.
-
When a component exports an interface that a test harness consumes (e.g., ChatInputQuoteHandle), have the test import it via type X = ExportedInterface โ never shadow with a local duplicate. Duplicate interfaces silently drift when the exported type gains a method; the tsc --noEmit failure surfaces only at build time.
-
When adding a new npm dependency, check the installed major version (node -e "console.log(require('<pkg>/package.json').version)") and read the type definitions before using API from docs or training data. Library APIs change across major versions (e.g., react-resizable-panels v4 uses Group/Separator/orientation/useDefaultLayout, not v2's PanelGroup/PanelResizeHandle/direction/autoSaveId).
-
For sizing APIs from third-party libraries, always pass explicit units as strings (e.g., "18%", "100px", "1rem") rather than bare numbers. Docstrings may claim a default unit but runtime parsers often treat numbers as pixels. Why: react-resizable-panels v4 doc said "Percentage of the parent Group (0..100)" for numeric sizes, but the runtime treated 18 as 18px, producing a ~18px-wide sidebar in production. Explicit units make intent visible at the call site and survive library version upgrades.
Test environment setup: If the project's test runner cannot run the type of test needed (e.g., React component tests require jsdom but vitest is configured for node), set up the test environment BEFORE starting the task. This is part of RED โ the test infrastructure must exist for the test to fail properly.
- When configuring bun preload scripts that register DOM globals (e.g., happy-dom), use dynamic
await import() for all subsequent dependencies โ static ES imports are hoisted before any imperative code, causing libraries like @testing-library/react to initialize without DOM globals. See knowledge-base/project/learnings/test-failures/2026-04-03-bun-test-dom-preload-execution-order.md.
- Never write a literal
*/ inside a /* โฆ */ / /** โฆ */ block comment โ it closes the comment early. The trap is documenting a regex that ends in */ (/--[^\n]*/g, foo/**/*): esbuild/tsc then parses the trailing prose as code and reports Expected ";" but found <token> at a line deep inside the docstring (a red herring โ the real cause is the stray */ upstream). Describe the regex in prose or use a // line comment; grep -nF '*/' <file> after authoring confirms every hit is real code. Why: #5920 โ a */g in a JSDoc comment broke collection of byok-rpc-body-markers.test.ts. See knowledge-base/project/learnings/build-errors/2026-07-03-jsdoc-block-comment-closed-early-by-regex-star-slash.md.
- When a test file calls a SUT that lazy-imports a heavy module (
pdfjs-dist, sharp, puppeteer, playwright, @xenova/transformers, onnxruntime), pre-warm the module in beforeAll(async () => { await import("<module>"); }, 30_000). The cold-start cost (~5-10s on CI runners) otherwise lands on the first it() and blows the default 5s vitest timeout โ the second test in the same file runs at warm ~9ms because subsequent calls hit the module cache. Cheapest detection: git grep -lE '(pdfjs-dist|sharp|puppeteer|playwright|@xenova/transformers|onnxruntime)' -- '*.test.ts' and check for sibling beforeAll. Why: PR #3681 pdf-text-extract.test.ts cold-start flake (7s vs 9ms, #3687).
- When uploading files via Playwright MCP, save files to repo-accessible paths (not
/tmp/). Playwright MCP restricts file access to the repo root. When Google Search Console offers Cloudflare auto-verification, prefer "Any DNS provider" manual flow โ the popup OAuth flow opens an external tab that crashes the Playwright browser context.
- Vendor-token extraction via Playwright MUST use
browser_evaluate(filename: ...) from the FIRST attempt โ the return value otherwise enters the conversation transcript and the token is leaked even after revocation. AND the filename parameter JSON-encodes the result (surrounding quotes), so the canonical pipe is python3 -c "import sys,json; sys.stdout.write(json.loads(open('<path>').read()))" | doppler secrets set <KEY> --no-interactive. Validate via the vendor's API (HTTP 200 + length check) before shredding the file โ some vendors silently tolerate quoted tokens via Authorization: Bearer "abc", but Terraform's HCL parser does not. For โโโ-masked UI tokens (Doppler personal tokens), click the in-page copy button via browser_evaluate, then xclip -selection clipboard -o > <path>; clear with xclip -i </dev/null. Doppler TF var storage convention: drop the TF_VAR_ prefix from the secret name โ --name-transformer tf-var ADDS the prefix at injection time (DOPPLER_TOKEN_TF โ TF_VAR_doppler_token_tf; storing the already-prefixed TF_VAR_DOPPLER_TOKEN_TF produces TF_VAR_tf_var_doppler_token_tf). See 2026-03-21-doppler-tf-var-naming-alignment.md. Why: PR #3973 (#3960) โ full pattern + recovery flow at 2026-05-18-vendor-token-mint-and-oci-image-content-carrier-patterns.md.
- After any
Write whose hook output emits a warning (security, style, rule), immediately Read the file to verify the full content landed. PreToolUse hooks that print error output but return non-blocking status can still cause partial writes โ detecting this only when tests fail wastes a debug round. See knowledge-base/project/learnings/2026-04-15-kb-share-binary-files-lifecycle.md.
- When adding source-reading regex tests (
readFileSync(path) + expect(src).toMatch(...)) as a negative-space regression gate after an extraction, put them in a standalone *.test.ts file โ never add them to an existing test file that already mocks node:fs or node:path. The existing vi.mock("node:fs", ...) factory likely omits readFileSync, and the new test will fail at collection with "No readFileSync export is defined" before any assertion runs. Also trim the gate to only the assertion that cannot be expressed behaviorally โ usually the negative "symbol-not-present" check. Positive assertions (import regex, await-call regex) duplicate coverage that mock-based behavioral tests already provide and are brittle to barrel re-exports, aliases, and whitespace. See knowledge-base/project/learnings/best-practices/2026-04-17-regex-on-source-delegation-tests-trim-to-negative-space.md.
- NARROWING THE SCOPE IS NOT THE FIX โ ANCHOR ON SYNTAX. The bash body-grep rule below generalizes to EVERY source-reading assertion (
readFileSync + toContain/toMatch over .tf/.yml/.ts), and the obvious correction โ slice a narrower region โ FAILS when a file puts explanatory comments INSIDE the construct: there is then no scope that holds the config but no prose. Anchor on something a comment cannot produce (^\s*key\s*= โ a comment line starts with #; a call shape Fn\(\s*arg), never a bare word. Treat every toContain of a token that also appears in a nearby comment as guilty until mutation-tested, and give every slice helper an explicit lower bound plus an indexOf === -1 guard (slice(-1) yields the last character, so .not.toMatch() against it always passes). Why: #6456 shipped FOUR โ /value\s*=\s*2/ matched its own "WHY value = 2 AND NOT 3" comment; toContain("IssueOwners") matched an in-body comment while the entire actions_v2 block was deleted (the rule then paged nobody โ the outcome the test was named for) and stayed 10/10 green; a boundless scopeResource swallowed the next resource's comment so a GROUPING-anchor check was satisfied by the pointer to the paragraph it was meant to find. All four were read-and-believed; only mutation caught them. Count failures from the runner's summary line after stripping ANSI โ grep -cE '^\s+ร' always returns 0. See knowledge-base/project/learnings/2026-07-15-narrowing-is-not-anchoring-and-a-documented-class-recurred-four-times-in-one-pr.md.
- When adding a static-grep assertion over a SCRIPT BODY (a
.sh/.test.sh body-grep gate, an AC grep -n โฆ | head -1 order check), anchor it on the syntactic write/call construct (rest/v1/<table>, a function-call shape) โ NEVER a bare token (<table>, <flag-name>) that the same file also names in a COMMENT or header-inventory. A body-grep sees comments too, so the moment a task requires both a "must / must-not contain X" assertion AND documenting X in a comment, they collide: a negative ! grep -qE 'X' false-FAILs on the explanatory comment, and a grep -n X | head -1 order check returns the comment line, not the code. Reword forbidden-literal comments to drop the literal. Same class as the source-reading-regex rule above, for bash. Why: PR for #5501 โ seed-live-verify-user.sh's user_session_state upsert: the test's ! grep '/rpc/set_current_workspace_id' tripped on a comment naming the RPC, and the AC3 bare grep user_session_state matched the new header-inventory line. See knowledge-base/project/learnings/test-failures/2026-06-17-grep-assertion-over-script-body-false-matches-own-comments.md.