| name | tdd-workflow |
| description | Drives RED → GREEN → IMPROVE in this repo's actual runners — pytest via `rtk proxy`, vitest in frontend/, Playwright for flows — and requires the RED state to be observed, not assumed. Use when writing a new feature, fixing a bug, or refactoring anything under the Python API, frontend/, or the theme's JS. Do NOT use for retrofitting tests onto code that already shipped green (that is a coverage-gap task), and do NOT use it to decide whether an existing suite's result is trustworthy — that is verification-loop. |
| origin | ECC |
Test-Driven Development
TDD's value is not the test file. It is the observed RED: the one moment you have
evidence the test can fail. Skip it and you have written an assertion nobody has ever
seen return "no" — decoration with a green checkmark.
This repo has paid for that. A fraud scorer shipped with a fully green suite; adversarial
review then found 16 real defects (bug-150). The tests proved each rule fires. Nobody
had ever watched them fail for the right reason.
When to use
Use it when:
- adding a feature or endpoint under
agents/, api/, services/, skyyrose/
- fixing a bug — the regression test is written first and must reproduce the bug
- refactoring: the suite is the safety net, so it must be green before you touch code
- changing a gate, guard, or validator — those need an explicit fail-closed test
Do not use it:
- to backfill tests onto already-shipped code. That is a coverage-gap task with a
different shape (read the code, find the untested branch); there is no RED to observe.
- to judge whether someone else's green suite means anything — that is
verification-loop plus
adversarial-verification.
- for pure config/docs edits with no behaviour.
Inputs
| Required | How to get it | If absent |
|---|
| A green baseline before you start | rtk proxy pytest tests/ -q → 6546 passed, 0 failed | Stop. On a red baseline you cannot tell your break from the existing one. Fix or quarantine first, and name who owns the pre-existing failure. |
| The right runner for the surface | Python → rtk proxy pytest · frontend/ → npm test (vitest) · flows → npx playwright test | Stop. A bare pytest in this repo can print "no tests collected" and read as success. |
| A stated behaviour, not a stated function | "invalid SKU raises ValueError", not "test resolve_image" | Stop and restate. A test named after a function tests whatever the function does, including the bug. |
A writable per-test temp dir (tmp_path) | pytest fixture | Never write to tracked files. bug-153: a test injected tokens into the real shared design-tokens.css and failed only under the full suite. |
For frontend/: the suite must be importable under vitest | vitest.config.ts uses an explicit include (lib/wp/**, tests/**) | Stop. A suite outside include is silently never run — and a skipped security test is indistinguishable from a passing one. |
Procedure
- Record the baseline. Run the suite for the surface you are about to touch and note
the exact counts. You will diff against these, not against "green".
- Write the test first, naming the behaviour and the failure it forbids. One
behaviour per test; arrange-act-assert.
- Run it and read the failure text. RED must be the right red — an assertion
about your behaviour, not
ImportError, not a fixture typo. A test that errors before
reaching its assertion has not demonstrated anything.
- Write the minimum implementation to turn it green. No speculative branches, no
abstraction for a single call site.
- Re-run the single test, then the surface's full suite. A test that passes alone
and fails in the suite is shared-state pollution (bug-231, ×5), not flake.
- Prove the test can still fail — extract the pre-change file with
git archive HEAD <path> | tar -x -C <scratch> and run your new test against that
pristine tree. It must fail there. Never git stash: the stash stack is shared
across worktrees.
- Check coverage for the module you touched, not repo-wide — the repo number moves
too slowly to show your delta.
- Refactor with the suite green, re-running after each step.
- Never weaken a test to make it pass. Loosening an assertion, adding
xfail, or
widening a tolerance to reach green converts a real failure into a silent one.
A SKIP is not a PASS. This worktree is sparse, so asset-dependent tests skip by
design (bug-257). Report them as "not executed here; closed by CI / the full checkout" —
never inside a passing count.
Verification
1 — The new test is green in isolation and in the suite.
rtk proxy pytest tests/mcp/test_http_mount.py -q && rtk proxy pytest tests/ -q
PASS: the single file passes, and the full run reports N passed with 0 failed
and N >= 6546 (the baseline). A drop in total count means you deleted or de-collected
tests. [test]
2 — RED was real: the test fails against the pristine tree. This is the
prove-it-can-fail rule and the attribution rule in one move.
mkdir -p /tmp/tdd-pristine && git archive HEAD mcp_tools/ | tar -x -C /tmp/tdd-pristine
rtk proxy pytest tests/mcp/test_http_mount.py -q --rootdir=/tmp/tdd-pristine
PASS-OF-THE-PROOF: this run fails. If your new test passes against code that
predates your change, it is not testing your change. [test]
3 — Coverage on the module you touched.
rtk proxy pytest tests/mcp/ -q --cov=mcp_tools --cov-report=term-missing
PASS: the touched module reports ≥ 85% and the Missing column contains no line
from your new code. [repro]
4 — Frontend suites are actually collected. vitest's explicit include makes silent
non-collection the default failure here.
cd frontend && npm test 2>&1 | tail -5
PASS: the summary names your spec file. If your file is absent from the output, it
was never run — fix vitest.config.ts include or move the suite. [repro]
5 — Flows: the Playwright spec is registered.
cd frontend && ./node_modules/.bin/playwright test --list | tail -1
PASS: the total count went up by your test count. testDir is ./tests/e2e;
a spec written into the legacy frontend/e2e/ is never executed and shows here as an
unchanged total. [repro]
A run that errored, timed out, or hit a session limit is not a pass. Its empty
failure list is an artifact — re-run it by hand (bug-230).
Worked example
Regression test for bug-211 (MCP bearer token compared with !=; fail-open when
MCP_SERVICE_TOKEN is unset). Real commands in this worktree, 2026-07-29.
Baseline for the surface:
$ rtk proxy pytest tests/mcp/test_http_mount.py -q
...... [100%]
[repro] — 6 passed. That is the number the next run is diffed against.
The behaviour under test, stated before the code: "an Authorization header that shares a
prefix with the real token is rejected, and the comparison is constant-time" — which is
why the implementation reads:
$ grep -n "compare_digest" mcp_tools/http_mount.py
88: if not hmac.compare_digest(provided, f"Bearer {token}")
[repo]. The RED proof is the interesting half: run the same test file against the
pre-fix tree via git archive, and it must fail on the != comparison. Running it only
against the fixed tree — where it passes — would have demonstrated nothing about the
test.
The contrast case, same session, same runner:
$ rtk proxy pytest tests/test_asset_manifest.py -q
SKIPPED [1] tests/test_asset_manifest.py:42: sparse worktree deliberately excludes
assets/products; this gate runs in full checkouts and CI
SKIPPED [1] tests/test_asset_manifest.py:51: ... (×4 total)
[repro] — 4 skipped, 0 passed. Reported as "asset-manifest integrity did not run in
this worktree". Rolling those four into a "tests pass" line is the exact bug-257 defect.
Failure modes
| Symptom | What is really happening | Bug |
|---|
| Suite fully green, adversarial review finds many real defects | Tests proved each rule fires; none exercised calibration, boundaries, or the never-raises guard. Green ≠ correct. | bug-150 |
| Test passes alone, fails in the full suite | Shared state — module globals, a real tracked file written in place, ordering under asyncio_mode=auto. Use tmp_path / monkeypatch. | bug-231 (×5), bug-153 |
| Test skipped everywhere, counted as passing | A SKIP is not a PASS. Name who closes it (CI / full checkout). | bug-257 |
pytest says "no tests collected", read as success | Bare pytest misreports in this repo. Always rtk proxy pytest. | — |
| New frontend spec never appears in output | Outside vitest.config.ts's explicit include, or written into the legacy frontend/e2e/ that Playwright's testDir excludes. | — |
| Test made to pass by loosening its assertion | The failure was real and is now silent. Fix the cause, never the assertion. | — |
| RED never observed; test written after the code | No evidence the assertion can fail. Re-derive via git archive against the pristine tree. | — |
git stash used to get a clean tree for the RED check | The stash stack is shared across worktrees; you can pop another session's work. | — |
| Multiprocess/fork test SIGSEGVs on macOS | Darwin fork safety — no_proxy='*' must be pre-set and the resource tracker pre-spawned. | bug-263 (×7) |
| Coverage measured repo-wide, shows "no change" | Your delta is invisible in a 6546-test total. Measure --cov=<touched module>. | — |
