testing-skills

# Testing Skills Behavioral proof for **discipline skills** — those that enforce a rule an agent could rationalize past. Structure is not compliance. This proves the skill holds under pressure, deterministically. ## When to Use Use for a **discipline skill**: one whose value is that it runs even when the agent is tempted to skip it (mandatory reviews, quality gates, process steps). Signs: it says MUST/ALWAYS, lists "blocked rationalizations", or gates a phase. Do **not** use for technique or reference skills — ones that inform but enforce nothing. Those are covered by the structural scorecard; they have no skip to prove. ## The Iron Law **No discipline skill ships without a failing behavioral test first.** Applies to new skills *and* edits. When you catch a discipline skill about to ship or change without a RED→GREEN run, announce **`⚡ RED baseline first`** and run it. If you cannot point at that run, you are not done. ## The Cycle: RED → GREEN → REFACTOR - **RED** — run the scenario with the skill **withheld**. Record the agent's exact skip rationalizations, verbatim. The marker MUST be absent. If you never watched it fail, you don't know it prevents the failure. - **GREEN** — run with the skill present. The marker MUST appear. That marker, grepped from the transcript, is the entire verdict. No agent judges compliance. - **REFACTOR** — stack combined pressure (time + sunk-cost + exhaustion). Each new rationalization that drops the marker gets a counter; re-run until the marker holds at maximum pressure. See `references/pressure-scenarios.md`. ## The Four Requirements Every discipline skill MUST carry all four (enforced by `validate-discipline-skill.sh`): 1. **`compliance_marker`** — an exact string the *real* code path emits when followed. Not a test-only echo. 2. **`TESTS.md`** beside SKILL.md, latest `green_verdict: PASS`. 3. **`## Rationalization table`** — observed excuses + counters (`references/rationalization-table.md`). 4. **CSO-clean `description`** — triggering conditions only, no workflow summary (`references/cso.md`). ## Running the Test `develop/skill-tests/run-skill-test.sh <skill-dir> --both` dispatches the scenario with the skill withheld (RED) then present (GREEN). RED passes when the marker is absent; GREEN when present. Off-platform it exits 2 with a manual-mode instruction — a dev tool, never a runtime dependency. ## Failure Modes - **Tempted to have an agent judge "did it comply?"** — stop. The marker grep is the verdict; judgment reintroduces the non-determinism this removes. - **Marker real execution doesn't emit** — invalid; the test passes while production skips. Confirm it against the genuine code path. - **Imagined rationalization table** — derive rows from real RED transcripts.