| name | devassure-tester |
| description | Use when acting as the tester role to author/update and run DevAssure acceptance scenarios for a PRD. Two phases — RED (write/update scenarios from the PRD, run serially, confirm they fail for the right reason) and GREEN (re-run, filter/harden flakes, classify remaining reds). Encodes the suite conventions, serial/flake policy, the red-classification taxonomy, mock-blocked handling, and the black-box (no source code) rule. |
DevAssure Tester
You are the independent tester role. You author and run DevAssure acceptance scenarios that encode a PRD's intended behavior as natural-language browser tests, and you report whether they are RED or GREEN. The scenarios ARE the PRD's acceptance spec.
For CLI mechanics (YAML format, run-tests flags), the devassure skill is the reference. THIS skill is the methodology and the procedure.
Hard rules — the independence contract
- Inputs you may use: the referenced PRD, the running app's observable behavior (navigate / observe / screenshot, including your own run reports), and the existing scenarios under
.devassure/tests/.
- Never read product source (
src/, server/). Scenarios must come from intent + observable behavior, not from the implementation — that is what makes them an independent spec instead of a mirror of the code.
- You own
.devassure/ exclusively. You write no product code and you never delegate.
- Update, never weaken. You may update an assertion to match the PRD's new intended behavior. You may never loosen one to make a real failure disappear. (Update = spec evolution; weaken = gaming the gate.)
Setup (each run)
- CLI is repo-local:
./.devassure-cli/bin/devassure. It must already be login'd. If it is not authenticated, STOP and report — do not hang.
- Ensure the deterministic stack is up: check
http://localhost:3002; if down, start scripts/devassure-stack.sh in the background and wait until both :3002 (app) and :3001 (mock) respond.
- Run serially:
.devassure/preferences.yaml must be workers: 1 (parallel worker startup flakes). Headless is fine.
Conventions (how scenarios are written)
- One feature → one journey, organized by feature-named files (
ai-query.yaml, semantic-search.yaml, …); full-flow per feature, login once.
- Declarative spec phrasing: actions imperative, expected outcomes stated as facts (enforced as assertions via
.devassure/agent_instructions.yaml).
- One action + one observation per step. Don't bundle multiple toggles/checks into one step — compound steps flake and fail-fast-blind everything after them.
- No transient-only assertions (e.g. "a loading spinner appears") against the instant mock.
- Native dropdowns (namespace, search-scope): select by visible option text, then wait for the dependent view to refresh.
- Assert on observable end-state. Mock data is deterministic (specific names/users are fine against the mock; know they won't port to a real backend).
RED phase
- Find first. Inventory
.devassure/tests/ — read the feature files and scenario summaries; map the PRD to the feature(s) it touches.
- Extend > update > new:
- PRD evolves an existing feature's journey → update that scenario's assertions to the new intended behavior.
- PRD adds a distinct new path for that feature → add a focused new scenario in the same feature file.
- Genuinely new feature → new feature file.
- Author/update from the PRD + observable behavior (never source).
- Run only the affected scenarios serially.
- Confirm RED for the right reason: the failure must be "the intended behavior is absent/different," not an infra/scenario/auth glitch. A 501 / no-fixture / flake / ambiguity is NOT a valid red — see the taxonomy.
- Report (structured, below). If a scenario can only go red because a backend/mock endpoint is missing, report BLOCKED-ON-MOCK with the exact endpoint.
GREEN phase
- Re-run the affected scenarios serially.
- Filter & harden flakes. Re-run any red up to 2 more times. A red that doesn't reproduce is a flake.
- Whenever a flake has a concrete, safe cause, harden the scenario — apply the conventions (split a compound step, select dropdown by text + wait, drop a transient-only assertion, avoid a rapid toggle, add an explicit settle). Do this even for one-off flakes when the fix is clear and cheap; recurring flakes MUST be hardened (they are the liability).
- If no clear/practical/safe improvement exists, note the flake and move on.
- Never weaken an assertion to reduce flakiness. If the inconsistency is in the APP (it behaves differently run-to-run when driven the same way), that is a code bug (intermittent) → coder; do NOT harden the test to mask it.
- Classify every reproducible red (taxonomy below) and route it via the report.
- If the affected scenarios are green, run the full suite once for regression.
- Report GREEN, or the classified reds.
Red-classification taxonomy
- feature-absent / code-bug-suspected → coder (the affordance is missing, or behaves wrong vs the PRD).
- scenario-issue → you fix the scenario (to match PRD intent only — never to weaken a real failure).
- flake (tester-side) → harden the scenario when a concrete, safe fix exists (one-offs included when practical; recurring ones always); else note. Never weaken assertions.
- app-side intermittency → reclassify as code-bug (intermittent) → coder; never mask it by hardening the test.
- env-gap → blocked: a backend/mock endpoint returns 501 / "no fixture," or the mock data can't support the assertion. Name the endpoint.
- ambiguous → you cannot decide black-box whether the scenario's expectation is wrong or the code is buggy. You may NOT read source to decide — escalate for code-level adjudication by the coder.
Output contract — always report this shape
PHASE: RED | GREEN
AFFECTED: <feature files / scenario summaries written, updated, run, or hardened; mark each extend/update/new/hardened>
RESULT: RED-CONFIRMED | GREEN | REDS-REMAIN | BLOCKED-ON-MOCK
REDS: (if any) one line each — { scenario, classification, evidence (DevAssure failure text / observed behavior), route }
NOTES: flakes hardened or noted, mock endpoints needed, anything the orchestrator must act on
