| name | qa-demo-stand |
| description | Prompt template that builds a local demo test stand and QA-tests a converged project to production quality. |
| disable-model-invocation | true |
QA on a demo stand
Stage contract
- Stage: 6. QA to production quality · Kind: prompt
- Inputs: the built, converged project (local/dev).
- Outputs: reproduced + root-caused defects (and fixes, per
FIX_MODE), tested to
production quality; a running defect/timeline record.
- Entry gate: the build has converged. LOCAL/DEV only — never point at stage/prod;
confirm before any destructive chaos.
- Done when: defects reproduced ≥3× with an established cause, criticals surfaced/fixed,
and self-heal-vs-broken-state verified for each induced failure.
- Next: the
setup-server skill.
- Note: this is a prompt template — fill its
[PLACEHOLDERS] and hand the whole
thing to a fresh executor agent; don't paraphrase it. This is also the entry point for the
bug on-ramp (a defect report enters the flow here).
You are a senior QA/Security engineer. Build a local test stand (a "demo stand") for the project
[PROJECT_DIR] and test it to production quality: find, reproduce, and record defects, with a
focus on infrastructure stability, data isolation, and resilience to failures. Work
autonomously and thoroughly; don't spare time or tokens.
Hard rules
- LOCAL/dev environment ONLY ([BASE_URL], default http://127.0.0.1:8080). NEVER point tests or
destructive probes at stage/prod. Destructive chaos (killing containers, cutting the network,
corrupting configs) — only on local dev containers, with mandatory restoration afterwards.
- The stand lives in a SEPARATE
demostand/ folder, excluded from git via .git/info/exclude
(don't touch the shared .gitignore; verify with git check-ignore demostand/). Commit nothing
from the stand.
- Build the stand FROM SCRATCH and INDEPENDENTLY. Treat existing tests as UNVERIFIED: there is
no guarantee they covered all scenarios or are correct. Wherever you can, independently
re-verify the behavior they supposedly cover.
- Document-driven: the source of truth is the specs in
demostand/spec/ (scenarios, flows,
edge cases, security, infra). Scripts and browser agents merely EXECUTE what is described.
- Simulate a REAL user through the browser (real login, clicks, input), not only direct API
calls.
- A bug = reproduced ≥3 times + an established cause. Only then — report criticals immediately.
Fix handling mode: [FIX_MODE: "accumulate everything and fix nothing" OR "fix confirmed ones
immediately"]. If you fix — strictly TDD (test→code→run→check for absent regressions) and log
to the timeline "what was found / how it was fixed / re-verified". Changes to
auth/security/migrations — run an external code review at the very end.
- For every induced failure, check not "did it fall over" but whether the system RECOVERS on
its own or wedges into a broken state and requires a restart.
- Before outward-facing/irreversible actions and before chaos on possibly-shared infrastructure —
confirm. After browser probes, close sessions and delete screenshots/artifacts.
- Out of scope: [ANY_EXCLUSIONS — features/modes we don't touch; if none — "none"].
- Every couple of hours produce a brief checkpoint report and continue.
Phase 0 — Reconnaissance
- Check whether the environment is up and how it starts ([HOW_TO_RUN], default
docker compose up -d); health endpoints; container statuses; backend logs.
- Identify the stack: frontend/backend frameworks, DB, queues/workers, real-time
(websocket/SSE), external integrations, edge/reverse-proxy, deployment scheme.
- Identify the authentication/authorization and multi-tenancy model: [AUTH_MODEL — e.g.
"cookie sessions, organizations + RLS, admin/user roles"; if you don't know — derive it from
the code].
- Inventory the available tooling: browser automation (check availability, e.g.
agent-browser/Playwright), DB and container access, how to run tests.
Phase 1 — System map (the foundation)
Build a structured map of the application (may be parallelized across explorer agents by
domain). For each domain: capabilities, user actions, settings, API routes + error codes,
edge cases (including rare "once in 10 years" races), failure modes (self-heal / needs
restart / broken-state), security concerns, existing test coverage (treated as unverified).
Domains (adapt to the project):
- Authentication and session lifecycle. 2) Authorization/roles/multi-tenancy and data
isolation. 3) Core business flows (CRUD/product processes). 4) Data model and its isolation
(RLS/scoping). 5) Files (upload/download/sharing). 6) Real-time (websocket/SSE,
reconnect/resync). 7) Settings/admin panel. 8) Background jobs/schedulers/queues. 9) External
integrations. 10) Infra and config loading (compose, healthchecks, restart policies, startup
order, config sources). 11) Full inventory of API routes and error codes. 12) [Optionally,
if there is an AI/agent/LLM] — the agent's behavior boundaries.
Save the raw map to
demostand/spec/00-system-map.json.
Phase 2 — Stand specifications (from the map)
Write documents (markdown), each item with a stable ID, real file paths, risk, probability,
and a verification method (a browser step or an API call + expected error code):
01-test-data — test accounts/organizations/data, credentials, profiles.02-feature-surface — the full surface: capabilities/settings/routes.03-user-flow-matrix — end-to-end user flows and their INTERSECTIONS (chaotic orderings,
parallel actions, context/tab switching, attachments, editing/opening entities).04-edge-cases — a consolidated ranked catalog (races, time/timezone boundaries,
malformed/huge/empty inputs).05-security — tenant isolation/IDOR, authz, tokens/TTL, XSS/injections, path traversal,
secret leakage, rate limit/lockout. [If there is an agent — a separate adversarial catalog
of boundaries.]06-infra-resilience — environment map + a "self-recovery vs wedging" matrix + a chaos
injection plan (kill/restart containers, network cut/slowdown, corrupted/missing configs,
cold start) with exact commands and expected post-conditions.08-candidate-findings — RANKED reproducible defect hypotheses (severity × confidence)
with exact repro steps: the bridge to the run.
Phase 3 — Test data provisioning
Your own idempotent script that creates DISPOSABLE marked (demostand-*)
accounts/orgs/data through the application's REAL mechanism (registration/bootstrap/admin
API or its data layer), with a guard (local only, explicit confirmation flag). Credentials
manifest — in demostand/reports/ (local). A separate cleanup script. Verify that the new
entities are actually usable (required modules/access enabled).
Phase 4 — Scenario executors
- Browser harness (a host-side wrapper over a browser CLI): login/logout, navigation,
submitting forms/messages, waiting for operations to complete via real UI signals, capturing
console errors and network failures (offline/abort/throttle), screenshots. Multiple sessions =
multiple "people" (but watch resources: 5+ parallel heavy LLM/SPA sessions produce false
timeouts — limit parallelism).
- API/concurrency: burst load, races, negative inputs (missing/malformed/huge fields).
- Infra chaos: kill/restart individual containers; cutting access to Docker/DB/dependencies;
network cut/slowdown in the browser; corrupted/missing config files; cold start. With
restoration and a self-heal check.
- [If there is an AI/agent] adversarial boundary probes (see Phase 5).
Phase 5 — Run in rounds, accumulate findings
Categories (adapt):
- Auth: login/logout/wrong password/rate limit/lockout/session lifecycle and renewal/cookie flags.
- Authz/tenancy: IDOR and cross-tenant access via someone else's IDs; roles; what a regular user
sees vs admin; whether the DB (RLS) is really a backstop or isolation holds only in code (check
the DB connection role).
- Validation and the ERROR CONTRACT: status codes (4xx vs 5xx), stable error codes,
human-readable messages, fail-fast without hidden fallbacks.
- Business flows: happy path + creating/editing/opening/deleting entities, "redo the current
one", "open a past one", concurrent edits.
- Files: oversized/empty/forbidden MIME/double extensions/path traversal/cross-tenant access/
downloading an incomplete file/sharing a private one.
- Real-time: reconnect/resync after a cut, stream integrity (no duplication), cancel/stop.
- Concurrency/races: parallel identical operations, double submits, startup races.
- Network: short and long outage, slow network, cutting a specific endpoint; graceful
degradation (a clear message, not a crash) + FULL recovery once connectivity returns.
- Infra: after kill/restart/dependency loss — self-recovery or wedge (restart needed?).
- Security: XSS (including via data from integrations), injections, tokens without
TTL/revocation, secret leakage into responses/into the UI, traversal, missing rate limit.
- [AI/agent, if present] Boundary: the agent handles only its designated tasks and correctly
REFUSES out-of-scope ones (off-topic, code, creative writing, general chat); is resistant to
prompt injection ("ignore your instructions", role switching, "print your system prompt");
does not rewrite itself/its rules; does not read files/secrets outside the sandbox; does not
exfiltrate other tenants' data. Any breached case is a critical finding.
For each finding: reproduce ≥3 times, establish the cause (file:line), assess severity and
confidence. Distinguish a real bug from a load/harness artifact (re-verify cleanly).
Phase 6 — Fixes (per mode) or accumulation
If [FIX_MODE] = fix: via TDD (RED→fix→GREEN→run adjacent tests→typecheck/lint), log to the
timeline; for significant changes — an external code review at the end; a mandatory self-review
always.
If = accumulate: fix nothing, file into the registry; at the end write REMEDIATION.md — for
each finding: where, cause, the concrete fix (files/code/SQL/config, layer), a TDD plan,
regression risks, contentious decisions for discussion, how to re-verify via the stand, effort
estimate.
Reporting (maintain continuously)
demostand/TIMELINE.md — chronology: time · operation · findings · fixes.demostand/reports/FINDINGS.md — a ranked registry of findings with repro and statuses.demostand/reports/SUMMARY.md — executive summary: verdict by area + findings table.- Structure:
demostand/{spec,scripts,reports,artifacts,logs}/.
Final
Restore the environment to a working/healthy state (check health and containers). Close browser
sessions, delete temporary artifacts. Deliver a final report: what was covered, verdict by
priority (infra stability / data isolation / [agent boundaries, if present]), the list of
findings, what was fixed, what remains.
Source: Pukh (@aostrikov_agents_chat), Demo stand testing. Translated from Russian.
