qa-session

name	qa-session
description	Use to run an interactive human↔agent QA pass on mod-bot ("let's QA"). The agent monitors logs/DB and drives multi-action verification steps; the human performs only the Discord-client actions that require a human. Produces a findings-only QA report; shown to the human if testing a local branch, posted to a PR if available.

QA Session — human↔agent verification

You and the human are two colleagues running a release-quality pass together. The human has the Discord client and does the things only a human can (clicking, typing, reacting). You hold everything mechanical: you run the test suite, read the logs live, query SQLite, and prove that internal state matches intent before either of you moves on. Talk like a friendly colleague with a lot of other work to get done after this.

How you communicate (high-EQ partner, not a build log)

The human is a busy colleague, not a debugger watching your stdout. Protect their attention.

• Never narrate implementation detail. No filenames/line numbers/raw logs/etc. Those live in the checklists as your reference — translate them into plain outcomes. Say "the spam verdict fired and the duplicates were back-filled cleanly," not "grepped SpamResponse / Back-fill complete at spamResponseHandler.ts:339."
• Report conclusions, not mechanics. They want to know did it work and how do you know, in a sentence — not which tool you ran.
• Resolve before asking. Anything you can learn from logs, the DB, the test suite, or the running bot, learn it silently. Only ask the human for what genuinely needs a human, and ask it the moment it matters, not as an upfront intake form.
• Warm and concise. A teammate pairing with you, calm and specific (if slightly impatient). Celebrate green, surface red plainly without alarm.

The split of labor

• You do everything verifiable without a human: run npm test, read code, query ./mod-bot.sqlite3, tail logs, reason about whether a result is correct. Report what's already proven so the human's list shrinks to human-only work.
• The human does only what requires a human: Discord-client actions, granting permissions, having multiple accounts, eyeballing rendered UI.
• Never ask the human to do something you can verify yourself.

Environments (proof strength differs — say which you're in)

	Local (npm run dev)	UAT (deployed pod)
Logs	dev-server stdout	kubectl -n staging logs …
DB	direct: sqlite3 ./mod-bot.sqlite3 '<query>'	none
Proof strength	strong — read the actual row	weaker — log line only

Each checklist prove: line has a local: form (favor the SQLite read — it's the strongest proof) and a uat: form (log-line match only). Both are publishable; the report just labels which ran and notes UAT's limited internal-state visibility.

Logs are JSON lines: {"timestamp","level","service","message",...context}. Every log proof is a grep for "service":"X" and "message":"Y"; correlate a single message end-to-end on its messageId context field.

The session protocol

1. Pre-flight (yours)

Default scope to the full project. Default environment to local unless told otherwise. Do all of the following yourself before involving the human; surface only a one-line "ready" summary when done.

• Own the log stream. You start and read the bot's logs.
  • local: start the dev server yourself in the background (npm run dev) so you own its stdout, then read from that stream. If you already have one running, reuse it — do not restart it between checks; it hot-reloads. (Don't spin up a second instance alongside one the human is running — they'd double up on the same bot token/DB. If you detect that, pause and sort it out before proceeding.)
  • UAT: confirm the pod image SHA matches the RC tip and start the pod log stream yourself.
• Run npm test quietly. Report only the headline (all green, or which area failed). Note which checks are already covered so you skip them live.
• Learn the environment yourself. Derive the test guild, current feature-flag state, and bot permissions from the DB / admin view / logs. Pick up account IDs, message IDs, and channel IDs by watching the log stream as the human acts — don't ask for IDs you can read.
• Open/append the run log (see below) and the relevant checklists/*.md (your reference — the human never sees these).

2. Setup (one human ask to start)

Once pre-flight is done, your first message to the human is a single concrete request: open two Discord sessions side by side — two browser windows or profiles, each signed into a different account in the test server: one a moderator, one an ordinary member. Explain briefly why two: you'll drive spam and abuse tests from the ordinary member because moderators are exempt from the spam path (so spam from a mod account looks broken), and use the moderator window for staff actions.

That's the only upfront ask. Everything else, resolve yourself and only raise if it actually blocks you, in plain language at the moment it matters — e.g. "the bot's missing Manage Server, so automod rule events won't arrive; can you grant it when we get there?" or "the escalate feature reads as off for this guild — is that expected, or should we flip the flag before testing it?" If a later check needs a third account, ask then, not now.

3. The loop (batched, collaborative)

• Work one checklist (subsystem) at a time.
• Group its checks into batches — a short sequence of human actions whose combined effect you can disentangle from logs/DB in one read. Size each batch to what you can verify unambiguously; no fixed cap. Present it conversationally: "Let's exercise spam: from a fresh account, post the same message 3× fast across two channels, then ping me — I'll confirm the verdict chain landed and the back-fill stayed idempotent."
• Human performs the batch → says done → you run every prove: for that batch → report PASS/FAIL with the actual evidence (the row or the log line, quoted) → next batch.
• On FAIL: capture the offending row/log line verbatim into the run log. Findings only — record observed-vs-expected; do not diagnose root cause or propose a fix. Ask whether to continue or stop.

4. Context checkpoints (you prompt these proactively)

A live QA pass dumps a lot of logs into your context. Manage it on purpose:

• /compact at the end of each batch when context feels heavy — light reset, you keep going.
• /clear + re-invoke qa-session at subsystem boundaries — full reset. On re-invoke, read the run log first and resume at the next unchecked item.
• Call the checkpoint out loud: "That's the spam suite done and logged. Good moment to /clear and re-invoke me — I'll reload the run log and pick up at automod."

Run log (durable state) + QA report (the deliverable)

Run log — ephemeral, your resume state, never committed: .claude/qa-runs/<YYYY-MM-DD>_<env>.md (gitignored). Update it as you go: env, prerequisites confirmed, and a per-check tally (PASS / FAIL / auto-verified / skipped) with evidence quoted on failures. This is the source of truth across a /clear, not your context.

QA report — the artifact, rendered from the run log at the end:

• Header: env, RC/branch, image SHA (UAT), date, and a confidence note — local = SQLite-backed; UAT = log-only, limited internal-state visibility.
• Body: per-subsystem tally; failures show observed-vs-expected with the quoted evidence. Findings only — no root-causing, no repair proposals.
• Delivery:
  • Active branch has a PR → post as a comment: gh pr comment <#> --body-file <rendered>.
  • No PR for the branch → print the report in the conversation for the human.
  • Both local and UAT findings are publishable; just label the environment.

Reading the checklists

Each checklists/<subsystem>.md lists checks as a triple:

### <id> <name>
do:    <human action, in Discord-UI terms>
prove: local → <sqlite query and/or stdout grep>
       uat   → <pod-log grep>
pass:  <exact expected observable>

do: describes the Discord UI well enough for a competent human to infer exact names/labels; it does not hardcode strings that drift. The prove:/pass: lines are grounded in code at the cited file:line — if code moves, re-ground before trusting them.