red-team

name

red-team

description

Adversarial bug-hunting review of a PrairieLearn diff, PR, or worktree change — hunts REAL bugs (not style) in the seams where PL breaks: the TS⇄Python code-caller boundary, zygote serialization, SQL/Drizzle edge cases, assessment/grading state, socket event isolation, auth/CSRF, and migrations. Severity-grouped findings with file:line, each with a concrete trigger. Read-only, runs inline. Use when the user says "red team", "bug hunt", "adversarial review", "what could break", or before posting a fix to a PR.

red-team

Adversarial review of a PL change. Find real, triggerable bugs with a concrete failure path — not nits. Read-only; output is the artifact. Scope: a PR, a worktree diff, or named files — including your own fix: a good first use is to red-team a patch you just wrote against sibling inputs in the same class as the bug (does it re-create the failure for a neighbor?).

Setup (skim, don't deep-read — you're hunting, not summarizing)

Establish the change set: git -C <worktree> diff (or gh pr diff N). Then skim the load-bearing surfaces it touches, using reference/control-surfaces.md for where the seams are:

• apps/prairielearn/src/lib/code-caller/* — the TS side of the Python boundary
• apps/prairielearn/python/zygote.py + element code — the Python side
• apps/prairielearn/src/lib/assessment.ts, question-render.ts — grading orchestration
• apps/prairielearn/src/models/*, packages/postgres/* — SQL/queries
• apps/prairielearn/src/lib/socket-server.ts, workspace.ts — realtime
• apps/prairielearn/src/migrations/* — schema drift

Hunt (walk the list; skip what the change doesn't touch)

1. Code-caller boundary — JSON in/out shape mismatch; a Python exception swallowed into output and surfaced as success; FD-3 vs stdout confusion; restart/timeout state-machine gaps.
2. Zygote / element correctness — non-serializable return; mutation of shared data; PRNG re-seed assumptions; module cache state leaking across calls.
3. Grading state — partial grade on error; variant/submission race; double-grade; points/score rounding.
4. SQL — IN () with empty array; eq(undefined); LEFT JOIN null handling; non-deterministic pagination (missing ORDER BY); query inside a loop (N+1).
5. Transactions / TOCTOU — read-then-write across two round-trips without a tx; audit/data write not in the same tx.
6. Sockets — a subscriber error swallowed; event delivered to the wrong room; reconnect dropping state.
7. Auth / boundary — missing instructor/admin check on a route or action; CSRF token not enforced; dev-auth header trusted in prod path.
8. Migrations — non-idempotent; missing down/rollback; column used before it's added; long lock on a big table.
9. Removed-mechanism side-effects — a change that replaces a broad refresh (full page reload, redirect, refetch, global re-render) with a narrower in-place update silently stops refreshing everything else that mechanism touched: enumerate each surface the old path re-rendered (sibling panels, derived columns/counts, lists) and confirm each still updates. A narrow repro may not exhibit the stale surface — reason about it even when you can't trigger it.

Verify before reporting

For each candidate, state the trigger: the input/sequence/role that fires it. If you can't name a trigger, it's a hunch — drop it or mark it LOW/uncertain. Prefer confirming against the trace timeline (the trace skill) over speculation.

Output (inline, ~300–500 words)

Group by severity (CRITICAL / HIGH / MEDIUM / LOW). Each finding:

• file:line (clickable) — one-line bug — one-line trigger.
• No fixes unless asked. End with a one-sentence delta vs. the prior red-team if there was one.

Cleanup

Read-only — no edits, no scratch files. End with git -C <worktree> status to confirm a clean tree.