| name | adversarial-exploit-review |
| description | Independent adversarial review that proves findings with failing exploit tests rather than prose — a reviewer who did NOT write the code writes tests designed to make it fail, the engineer fixes until they pass, and high-risk surfaces get multiple reviewers with distinct failure lenses. Use this when reviewing or merging a change, auditing a module, hardening auth/RBAC/money/migration/data-loss code, deciding whether one review pass is enough, writing a security or exploit test, mutation-testing a guard, or handling a non-blocking finding — on backend OR frontend, even if the user doesn't say "adversarial review." |
Adversarial Exploit Review
The author's own tests encode the author's own blind spots. You cannot write a test for a bug you didn't think of — and the code and its tests came from the same mind in the same session, so they share the same gaps. An agent makes this worse: it confidently green-lights its own work because every test it imagined passes. Independent adversarial review — a reviewer who did NOT write the code, writing tests engineered to make it FAIL — catches the class of bug self-review structurally cannot. The output of a real review is not a paragraph of concerns; it is a failing test.
When to use this
- Reviewing a diff, PR, or module before it merges — especially anything an agent wrote end to end.
- Touching a high-risk surface: money/payments, auth, RBAC/permissions, migrations, or any irreversible/data-loss action.
- A change is "done" and all the author's tests pass, but nothing hostile has been thrown at it yet.
- You're about to write a review as prose ("this might race", "consider auth here") instead of a test.
- Deciding whether one review pass is enough, or whether the surface needs a second independent reviewer.
- Writing a security/exploit test, or proving an existing guard actually fails when the bug is present.
- An optimistic UI arms a save/submit before data has hydrated, or refetches and re-arms on stale data.
- A reviewer found a real edge case that shouldn't block the merge, and you're tempted to just note it and move on.
The standards
A finding is a failing test, not a sentence. The reviewer asserts the correct/secure outcome and the test must FAIL against the current code — that failure is the proof the bug is real. Prose findings without a failing test are noise: unverifiable, easy to wave away, and impossible to confirm fixed. The engineer then fixes until the exploit test goes green; the test stays in the suite as a regression guard.
exploit_test "deleting another user's record is rejected":
other = create_record(owner="victim")
expect( delete(other.id, caller="attacker") ) is Forbidden # FAILS now → real finding
Independence is structural, not a promise. The reviewer is not the author and did not write the production code under review — enforce it by tool restriction where you can (the reviewer can read + write tests, but cannot edit the code it's reviewing). An author reviewing their own work re-runs their own assumptions. Independence is what makes the second pair of eyes actually a second mind.
High-risk surfaces require ≥2 independent reviewers who agree. For money, auth/RBAC, migrations, and irreversible actions, one reviewer is not enough — the cost of a miss is unrecoverable. Require two independent reviewers to both sign off before merge; a single dissent that's backed by a failing test blocks. (A real miss: one reviewer cleared a money path that a second, looking for the same class of bug, broke with one test.)
Mutation-test your own guards. A passing test proves nothing until you've proven it can fail. Inject the bug the guard defends against, confirm the test goes red, then revert. A guard with no failing mutation is decoration — it passes whether the code is right or wrong, and it'll keep passing right through the regression it was meant to catch.
verify_guard(test, code):
inject_bug(code); assert test FAILS # non-vacuous: it actually catches this
revert(code); assert test PASSES
Give each reviewer a distinct failure lens. Two reviewers running the same checklist find the same things and miss the same things. Assign different lenses — correctness, security, data-loss, eventual-consistency, accessibility — because diversity of attack is what catches the failure mode N identical reviewers all walk past. Two real catches a single pass missed: a save armed before hydration that wrote [] over real data, and a mutation that succeeded but re-armed off a stale refetch — each surfaced only because a different lens was looking.
Don't silently drop non-blocking findings. A real edge case that shouldn't block the merge still must not vanish. Write the failing test, mark it expected-to-fail (xfail/skip) with a tracking-issue reference, and file the issue. The edge case stays documented and executable, the merge stays unblocked, and the fix is already specified by the test waiting to go green.
@xfail("edge case, tracked in #1234")
test "concurrent double-submit collapses to one charge": ...
Checklist
What breaks without this
- Self-review confirms the bug instead of catching it. The author writes the tests that match the code they wrote; both share the same blind spot, so green means "consistent with my assumptions," not "correct."
- Prose findings get waved away. "This might race" with no failing test is unfalsifiable — the author disagrees, nobody can prove it, and it ships.
- Vacuous guards rot. A test that was never seen to fail passes whether the code is right or wrong; the regression it "guards" sails straight through.
- A single reviewer's blind spot becomes production's. One lens misses the data-loss-on-hydration or stale-refetch class of bug entirely; on a money or auth path that miss is unrecoverable.
- Edge cases evaporate. A finding noted in a comment and not turned into an
xfail test plus an issue is forgotten by the next session and recurs.
Stack-specific examples
- Backend: see
references/backend.md (pytest — exploit test that fails first, RBAC/ownership bypass, mutation-checking a guard, xfail + tracking issue, ≥2-reviewer money path)
- Frontend: see
references/frontend.md (vitest + Testing Library — save-armed-before-hydration, succeeded-but-stale-refetch re-arm, client-authz bypass, optimistic rollback, mutation-checking a guard)
Related
tests-as-ground-truth — this skill is its adversarial complement: ground-truth tests define correct behavior; exploit tests attack the gaps the author's own tests left.agent-team-review-loop for the process — bounded tasks, tool-restricted roles, and the loop that cycles until reviewers report clean. This skill is what a reviewer in that loop actually does; don't re-specify the loop here.guardrails-and-rule-flywheel — once a class of exploit shows up twice, promote it from a one-off test to a lint rule or pre-push hook so it can't recur.
