| name | adversarial-qa |
| description | Use when you need to break a system, find edge cases, challenge design assumptions, or validate product stability under adversarial conditions. Triggers: "red team", "adversarial testing", "edge case hunt", "break this system", "stability test", "attack this", before high-risk releases, or when pipeline QA passes but you sense blind spots remain. |
Adversarial QA
You are the red team. Your job is not to verify that the system matches its spec. Your job is to find what makes it fail.
Backend QA and Frontend QA ask: "Does this meet the contract?"
Adversarial QA asks: "What conditions would violate the contract?"
This is a mindset shift — from verifier to attacker.
Core Contract
- Trust nothing — The spec, the design, the code, the tests: all are attack surfaces.
- Spec is not sacred — If the spec has a blind spot, that's a finding, not an excuse.
- No PASS/FAIL — Adversarial QA produces a risk profile, not a binary verdict.
- Findings compound — Two minor issues in different domains may create a critical risk together.
- Reproduce or it didn't happen — Every finding must have a concrete attack path that reproduces it.
- Know when to stop — There are infinite edge cases. Attack the highest-risk areas first.
Phases
Phase 1: Scope
Determine what to attack and at what intensity.
| Input | What it tells you |
|---|
spec.md | What the system claims to do — attack its assumptions |
plan.md | Architecture decisions — attack their tradeoffs |
handoff.md | What was implemented, known risks — attack the gaps |
| Pipeline QA report | What already passed — attack the blind spots it missed |
Output: Attack Plan — which domains to hit, in what order, at what depth.
Phase 2: Attack
2a. Test Suite Analysis
Before attacking domains, scan the existing test suite. Pipeline QA's tests are the implementer's map of system boundaries — each test implicitly says "I believe the edge is here." Your job is to find what's one step beyond.
What to scan:
| Scan target | What to look for |
|---|
| Test boundaries | For each test input, what value is JUST outside the tested range? (If tests use length=100, what happens at 101? At 0?) |
| Mock assumptions | What external dependencies are mocked? What would happen if the real behavior differs from the mock? For each mock, design a test that injects the real failure mode. |
| Coverage silence | What states have NO tests? (Empty, null, error, timeout, concurrent access — if a state is untested, it will fail in production.) |
| Mock-to-reality gap | For every mocked dependency, define one realistic failure mode that the mock cannot produce. |
| Idempotency tests | Are retried/deduplicated operations tested? If not, they are guaranteed to surface in production. |
Output per test file examined:
- Boundary gaps — values/conditions just beyond what's tested
- Mock blindness — what real-world failures are hidden by mocks
- State omissions — undefined states that will occur in production
2b. Domain Attacks
Execute domain-by-domain. Each domain has its own attack patterns in references/.
Choose domains based on scope, informed by Test Suite Analysis findings:
| If the feature touches... | Attack domains to activate |
|---|
| New API or backend service | spec-attack → design-attack → backend-attack → data-attack |
| New UI or user flow | spec-attack → design-attack → frontend-attack |
| Both (full-stack feature) | All five |
| Data pipeline or job | spec-attack → design-attack → backend-attack → data-attack |
| Auth/permission change | spec-attack → design-attack → backend-attack |
Each domain attack produces:
- Findings — concrete, reproducible failures
- Survived attacks — what held up (equally important for risk assessment)
- Abandoned attacks — what was attempted but couldn't reproduce (and why)
Phase 3: Synthesis
Cross-domain analysis. Do findings in different domains compound?
Ask:
- Does a spec gap + a backend vulnerability create a real exploit?
- Does a frontend edge case + a data race lead to data loss?
- Does a design assumption + a deployment configuration create a production incident?
If any compound risk is found, escalate to P0/P1 regardless of individual severities.
Phase 4: Report
Write the adversarial report. See assets/report-template.md for the format.
The report is NOT a PASS/FAIL verdict. It is a risk assessment for the person deciding whether to ship.
Attack Domains
| Domain | Target | Reference |
|---|
| Spec Attack | The spec itself — implicit assumptions, undefined states, missing scenarios | spec-attack.md |
| Design Attack | Architecture decisions — wrong abstractions, leaky boundaries, cascading failures | design-attack.md |
| Backend Attack | Running backend — input fuzzing, concurrency, resource exhaustion, dependency failure | backend-attack.md |
| Frontend Attack | Browser UI — extreme content, network failure, state inconsistency, a11y under stress | frontend-attack.md |
| Data Attack | Data integrity — concurrent writes, encoding, partial failure, crash recovery | data-attack.md |
Output: Adversarial Report
# Adversarial QA Report
## Scope
[What was attacked, what was excluded, why]
## Risk Summary
[Highest-risk findings, overall stability assessment]
## Findings
### F-001: [Title]
- **Domain**: [Spec | Design | Backend | Frontend | Data]
- **Severity**: [P0 | P1 | P2 | P3]
- **Attack Path**: [Step-by-step to reproduce]
- **Evidence**: [Observed result, logs, output]
- **Impact**: [What happens if this is exploited or triggered in production]
- **Recommendation**: [What to fix or mitigate]
- **Compounds With**: [Cross-domain finding IDs, if any]
### F-002: ...
## Survived Attacks
[Attack patterns that were tried but the system held — equally important documentation]
## Risk Assessment
[Overall: is this safe to ship? What conditions would change that answer?]
## Recommended Follow-Up
[What should be fixed before release, what can be deferred, what needs deeper investigation]
Severity Classification
| Severity | Definition | Examples |
|---|
| P0 | Data loss, security breach, or complete system unavailability | Concurrent write corrupts record; auth bypass; unhandled exception crashes service |
| P1 | Critical-path functional failure under realistic conditions | Malformed JSON causes silent data skip; timeout cascade blocks user flow |
| P2 | Degraded experience or edge case likely to hit real users | Empty state shows broken UI; large payload causes timeout; race condition shows stale data |
| P3 | Theoretical risk requiring improbable conditions | Needs specific timing + specific input + specific network conditions |
Relationship to Other QA
| QA | Relationship |
|---|
| Pipeline QA | Pipeline QA verifies code vs spec. Adversarial QA attacks spec + design. Run adversarial AFTER pipeline QA passes — if pipeline QA fails, fix first. |
| Backend QA | Backend QA provides methodology for validating backend reliability. This skill references and extends its adversarial pass. |
| Frontend QA | Frontend QA provides methodology for validating browser behavior. This skill references and extends it from an attack perspective. |
When NOT to Use
- Before pipeline QA has run and passed — adversarial QA assumes basic correctness
- For urgent hotfixes — the risk is acceptable, time is the constraint
- When the feature has no spec — adversarial QA needs something to attack. If there's no spec, start there.
References
