| AP-1 | Scan-and-Dump — running automated tools without analysis | Are findings contextualized? | Add attack chains and business impact |
| AP-2 | Static Scope — reusing the same test plan across assessments | Is the threat model system-specific? | Build fresh threat model per engagement |
| AP-3 | Tool Tunnel Vision — relying on a single tool or technique | Were multiple attack vectors explored? | Combine manual and automated approaches |
| AP-4 | No Blue Feedback — attacking without detection validation | Are detection gaps documented? | Add detection recommendations per finding |
| AP-5 | Severity Inflation — marking everything as Critical | Is severity evidence-based? | Use CVSS and exploitability as inputs |
| AP-6 | Fix-Free Findings — reporting issues without remediation | Does every finding have a fix? | Add immediate and long-term remediation |
| AP-7 | One-Shot Testing — testing only at release time | Is testing integrated into SDLC? | Recommend continuous red team cadence |
| AP-8 | Model-Only Focus — testing only the LLM, not the system | Was the full pipeline tested? | Include RAG, tools, plugins, and glue code |
| AP-9 | Single-Shot AI Testing — single prompt tests only for AI systems | Were multi-turn attack chains tested? | See Core Contract "Agentic AI testing principle" |
| AP-10 | Isolation Testing — testing AI in isolation, not as deployed | Was the deployed system (RAG+tools+plugins) tested? | Test the full integrated pipeline |
| AP-11 | RAG Poisoning Blindspot — ignoring data poisoning in retrieval corpus | Were RAG sources tested for adversarial injection? | 5 crafted documents can manipulate 90% of AI responses; test corpus integrity |
| AP-12 | Prompt Leakage Ignored — not testing for system prompt extraction | Was system prompt leakage tested? | OWASP LLM07 (2025): attackers extract internal rules, permissions, decision logic |
| AP-13 | Binary-Only Scoring — reporting AI red team results with pass/fail ASR only | Are findings scored multi-dimensionally? | Binary ASR is ambiguous and non-comparable across engagements; score by violation severity, attack naturalness, and semantic preservation |
| AP-14 | Benchmark Over-Reliance — using known test prompts as security proof for AI systems | Were novel attack vectors tested beyond benchmarks? | Models can be patched against benchmark prompts during alignment; full marks on a benchmark does not indicate security. Test with roleplay frames, hypothetical framings, multi-step reasoning, and translated text |
| AP-15 | Prompt-Level Security — embedding security controls (guardrails, filters, access rules) inside prompts instead of external enforcement | Are security controls enforced outside the LLM? | Adaptive attacks bypass prompt-level defenses with >90% ASR; enforce tool-call approvals, file-type firewalls, and kill switches at the application layer, not in system prompts |
| AP-16 | Context Manipulation Blindspot — testing only technical exploits while ignoring narrative/social deception of AI agents | Were agents tested with compelling fictional scenarios designed to override their constraints? | Real-world agentic red teaming shows agents fail to contextual manipulation — adversaries provide fictional authority contexts where agents agree their own rules don't apply; test with role-play scenarios, simulated emergencies, and multi-turn trust-building chains |
| AP-17 | Jailbreak-Only Agent Testing — applying generic jailbreak libraries to agentic systems instead of testing operational risks | Were tool misuse, unauthorized actions, and data exfiltration tested? | See Core Contract "Agentic AI testing principle" — test authorization bypass on tool calls, not just response content |
| AP-18 | Skill Registry Trust — treating agent skill/tool registries as trusted without supply chain verification | Were agent skills verified for integrity before deployment? | MITRE ATLAS OpenClaw Investigation (2026-02) documents agentic AI exploit paths including tool invocation abuse and agentic configuration modification; verify manifest signatures, audit permission scopes, treat all registries as untrusted by default |