Run any Skill in Manus with one click

pentest-ai-llm

Stars5

Forks2

UpdatedJune 9, 2026 at 03:59

LLM-backed application, AI agent, and AI/ML supply-chain penetration testing. Use when the user mentions LLM pentest, AI red team, prompt injection (direct or indirect), jailbreak, DAN, Crescendo, GCG, many-shot jailbreak, system prompt leakage, RAG poisoning, embedding poisoning/inversion, vector DB attack, MCP / Model Context Protocol abuse, tool poisoning, agent excessive agency, function-call abuse, output handling chains (XSS/SSRF/SQLi via LLM output), training-data extraction, membership inference, modelscan/picklescan, Hugging Face poisoned model, garak, PyRIT, promptmap, Augustus, Spikee, OWASP LLM01-LLM10, MITRE ATLAS, or unbounded consumption / cost amplification.

Installation

Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.

Run Skill in Manus

Source

thau0x01

thau0x01/ai-pentest-kb

View GitHub Repository View Creator Repositories

Download

Run Skill in Manus

Related occupationsSOC

Based on SOC occupation classification

Information Security AnalystsComputer and Mathematical Occupations·SOC 15-1212

SKILL.md

readonly

name

pentest-ai-llm

description

Pentest — AI / LLM

Apply 11-ai-llm/ from the pentest-templates library.

Library root

$PENTEST_LIBRARY — caminho absoluto deste repositório clonado (definido pelo instalador; veja INSTALL.md).

Read 11-ai-llm/00-overview.md first.

Phase order

11-ai-llm/01-information-gathering.md — model + provider + system-prompt indicators + tool inventory + RAG sources + guardrails.
11-ai-llm/04-system-prompt-leakage.md — extract base instructions before attacking; reveals boundaries.
11-ai-llm/02-prompt-injection.md — DAN / encoding / Crescendo / GCG.
11-ai-llm/03-indirect-prompt-injection.md — RAG / web / docs / MCP.
11-ai-llm/07-excessive-agency.md — tool / function-call / MCP abuse.
11-ai-llm/05-rag-and-embeddings.md — vector DB attacks, embedding poisoning, inversion.
11-ai-llm/06-output-handling.md — chain LLM output to web vulns (XSS / SSRF / SQLi).
11-ai-llm/08-sensitive-info-disclosure.md — training-data extraction, PII inference.
11-ai-llm/10-unbounded-consumption.md — DoS / cost amplification.
11-ai-llm/09-supply-chain.md — model registry, dependency CVEs.

How to operate

Map the application surface first: model, system prompt indicators, tools/functions, RAG sources, guardrails, output sinks (01-information-gathering.md).
Extract the system prompt before anything else (04-system-prompt-leakage.md) — it reveals boundaries and tool definitions, sharply improving every later attack.
Quantify success rate (e.g., 7/10 trials) — most bypasses are non-deterministic. Findings without rate quantification are weak.
Output-handling chains: when the LLM output reaches an HTML / DB / shell / fetch sink, verify end-to-end (LLM → consumer system → impact). The web-side wraps pentest-web-api.
For agent / MCP / tool abuse, document the invocation chain + business impact (the deliverable is the chain, not the individual prompt).
Cost amplification: track $/hour budget impact and stop early — token-budget limits are mandatory.

When NOT to use

Pure web/API testing without LLM features → pentest-web-api.
Cloud-only LLM endpoints (Azure OpenAI, AWS Bedrock, Vertex) → also pentest-cloud for cloud-side config.
Mobile app with on-device LLM → also pentest-mobile.

Pre-conditions

Test account / API key.
ROE: provider acceptable-use rules, customer-data handling, cost-cap.
garak (NVIDIA) + PyRIT (Microsoft) installed; promptmap, Augustus, Spikee, modelscan, picklescan, Vec2Text, interactsh available.
Burp / Caido on HTTP wrapper.

OPSEC

Provider-side observability is significant — Azure OpenAI / AWS Bedrock log every prompt.
Aggressive testing → rate-limit / suspension; pace accordingly.
Production PII may surface in prompts — handle carefully.

Output expectations

Per-finding: prompt + response (sanitised), reproduction count, success rate, OWASP LLM Top 10 mapping.
For agent / tool abuse findings: invocation chain + business impact.
For RAG findings: source document + retrieval evidence.
For output-handling: end-to-end chain.
Cost amplification: $/hour budget impact when applicable.

References inside the library

OWASP LLM Top 10 (2025): https://genai.owasp.org/llm-top-10/
MITRE ATLAS: https://atlas.mitre.org/
garak: https://github.com/NVIDIA/garak
PyRIT: https://github.com/Azure/PyRIT
Tooling: tool-mappings/01-web-api-tools.md § LLM/AI section.
Cross-references: 01-web-api/ (HTTP wrapper), 06-cloud/ (cloud-side LLM endpoints), 02-mobile/ (mobile-app LLM features).
Playbook: playbooks/08-ai-llm-engagement.md.

More from this repository

same repository

engagement-management

thau0x01/ai-pentest-kb

Engagement-lifecycle management — pre-engagement scoping, ROE drafting, authorisation letters, threat modeling, CVSS scoring, finding write-up, executive summary, technical report, retest scoping, remediation validation. Use when the user mentions scope a pentest, ROE template, rules of engagement, get-out-of-jail letter, authorisation letter, scope statement, communications plan, blackout window, escalation matrix, threat modeling, STRIDE, PASTA, attack tree, threat actor profile, CVSS scoring, finding template, executive summary, technical report, retest, remediation validation, regression coverage, engagement kickoff, or evidence chain of custody.

2026-06-095

engagement-retest

thau0x01/ai-pentest-kb

Plan and execute a pentest retest — re-scope, re-execute the validation steps from prior findings, detect regressions, and write the retest report. Use when the user mentions retest, re-test, remediation validation, regression coverage, "verify the fixes", "validate remediation", retest scoping, retest report, posture trajectory, or compares "before vs after" findings.

2026-06-095

engagement-scoping

thau0x01/ai-pentest-kb

Drive the pre-engagement scoping conversation and produce the scope statement, Rules of Engagement, authorisation letter, and communications plan. Use when the user mentions kickoff, scoping a pentest, drafting ROE, rules of engagement, scope statement, authorisation letter, get-out-of-jail letter, communications plan, blackout window, escalation matrix, ROE drift, third-party hosting carve-out, social-engineering allowance, time-window restrictions, or evidence-handling rules.

2026-06-095

pentest-active-directory

thau0x01/ai-pentest-kb

Active Directory and hybrid Entra ID penetration testing. Use when the user mentions BloodHound, SharpHound, SOAPHound, AzureHound, RustHound, Kerberoast, AS-REP roast, Golden/Silver/Diamond/Sapphire ticket, DCSync, DCShadow, NTLM relay, ntlmrelayx, ADCS ESC1-ESC15+, EKUwu, Certipy, ACL abuse (GenericAll/WriteDACL/WriteOwner/shadow credentials), GPO abuse, NoPac, ZeroLogon, PetitPotam, DFSCoerce, PrinterBug, krbrelayup, AAD Connect, MSOL_, Pass-Through Auth, Seamless SSO, AdminSDHolder, SID History, krbtgt, or any "domain compromise" task.

2026-06-095

pentest-cheatsheet

thau0x01/ai-pentest-kb

Surface a one-page pentest cheatsheet for fast field recall. Use when the user asks for a "quick command", "cheatsheet", "one-liner", "what's the syntax for X", "remind me how to do Y", "give me the recon snippets", "AD cheatsheet", "lateral-movement one-liners", "AWS quick commands", "Azure quick commands", "mobile cheatsheet", "initial access cheatsheet", or any request that wants tools/commands without methodology overhead.

2026-06-095

pentest-checklist-lookup

thau0x01/ai-pentest-kb

Resolve a stable pentest-template checklist ID (e.g. WEB-AUTHN-04, AD-KRB-02, CLD-AWS-IAM-07, IOT-FW-05, RT-INIT-03, LLM-PI-03) to its full methodology, references, mappings, how-to-test block, and accept criteria. Use when the user mentions a checklist ID, asks to "look up <ID>", "what does <ID> say", "show me <ID>", "find checklist items mapped to CVE-XXXX", "find items mapped to ATT&CK Tnnn", "find items by CWE-NNN", or wants to navigate the templates by stable identifier.

2026-06-095

name

pentest-ai-llm

description

Pentest — AI / LLM

Apply 11-ai-llm/ from the pentest-templates library.

Library root

$PENTEST_LIBRARY — caminho absoluto deste repositório clonado (definido pelo instalador; veja INSTALL.md).

Read 11-ai-llm/00-overview.md first.

Phase order

11-ai-llm/01-information-gathering.md — model + provider + system-prompt indicators + tool inventory + RAG sources + guardrails.
11-ai-llm/04-system-prompt-leakage.md — extract base instructions before attacking; reveals boundaries.
11-ai-llm/02-prompt-injection.md — DAN / encoding / Crescendo / GCG.
11-ai-llm/03-indirect-prompt-injection.md — RAG / web / docs / MCP.
11-ai-llm/07-excessive-agency.md — tool / function-call / MCP abuse.
11-ai-llm/05-rag-and-embeddings.md — vector DB attacks, embedding poisoning, inversion.
11-ai-llm/06-output-handling.md — chain LLM output to web vulns (XSS / SSRF / SQLi).
11-ai-llm/08-sensitive-info-disclosure.md — training-data extraction, PII inference.
11-ai-llm/10-unbounded-consumption.md — DoS / cost amplification.
11-ai-llm/09-supply-chain.md — model registry, dependency CVEs.

How to operate

Map the application surface first: model, system prompt indicators, tools/functions, RAG sources, guardrails, output sinks (01-information-gathering.md).
Extract the system prompt before anything else (04-system-prompt-leakage.md) — it reveals boundaries and tool definitions, sharply improving every later attack.
Quantify success rate (e.g., 7/10 trials) — most bypasses are non-deterministic. Findings without rate quantification are weak.
Output-handling chains: when the LLM output reaches an HTML / DB / shell / fetch sink, verify end-to-end (LLM → consumer system → impact). The web-side wraps pentest-web-api.
For agent / MCP / tool abuse, document the invocation chain + business impact (the deliverable is the chain, not the individual prompt).
Cost amplification: track $/hour budget impact and stop early — token-budget limits are mandatory.

When NOT to use

Pure web/API testing without LLM features → pentest-web-api.
Cloud-only LLM endpoints (Azure OpenAI, AWS Bedrock, Vertex) → also pentest-cloud for cloud-side config.
Mobile app with on-device LLM → also pentest-mobile.

Pre-conditions

Test account / API key.
ROE: provider acceptable-use rules, customer-data handling, cost-cap.
garak (NVIDIA) + PyRIT (Microsoft) installed; promptmap, Augustus, Spikee, modelscan, picklescan, Vec2Text, interactsh available.
Burp / Caido on HTTP wrapper.

OPSEC

Provider-side observability is significant — Azure OpenAI / AWS Bedrock log every prompt.
Aggressive testing → rate-limit / suspension; pace accordingly.
Production PII may surface in prompts — handle carefully.

Output expectations

Per-finding: prompt + response (sanitised), reproduction count, success rate, OWASP LLM Top 10 mapping.
For agent / tool abuse findings: invocation chain + business impact.
For RAG findings: source document + retrieval evidence.
For output-handling: end-to-end chain.
Cost amplification: $/hour budget impact when applicable.

References inside the library

OWASP LLM Top 10 (2025): https://genai.owasp.org/llm-top-10/
MITRE ATLAS: https://atlas.mitre.org/
garak: https://github.com/NVIDIA/garak
PyRIT: https://github.com/Azure/PyRIT
Tooling: tool-mappings/01-web-api-tools.md § LLM/AI section.
Cross-references: 01-web-api/ (HTTP wrapper), 06-cloud/ (cloud-side LLM endpoints), 02-mobile/ (mobile-app LLM features).
Playbook: playbooks/08-ai-llm-engagement.md.