| name | cve-test-generator |
| description | Guides Generator Agent to write high-quality dynamic test_vuln.py for CVE reproduction. Provides PoC level classification (P1-P5), CWE-specific payload collections, runtime judgment criteria, and structured self-check workflow. Use when Generator writes vulnerability tests, mentions CWE, test_vuln, PoC, exploit, security test, XSS, SQL injection, command injection, path traversal, CSRF, buffer overflow, use-after-free, missing authorization, or file upload. |
CVE Test Generator — PoC Quality & CWE Attack Knowledge
This skill supplements generator.md with PoC quality standards and CWE-specific attack knowledge.
It does NOT replace generator.md. Follow generator.md for output formats, file structure, and general rules.
When to Use
- Generator Agent is writing test_vuln.py
- for_generator.md identifies a vulnerability mappable to CWE Top 10
- Generator needs to know what constitutes a valid PoC for this CWE type
When NOT to Use
- Writing test_func.py, solution.sh, task.yaml, or docker_requirements.md
- CWE type is not in the route table below (use generator.md general rules instead)
- for_generator.md lacks sufficient info to identify CWE type (request Analyzer to supplement first)
Workflow
Execute these steps in order. Do not skip any step.
Step 1: Read for_generator.md. Identify the CWE type.
Step 2: Determine PoC level using the decision tree below.
Step 3: Read the corresponding CWE reference file from the route table.
Step 4: Write test_vuln.py following the CWE reference template.
Include PoC Level declaration in the docstring.
Step 5: Run structured self-check (format below). Output the report.
Step 6: If any check is FAIL → return to Step 4, fix, then redo Step 5.
Max 3 retries. After 3 FAILs → stop, mark needs_human_review.
Step 7: All checks PASS → proceed to write remaining files per generator.md.
PoC Level Decision Tree
Is the target a web service or API?
Yes → P1 (Network Interaction)
No ↓
Is the target a compilable/runnable binary?
Yes → P2 (Process Execution)
No ↓
Is the target an importable library/module?
Yes → P3 (Library Call)
No ↓
Is the target unable to run in Docker? (kernel/mobile/embedded/hardware)
Yes → P4 (Algorithm Verification) — must declare downgrade reason
No ↓
Is the target a configuration/permission/network behavior?
Yes → P5 (Configuration Verification)
For full P1-P5 definitions (criteria, forbidden patterns, templates): see poc-levels.md
Prefer higher levels: P1 > P2 > P3 > P4 > P5. Higher = stronger dynamic verification.
CWE Route Table
If CWE is not listed: do not use this skill. Follow generator.md general rules.
Declare in docstring: CWE: Other (not in skill coverage).
test_vuln.py Required Metadata
Every test_vuln.py must start with this docstring:
"""
Vulnerability tests for CVE-XXXX-XXXXX
PoC Level: P<N> (<level name>)
CWE: CWE-<ID> (<name>)
Target: <what is being tested>
Judgment: <what runtime signal determines pass/fail>
"""
If PoC Level is P4, add:
"""
Downgrade Reason: <why P1-P3 are not feasible>
Algorithm Source: <file:line range from real source code>
"""
Structured Self-Check
After writing test_vuln.py, output this report. Fill in evidence with actual line numbers.
=== PoC Quality Self-Check ===
PoC Level: P<N>
CWE: CWE-<ID>
Check 1 - Dynamic execution:
[PASS/FAIL] Evidence: <function calls and line numbers>
If FAIL → return to Step 4: add requests/subprocess/import calls
Check 2 - Judgment basis:
[PASS/FAIL] Evidence: <assert statements and what they check>
If FAIL → return to Step 4: change asserts to check runtime output
Check 3 - Attack vectors:
[PASS/FAIL] Evidence: <number of test methods and payload list>
If FAIL → return to Step 4: add payloads from CWE reference
Check 4 - PoC level compliance:
[PASS/FAIL] Evidence: <does code match the level's criteria?>
If FAIL → return to Step 4: consult poc-levels.md
Check 5 - Vuln-state FAIL direction:
[PASS/FAIL] Evidence: <assert fails when vuln exists, passes when fixed>
If FAIL → return to Step 4: invert assert direction
Result: X/5 PASS
On Failure
- CWE not in route table: Do not load reference. Use generator.md rules. Declare
CWE: Other.
- for_generator.md lacks CWE info: Check public.md and analyzer-res.xml. Still unclear → use closest CWE, declare uncertainty in docstring.
- Multiple PoC levels apply: Choose the highest (P1 > P2 > P3).
- Self-check fails 3 times: Stop retrying. Record difficulty in generator-res.xml. Mark
needs_human_review.
- Reference payloads don't fit this CVE: Design custom payloads following the reference's pattern. Declare
Custom payload: <reason> in docstring.
