| name | judge-pentest |
| description | Penetration testing evaluation checklist for the decision Agent. Evaluates whether a pentest has sufficiently covered all attack surfaces, determines task completion, and provides specific feedback on uncovered areas. |
| metadata | {"tags":"judge,evaluation,pentest,coverage,decision","category":"general"} |
Penetration Testing Evaluation Checklist
Web Application Vulnerability Coverage Check
Check each attack surface below for whether it has been tested; mark untested ones as gaps:
Injection
Cross-Site
Authentication & Authorization
Business Logic
Information Disclosure
Server-Side
Configuration
API-Specific
Known CVE/CNVD
Evaluation Decision Logic
The goal of penetration testing is to discover as many vulnerabilities as possible; do NOT end prematurely.
# In pentest scenarios, complete is advisory only and does NOT trigger early exit
# The judge's core value is providing precise "what to test next round" feedback
if tested_categories >= 90% of total && two consecutive rounds with no new vulns:
complete = true, confidence >= 0.8
else:
complete = false
feedback = explicitly list untested attack surfaces with specific testing suggestions
missing_areas = names of untested categories
Important: Better to run one extra round than to miss one direction. Even if many vulnerabilities have been found, if there are still untested attack surfaces, return complete=false.
If a target product/tech stack was identified in this round but search_vulndb was never called, MUST return complete=false and require using search_vulndb(query="product name") to query known vulnerabilities.
Feedback Template
When complete=false, feedback should include:
- Completed work (acknowledge positively, avoid repetition)
- Specific missing directions (do NOT say "keep testing" vaguely — specify concrete endpoints + vulnerability types)
- Suggested test steps (e.g., "use sqlmap for deep injection testing on the query parameter of /api/search")
