ai-report-writing-guardrails

name	ai-report-writing-guardrails
description	Prevent common AI report pitfalls — bug blending, inflated threat models, and generic language. Train Claude with your best past reports for concise, technical submissions. Based on Critical Thinking Bug Bounty Podcast Episode 166.
domain	cybersecurity
subdomain	bug-hunting
category	Methodology
difficulty	intermediate
estimated_time	1-2 hours
platforms	["linux","macos","windows"]
tags	["report-writing","ai-reports","bug-bounty-reports","threat-modeling","quality-control","hackerone","bugcrowd"]
tools	["claude-code-cli"]
version	1.0
author	CyberSkills-Elite
license	Apache-2.0
source	Critical Thinking Bug Bounty Podcast Ep. 166
source_url	http://www.youtube.com/watch?v=qTX9u-EsjmM

AI Report Writing Guardrails

When to Use

• When using Claude or any LLM to draft bug bounty vulnerability reports.
• When reviewing AI-generated reports before submission to HackerOne/Bugcrowd.
• When AI reports are getting marked as N/A, Informational, or Duplicate due to quality issues.
• When teaching Claude your personal writing style to produce better reports.

Prerequisites

• 3-5 of your best past bug bounty reports (ones that got triaged quickly and paid well)
• Claude Code CLI configured for the target program
• Understanding of CVSS 4.0 scoring

Core Problem: AI Reports Get You Banned

"AI blends 2-3 separate bugs into one report. Program managers HATE that. And the threat modeling is weak — it calls a paywall bypass 'complete security degradation'." — Critical Thinking Podcast, Ep. 166

The 3 Deadly AI Report Sins

Sin	What Happens	Example
Bug Blending	AI mixes 2-3 separate vulnerabilities into 1 report	"Found XSS, CSRF, and IDOR in the dashboard" → should be 3 separate reports
Inflated Threat Model	AI exaggerates impact beyond technical reality	Paywall bypass → "complete security degradation of all financial systems"
AI Voice Detection	Triage team recognizes LLM language and downgrades	"It's worth noting", "This vulnerability poses a significant risk", "Certainly"

Result: Report gets N/A'd, your reputation score drops, you may get warned or banned.

Workflow

Phase 1: The Bug Blending Check

Before submitting any AI report, apply the One Bug = One Report rule:

## Bug Blending Checklist
For each AI-generated report, ask:

1. How many DISTINCT attack flows are described?
   - If > 1 → SPLIT into separate reports
   
2. Does each CWE map to the same root cause?
   - Different CWEs = different reports (CWE-79 XSS ≠ CWE-352 CSRF)
   
3. Can each finding be independently reproduced?
   - If yes → separate reports
   - If chained (A enables B) → one report with clear chain

4. Remove sentences containing "Additionally", "Furthermore", "Moreover" 
   that introduce DIFFERENT vulnerability classes

Before (AI Output — Bug Blended):

## Vulnerability Report: Multiple Security Issues in User Dashboard

The user dashboard at /dashboard contains a stored XSS vulnerability 
via the bio field. Additionally, the same endpoint is vulnerable to 
CSRF attacks as it lacks anti-CSRF tokens. Furthermore, the API 
endpoint /api/v2/users/{id} has an IDOR vulnerability...

After (Fixed — One Bug per Report):

## Report 1: Stored XSS in /dashboard Bio Field Allows Session Hijack
[XSS details only — CWE-79]

## Report 2: CSRF on /dashboard/settings Enables Profile Takeover  
[CSRF details only — CWE-352]

## Report 3: IDOR in /api/v2/users/{id} Exposes All User Records
[IDOR details only — CWE-639]

Phase 2: Threat Model Reality Check

AI tends to inflate impact. Run every AI report through this filter:

## Threat Model Verification Checklist

1. ❓ "What EXACTLY can an attacker steal/modify/destroy?"
   - ❌ "Complete security degradation" → meaningless
   - ✅ "Read 4.2M user email addresses without auth" → specific

2. ❓ "What prerequisites does the attacker need?"
   - ❌ Assumes zero prerequisites → unrealistic
   - ✅ "Requires authenticated low-privilege account" → honest

3. ❓ "Is the CVSS score defensible?"
   - Test: Would you argue this score to a security engineer?
   - Common AI inflation: rating a cosmetic issue as High (7.0+)
   
4. ❓ "Does the 'Impact' section contain at LEAST one real number?"
   - ❌ "Affects many users" → useless
   - ✅ "Affects 12,847 active accounts on this endpoint" → credible

5. ❓ "Is the remediation generic or specific?"
   - ❌ "Implement proper input validation" → tells nothing
   - ✅ "Add parameterized query at line 42 of UserController.js" → actionable

AI Output (inflated):

Impact: This vulnerability poses a significant security risk,
potentially leading to complete compromise of the application
and its underlying infrastructure.

Fixed (realistic):

Impact: An unauthenticated attacker can read the email, name, 
and phone number of any registered user by incrementing the 
user ID parameter. Tested on 5 IDs — all returned data. 
Estimated affected users: ~12K based on response to ID=12847.

Phase 3: Train Claude with Your Best Reports

"Give Claude your old best reports as examples. Tell it: strictly concise, strictly technical." — Episode 166

Create a report training file that Claude reads before writing:

# .claude/report-style-guide.md

## My Report Writing Rules (Claude MUST follow these)

### Voice
- First person, active voice: "I discovered", "I tested", not "It was found"
- Zero marketing language. No "significant", "robust", "comprehensive"
- Max 3 sentences per section (Summary, Impact, Remediation)

### Structure (HackerOne format)
1. Title: [VulnType] in [Endpoint] Allows [Impact]
2. Severity: CVSS 4.0 with vector string
3. Summary: 2-3 sentences MAX
4. Steps to Reproduce: Numbered, deterministic, copy-paste
5. PoC: Working curl/script — ZERO placeholders
6. Impact: ONE paragraph with at least one real number
7. Remediation: Before/after code, not generic advice

### NEVER write:
- "It's worth noting..."
- "This vulnerability poses a significant..."
- "Certainly..."
- "In conclusion..."
- "The implications are far-reaching..."
- Any sentence > 25 words

### Example Reports (my style):
[Paste 2-3 of your best reports here for Claude to learn from]

Prompt for Claude report generation:

Read .claude/report-style-guide.md first.

Write a HackerOne report for the IDOR vulnerability I found in 
findings/idor-user-profiles.md.

Rules:
- ONE vulnerability per report. Do NOT blend bugs.
- Verify your CVSS score against the actual impact.
- Remove ALL AI-sounding language.
- Keep total report under 400 words.
- PoC must be a copy-paste curl command.

Phase 4: Pre-Submission Quality Gate

Run every report through this automated check before hitting Submit:

// scripts/report-quality-check.ts
#!/usr/bin/env npx tsx

import { readFileSync } from "fs";

const AI_PHRASES = [
  "it's worth noting", "it is worth noting",
  "significant risk", "significant security",
  "poses a significant", "far-reaching",
  "certainly", "furthermore", "moreover",
  "comprehensive compromise", "complete degradation",
  "robust security", "critical importance",
  "in conclusion", "to summarize",
  "it should be noted", "importantly",
];

const QUALITY_CHECKS = {
  hasSingleCWE: (text: string) => {
    const cweMatches = text.match(/CWE-\d+/g) || [];
    const uniqueCWEs = [...new Set(cweMatches)];
    return { pass: uniqueCWEs.length <= 1, detail: `Found CWEs: ${uniqueCWEs.join(", ") || "none"}` };
  },
  hasConcreteImpact: (text: string) => {
    const hasNumbers = /\d{2,}/.test(text.split("Impact")[1] || "");
    return { pass: hasNumbers, detail: hasNumbers ? "Impact has specific numbers" : "Impact lacks concrete numbers" };
  },
  noAILanguage: (text: string) => {
    const lower = text.toLowerCase();
    const found = AI_PHRASES.filter((p) => lower.includes(p));
    return { pass: found.length === 0, detail: found.length ? `AI phrases: ${found.join(", ")}` : "Clean" };
  },
  hasWorkingPoC: (text: string) => {
    const hasCurl = text.includes("curl ") || text.includes("fetch(") || text.includes("POST ");
    return { pass: hasCurl, detail: hasCurl ? "PoC command found" : "Missing executable PoC" };
  },
  noBugBlending: (text: string) => {
    const blendWords = ["additionally", "furthermore", "also discovered", "another vulnerability"];
    const lower = text.toLowerCase();
    const found = blendWords.filter((w) => lower.includes(w));
    return { pass: found.length === 0, detail: found.length ? `Possible blending: ${found.join(", ")}` : "Single focus" };
  },
};

const reportPath = process.argv[2];
if (!reportPath) { console.error("Usage: npx tsx report-quality-check.ts <report.md>"); process.exit(1); }

const content = readFileSync(reportPath, "utf-8");
console.log(`\nReport Quality Check: ${reportPath}\n${"=".repeat(50)}`);

let failures = 0;
for (const [name, check] of Object.entries(QUALITY_CHECKS)) {
  const result = check(content);
  const icon = result.pass ? "✅" : "❌";
  if (!result.pass) failures++;
  console.log(`${icon} ${name}: ${result.detail}`);
}

console.log(`\n${failures === 0 ? "✅ READY TO SUBMIT" : `❌ ${failures} issue(s) — fix before submitting`}`);

Decision Point 🔀

flowchart TD
    A[AI generated a report] --> B{Run quality gate}
    B --> C{Bug blending?}
    C -->|Yes| D[Split into separate reports]
    C -->|No| E{AI language detected?}
    E -->|Yes| F[Rewrite in your voice]
    E -->|No| G{Impact inflated?}
    G -->|Yes| H[Add real numbers, reduce CVSS]
    G -->|No| I{PoC works?}
    I -->|Yes| J[✅ Submit report]
    I -->|No| K[Fix PoC first]
    D --> B
    F --> B
    H --> B
    K --> B

Creativity Directive

IMPORTANT: Build a personal report template library from your successful submissions. Train Claude on YOUR voice, not generic templates. Automate the quality gate into your CI pipeline. Think like an attacker. Adapt. Improvise.

🔵 Blue Team

• Deploy robust WAF rules to detect anomalies.
• Monitor logs for unusual access patterns.

📚 Shared Resources

For cross-cutting methodology applicable to all vulnerability classes, see:

• _shared/references/elite-chaining-strategy.md — Exploit chaining methodology and high-payout chain patterns
• _shared/references/elite-report-writing.md — HackerOne-optimized report writing, CWE quick reference
• _shared/references/real-world-bounties.md — Verified disclosed bounties by vulnerability class

References

• Source: Critical Thinking Ep. 166
• HackerOne Report Guidelines: https://docs.hackerone.com/
• CVSS 4.0 Calculator: https://www.first.org/cvss/calculator/4.0