| name | validation-fuzz-dashboard |
| description | Use when the user asks for a live-validation dashboard, fuzzing-results dashboard, validation coverage roll-up, or "how many findings were confirmed exploitable / refuted" — e.g. "build the validation dashboard", "combine validation and fuzz results", "what did live validation confirm", "validation false-positive rate". Aggregates every validations/*/*validation*.json report plus FUZZ-CAMPAIGN-SUMMARY.md into one self-contained HTML dashboard. |
Live Validation & Fuzzing — Portfolio Dashboard
Roll up every live-validation report (verdicts rendered against static-audit
findings in real environments) and the offline fuzz campaign into a single
self-contained HTML dashboard: confirmed-exploitable hero number, verdict share
by claimed severity, validation coverage, top targets by confirmed findings,
attack chains with a confirmed step, fuzz pattern hit-rates, and the confirmed
fuzz-bug table.
Input
$ARGUMENTS is optional and may be any combination of:
| Token | Meaning |
|---|
| (none) | Standard run — scan <results-root>/validations/ + FUZZ-CAMPAIGN-SUMMARY.md, write the dashboard. |
open | Open the resulting HTML in the default browser when done. |
summary | After building, print verdict totals and top-5 targets to the conversation. |
<path> | Override the analysis-results root (directory containing validations/). |
Procedure
Step 1 — Locate the analysis-results root
Resolution order (same as /executive-summary-findings):
- A path passed in
$ARGUMENTS.
$AUDIT_RESULTS_ROOT environment variable.../analysis-results relative to the ai-security-harness checkout.- Walk upward from
$PWD looking for a directory containing validations/.
If none resolve, ask the user for the path.
Step 2 — Run the harness script
The skill ships build_validation_fuzz_dashboard.py alongside this file:
python3 "$SKILL_DIR/validation-fuzz-dashboard/build_validation_fuzz_dashboard.py" \
--results-root <resolved-root>
What the script does:
- Walks
<results-root>/validations/*/, loading every *validation*.json
(the live-validation report schema: validated_findings[] with
verdict ∈ CONFIRMED / REFUTED / INCONCLUSIVE / BLOCKED_BY_SCOPE /
NOT_ATTEMPTED and claimed_severity, plus attack_chains[] and
needs_credential[]).
- Parses
<results-root>/FUZZ-CAMPAIGN-SUMMARY.md pipe tables — batches,
confirmed bugs (repo / bug / CVSS / CWE / advisory), and pattern hit-rates.
- Aggregates: verdict × claimed-severity matrix, per-target confirmed counts,
confirmed-Critical finding list, attack chains with ≥ 1 confirmed step
(never the raw mapped-chain count — most steps are
not_attempted and the
raw number misleads), validation coverage (attempted / total checks).
- Writes:
<results-root>/Live-validation-fuzz-dashboard.html — single file,
inline SVG (no CDN), light/dark via prefers-color-scheme, hover tooltips
plus <details> table views so no value is color- or hover-gated.<results-root>/Live-validation-fuzz-dashboard.md — Markdown one-pager
(same numbers: headline, verdict×severity table, top targets, fuzz
hit-rates and bugs, confirmed-Critical list). Suitable for pasting into
a doc or Slack. Override with --out-md.
Design notes (dataviz conventions baked into the script — keep them if editing):
verdict segments use the fixed status palette (confirmed=critical red,
inconclusive=warning, blocked=serious, refuted=good) with 2px surface gaps,
legend, and in-segment % labels only where they fit; magnitude bars are
single-hue (categorical slot 1); NOT_ATTEMPTED is excluded from the stacked
bars and shown as a coverage meter instead, so validated and unvalidated
populations are never conflated.
Step 3 — Report back
Always tell the user:
- The two output paths (HTML + Markdown).
- Headline numbers: reports scanned, confirmed / refuted / inconclusive /
blocked totals, confirmed-Critical count, coverage %, fuzz bugs confirmed.
- The top 3 targets by confirmed findings.
If summary was requested, also print the verdict-by-severity table.
Outputs
| File | Purpose |
|---|
Live-validation-fuzz-dashboard.html | Self-contained dashboard: hero (confirmed-exploitable), KPI tiles, verdict-by-severity stacked bars, coverage meter, top-targets bars, fuzz pattern hit-rates, confirmed fuzz-bug and confirmed-Critical tables. |
Live-validation-fuzz-dashboard.md | Markdown one-pager with the same numbers — headline, verdict×severity table, top targets, fuzz hit-rates/bugs, confirmed-Critical list. Paste-ready for docs or Slack. |
The script is idempotent and read-only over its inputs — re-run it whenever new
validation reports land or the fuzz summary changes.
