audit-guide

# Audit Guide Run a comprehensive read-only audit of an existing guide `content.json` and produce a single prioritised report. This skill orchestrates three independent sub-agent analyses — structural, semantic, adversarial — and synthesises them. **Read-only on the guide.** This skill never writes to `content.json` or `manifest.json`. It only writes to `assets/`. **Do NOT read external reference files upfront.** Each phase's sub-agent loads its own references. Everything the orchestrator needs is inline below. This skill implements the **skill-memory convention**. See [../skill-memory.md](../skill-memory.md) for the shared `assets/`, manifest, and frontmatter standards. --- ## Workflow Overview ``` Input (guide directory path) │ ├─ Phase 0: Check for Prior Run ── orchestrator │ Compares hash of content.json against assets/manifest.yaml │ ├─ Phase 1: Acquire & Parse ───── orchestrator (no external reads) │ Reads: content.json, manifest.json (if present) │ Writes: assets/audit-input.json (flattened block tree with paths) │ ├─ Phase 2: Structural Audit ──── sub-agent loads .cursor/commands/lint.md │ │ + ../shared/critical-rules.md │ Writes: assets/structural-report.md │ ├─ Phase 3: Semantic Audit ────── sub-agent loads .cursor/commands/check.md │ │ + .cursor/authoring-guide.mdc │ │ + .cursor/best-practices.mdc │ │ + AGENTS.md (critical rules) │ Writes: assets/semantic-report.md │ ├─ Phase 4: Adversarial Audit ─── sub-agent loads .cursor/commands/attack.md │ │ + .cursor/edge-cases-and-troubleshooting.mdc │ Writes: assets/adversarial-report.md │ └─ Phase 5: Synthesise ────────── orchestrator Merges the three raw reports → assets/audit-report.md Writes/updates assets/manifest.yaml ``` When `focus area` is provided, the orchestrator skips Phase 2/3/4 phases not in the focus and synthesises a focused report. See **Inputs** below. --- ## Inputs - **Required**: `guide_dir` — path to a directory containing `content.json` (e.g., `alerting-101/`). - **Optional**: `severity_floor` — `errors` | `warnings` | `all` (default `all`). Findings below the floor are omitted from the synthesised report. - **Optional**: `focus` — `structural` | `selectors` | `lazy-render` | `requirements` | `editorial`. When set, skip phases that don't contribute to that focus. Mapping: - `structural` → Phase 2 only - `selectors` → Phase 2 + Phase 4 - `lazy-render` → Phase 2 + Phase 4 - `requirements` → Phase 3 - `editorial` → Phase 3 + Phase 4 - (unset) → all phases If the user provides no arguments, ask them for `guide_dir` and offer the defaults for the other two. --- ## Critical Rules Read [../shared/critical-rules.md](../shared/critical-rules.md) for the canonical 18 + 3 critical rules. Phase 2 and Phase 3 sub-agents must cite rule numbers when reporting findings. The shared file forwards to [../../../AGENTS.md](../../../AGENTS.md) — that is the authoritative list. --- ## Severity Rubric Read [severity-rubric.md](severity-rubric.md) for the canonical severity definitions used by all three audit phases. The rubric defines what each phase classifies as **blocking**, **warning**, or **info** and how Phase 5 prioritises across the three reports. --- ## Generated Files The skill's deliverable is `assets/audit-report.md`. Everything else in `assets/` is intermediate. ``` {guide_dir}/ content.json ← INPUT (never modified) manifest.json ← INPUT (never modified, may be absent) assets/ manifest.yaml ← skill memory (hash, focus, severity_floor) audit-input.json ← flattened block tree (Phase 1) structural-report.md ← Phase 2 raw output semantic-report.md ← Phase 3 raw output adversarial-report.md ← Phase 4 raw output audit-report.md ← Phase 5 deliverable (prioritised findings) ``` ### Frontmatter disclaimer Every generated markdown file in `assets/` **must** start with: ```markdown --- disclaimer: Auto-generated by audit-guide skill. Do not edit manually. notice: To regenerate, re-run the skill against this guide. input_sha256: {hex digest} guide_id: {id from content.json} --- ``` ### Manifest file After Phase 5, write `{guide_dir}/assets/manifest.yaml`: ```yaml # Auto-generated by audit-guide skill. Do not edit manually. schema_version: 1 skill_name: "audit-guide" generated_at: "{ISO 8601 timestamp}" input_sha256: "{hex digest — see below}" guide: id: "{guide id from content.json}" total_blocks: {N} total_interactive_steps: {N} audit: focus: "{focus or 'all'}" severity_floor: "{errors|warnings|all}" finding_counts: blocking: {N} warning: {N} info: {N} files: audit_input: "assets/audit-input.json" structural_report: "assets/structural-report.md" semantic_report: "assets/semantic-report.md" adversarial_report: "assets/adversarial-report.md" audit_report: "assets/audit-report.md" ``` Compute `input_sha256` by hashing the raw `content.json` content: ```bash shasum -a 256 {guide_dir}/content.json | awk '{print $1}' ``` This means re-running the audit against the same `content.json` matches the prior run and triggers warm-start. Re-running after any edit (even cosmetic) triggers a fresh audit. --- ## Phase 0: Check for Prior Run (Orchestrator) Before starting the audit pipeline, check whether this guide has been audited before. 1. Check whether `{guide_dir}/assets/manifest.yaml` exists. 2. If it does: read the manifest, extract `input_sha256` and `generated_at`. 3. Re-hash the current `content.json`: `shasum -a 256 {guide_dir}/content.json | awk '{print $1}'`. 4. Compare hashes: - **Match** → no drift since the prior audit. Tell the user the audit is up to date (cite `generated_at`) and offer three options: (a) view the existing `audit-report.md`, (b) re-run a specific phase (Phase 2/3/4) only, (c) proceed with a full fresh audit anyway. - **Mismatch** → drift detected; tell the user the guide has changed since `{generated_at}`. Move prior reports to `assets/history/{generated_at}/` (preserve audit trail), then proceed with a fresh full audit. 5. If `manifest.yaml` does not exist: fresh run — proceed to Phase 1. If the user picks "view the existing report", read `assets/audit-report.md` and surface its top-level findings inline; do not re-run any phase. --- ## Phase 1: Acquire & Parse (Orchestrator) 1. Verify `{guide_dir}/content.json` exists and is valid JSON. If not, report the parse error and stop. 2. Read `{guide_dir}/manifest.json` if present (informational only — Phase 3 may reference it). 3. Walk the block tree and produce a flattened representation at `assets/audit-input.json`. Every block gets a stable path string: - Top-level blocks: `blocks[i]` - Nested in a section: `blocks[i].blocks[j]` - Nested in a conditional: `blocks[i].whenTrue[j]` or `blocks[i].whenFalse[j]` - Steps inside multistep/guided: `blocks[i].steps[k]` Shape of each entry: ```json { "path": "blocks[2].blocks[0]", "type": "interactive", "id": "open-dashboards", "action": "highlight", "reftarget": "a[data-testid='...']", "requirements": ["navmenu-open"], "objectives": [], "lazyRender": false, "skippable": false } ``` 4. Create `{guide_dir}/assets/` if it doesn't exist. 5. Determine which phases to run based on `focus`. Default: run all three. --- ## Phase 2: Structural Audit (Sub-agent) Skip if `focus` is set to a value that doesn't include structural (`requirements`, `editorial`). Dispatch a sub-agent (Explore subagent type) with the following brief: ```` You are the **Structural Audit** phase of the audit-guide skill. Read the following files in order, then produce a structured report. **Read (in order):** 1. [.cursor/commands/lint.md](../../../.cursor/commands/lint.md) — the lint checklist (this is your primary rule source) 2. [../shared/critical-rules.md](../shared/critical-rules.md) — the canonical 21 critical rules (cited by number in findings) 3. [./severity-rubric.md](./severity-rubric.md) — how to classify findings as blocking / warning / info 4. {guide_dir}/assets/audit-input.json — the flattened block tree you are auditing **Apply every check in lint.md against every entry in audit-input.json.** Also apply: - Critical rule 4 (no multistep singletons) — flag any `multistep` block with `len(steps) == 1` - Critical rule 17 (no focus-before-formfill) — flag any `interactive` with `action: "highlight"` targeting an input/textarea selector with `doIt` true or unset - Critical rule 18 (`schemaVersion` is optional but if set must be `"1.1.0"`) - Critical rule 19 (`popout` requires `targetvalue` of exactly `"sidebar"` or `"floating"`) - Critical rule 20 (`navigate` actions chaining to another guide should use `openGuide` not `?doc=`) - Critical rule 21 (virtualised targets need `guided` + `lazyRender: true`) — heuristic: any step targeting a selector containing `[data-cy='wb-list-item']`, `tr[data-row-id`, table-row selectors, or `:contains(` text-match patterns in a list context is a candidate. Flag plain `interactive` blocks with such targets. **Output: a structured markdown report at `{guide_dir}/assets/structural-report.md`** with the standard frontmatter disclaimer and these sections: ```markdown # Structural Audit ## Summary - Total findings: N (blocking: N, warning: N, info: N) - Total blocks audited: N ## Blocking For each finding: - **{path}** — {short title} - Rule: critical rule {number} OR lint.md item {number} - Detail: {one-sentence explanation} - Suggested fix: {concrete next step} ## Warning (same shape) ## Info (same shape) ``` Cite block paths exactly as they appear in `audit-input.json` (`blocks[i].blocks[j]` etc.). Cite rule numbers from the shared critical rules file. Do not invent rules. If you find zero findings in a section, omit the section header (don't write empty sections). **Stop after writing the report. Do not modify content.json. Do not run further phases — the orchestrator handles synthesis.** ```` The sub-agent writes the report. The orchestrator then proceeds to Phase 3. --- ## Phase 3: Semantic Audit (Sub-agent) Skip if `focus` is set to a value that doesn't include semantic (`structural`, `selectors`, `lazy-render` without `requirements`). Dispatch a sub-agent (Explore subagent type) with the following brief: ```` You are the **Semantic Audit** phase of the audit-guide skill. Read the following files in order, then produce a structured report. **Read (in order):** 1. [.cursor/commands/check.md](../../../.cursor/commands/check.md) — the check checklist 2. [.cursor/authoring-guide.mdc](../../../.cursor/authoring-guide.mdc) — best practices and code smells 3. [.cursor/best-practices.mdc](../../../.cursor/best-practices.mdc) — decision trees, canonical code smells (§6), quality checklist (§7) 4. [../../../AGENTS.md](../../../AGENTS.md) — canonical 21 critical rules (cited by number in findings) 5. [.cursor/docs/requirements-reference.md](../../../docs/requirements-reference.md) — for chain validation 6. [./severity-rubric.md](./severity-rubric.md) — how to classify findings 7. {guide_dir}/assets/audit-input.json — the flattened block tree **Apply (in order):** - Every code smell in best-practices.mdc §6 (15 canonical items) - Every item in best-practices.mdc §7 quality checklist - Critical rules 1-21 from AGENTS.md — flag any violation, citing the rule number - Requirements/objectives chain logic: if section N has an objective `has-datasource:X`, section N+1's steps that need X should require it. If section M has a `section-completed:S` requirement, verify section S exists and precedes M in the block tree. - Variable references: every `var-<name>:<value>` requirement must correspond to an `input` block earlier in the guide with `variableName: <name>`. Every `{{<name>}}` in markdown content must correspond to an `input` block. - First-action page anchor: the first interactive action should have `on-page:/path` or be a `navigate` action. - Tooltip quality: tooltips must be ≤ 250 chars, single sentence, and not name the element being highlighted. - Section bookends (rule 14): each `section` should have a 1-sentence intro markdown **immediately before** it and a 1-sentence summary **immediately after** it in the parent `blocks` array. Do **not** flag missing *in-section* intro when outside bookends exist. You may treat a pre-section "To …, complete the following steps:" line as satisfying the intro. - Section intro markdown that may number as a step: if the first block inside a `section` is `markdown` that starts with "You'll " / "You will " or is a one-sentence action preview before the first interactive, flag as **Warning**. Cite the block path. Note that Block Editor may number this as step 1; LP review promotes to inline after live confirm. Fix guidance: move the intro outside the section (rule 14 placement). - False noops: `action: "noop"` whose content tells the learner to click, type, open, or select UI without a `reftarget` — flag as **Warning** (prefer `markdown` or a real highlight). **Output: a structured markdown report at `{guide_dir}/assets/semantic-report.md`** with the standard frontmatter disclaimer and the same Summary / Blocking / Warning / Info section shape as the structural report. Cite rule numbers and block paths. **Stop after writing the report. Do not modify content.json.** ```` --- ## Phase 4: Adversarial Audit (Sub-agent) Skip if `focus` is `structural` or `requirements`. Dispatch a sub-agent (Explore subagent type) with the following brief: ```` You are the **Adversarial Audit** phase of the audit-guide skill. Read the following files in order, then simulate a confused user. **Read (in order):** 1. [.cursor/commands/attack.md](../../../.cursor/commands/attack.md) — the attack protocol