whole-dashboard-factcheck-via-parallel-cluster-agents

name	whole-dashboard-factcheck-via-parallel-cluster-agents
description	Audit a whole live data-dashboard (or any multi-page user-facing product) for (1) numbers correctness against the source database and (2) language fit for stated audiences, in one synchronous ~30–45 minute round of parallel research subagents. Use when: (1) the user says "review this dashboard / site / app", "fact-check the numbers", "make sure the copy lands for X audience", or "do a whole-X audit"; (2) the product has 6+ user-facing pages backed by a queryable data source (BigQuery payload table, Postgres, read-replica, JSON cache, etc.); (3) the deliverable is a single written report the user can scan + decide what to fix; (4) the user has at least one written voice/persona doc (`PRODUCT.md`, brand guidelines, audience brief). Provides the cluster-split methodology (3–4 page-clusters by audience/purpose), the per-page diagnostic table schema (Element / Source / Claimed / Actual / Verdict), the subagent prompt template, the synthesis structure (TL;DR P0 table + per-page detail + cross-cutting themes), and the audit→issues→bundles PR flow. Different from `client-readiness-pass-on-rendered-html` (single page, sanitize), `document-review` (plan/requirements docs with persona agents), `agent-review-panel` (adversarial debate), and `dashboard-metric-label-vs-sql-definition` (single-metric semantics audit). This is the whole-product version: many pages, many numbers, two-audience copy lens, all in one report.
author	wan-huiyan + Claude Code
version	1.0.0
date	"2026-05-08T00:00:00.000Z"

Whole-dashboard fact-check + language audit via parallel cluster agents

Problem

User wants confidence that a whole live product is honest with its data and lands with its audiences before a leadership review / client demo / Phase 2 handover. Asking page-by-page synchronously takes hours; asking a single agent to cover 14 pages produces shallow output and runs out of context. A naive "review this dashboard" prompt produces a generic-sounding bullet list that misses the load-bearing problems (wrong numbers, broken anchors, jargon leaks).

The right shape is: split by audience-purpose into 3–4 page-clusters, dispatch one research subagent per cluster with explicit data-source access + output schema, then synthesize one deliverable.

Context / Trigger Conditions

Use when ALL of:

1. The product has 6+ user-facing pages/routes/screens backed by a queryable data source (BigQuery, Postgres, JSON cache, payload table). Single-page reviews use simpler skills.
2. The audit needs to span both correctness ("the numbers match the database") AND fit ("the copy lands for these specific audiences"). Either alone is a different skill.
3. There's a stated voice/persona anchor: PRODUCT.md, brand guidelines, audience brief. Without one, the language portion has no ground truth.
4. The deliverable is a report-only first, not "fix everything immediately" — let the user triage.
5. You have non-trivial data access (BQ project, DB read-replica, payload table) so subagents can probe live values, not just reason about templates.

NOT for: single-page polish (use client-readiness-pass-on-rendered-html), plan/requirements docs (use document-review), adversarial debate on a design (use agent-review-panel), single-metric semantics (dashboard-metric-label-vs-sql-definition).

Solution

Step 1: Discover the surface area

# Map every user-facing route + the data sources behind them
grep -nE "@app\.route|@bp\.route" <app-entry>.py | head -40
ls <templates-dir>/*.html | wc -l
grep -nE "FROM \`|cached_payload\(" <app-entry>.py | head -30

Also identify:

• The payload / cache table that templates read from (e.g. dashboard.dashboard_payloads_v1)
• The source-of-truth tables that the payload is derived from
• The freshness model (cron bake? live query? mixed?)
• The PRODUCT.md equivalent (read it; do not skip)

Step 2: Cluster the pages by audience-purpose (3–4 clusters)

Don't cluster by URL prefix. Cluster by what each page is for. Typical splits for a data dashboard:

• A — Headline / executive surfaces (overview, monitor, drivers, top-of-funnel). Read by leadership skim; loaded with KPIs that exec will quote.
• B — Operational / action surfaces (worklists, action lists, nudge queues, pipeline-health). Read by daily users (counselors, analysts, ops); contain cohort sizes that will be exported and acted on.
• C — Library / methodology / explorer (drilldown, signals, methods, findings, per-row explorer). Highest jargon density; rigor-proof surface. Read by analysts + power users.

Three is usually enough; four if the surface is large. More clusters means thinner per-page coverage.

Step 3: Dispatch one subagent per cluster — same prompt template

Each subagent gets a self-contained prompt with these blocks:

1. CONTEXT — product purpose + audiences (from PRODUCT.md verbatim)
2. DATA ACCESS — DB project name, payload table FQN, source tables, query examples
3. FILES — exact paths to templates + route handlers + payload accessor
4. PAGES IN SCOPE — list (don't let the agent guess)
5. PER-PAGE TASKS:
   a. Fact-check rendered numbers — catalog every number, classify as
      payload-derived / hardcoded / mock-data; for top 5–8 headline numbers
      re-derive from raw source tables and compare
   b. Language audit — jargon, caveats, headline framing, action verbs,
      em-dashes, terminology consistency — for BOTH stated audiences
   c. Per-page verdict (Ready / Needs fact-check / Needs language polish / Both) + top 3 fixes
6. OUTPUT FORMAT — Markdown:
   per-page: ### header → Numbers table → Language table → Verdict block
   end-of-cluster: cross-cluster themes
7. CONSTRAINTS — query budget cap, time-box (15–20 min), DO NOT EDIT FILES

The diagnostic table schema is the load-bearing part:

| Element | Source | Claimed | Actual (BQ) | Verdict |
|---------|--------|---------|-------------|---------|
| ...     | payload-derived / hardcoded / mock-data / route-computed | what page renders | what BQ says | ✅ / ⚠️ / ❌ |

This format makes mismatches scannable + makes "what to fix first" mechanical.

Dispatch in a single Agent tool call with multiple invocations so they run truly in parallel, not sequentially.

Step 4: Synthesize the report

Single markdown deliverable, conventional path: docs/reviews/<YYYY-MM-DD>_<product>_factcheck_language_audit.md. Structure:

# <Product> fact-check + language audit — <date>

**Scope / Method / Audiences** — 3 short paragraphs.

## TL;DR — what to fix first

### P0 — fact-check (numbers wrong on the page)
| # | Page | Element | Claimed | Actual | Severity |

### P0 — language (counselor / audience-facing jargon leak)
| # | Page | Current | Why it bites | Suggestion |

### P0 — IA / structural
| # | Issue | Recommendation |

## Per-page detail
### /<route1>
  Numbers / Language / Verdict
### /<route2>
  ...

## Cross-cutting themes
- ...

## What's solid (worth saying)
- ...

## Suggested fix-order if a single PR is cut from this audit
1. ...

## Reproducibility — SQL probes used
```sql ...

The "What's solid" section matters: it prevents the report from reading as pure complaint and gives the user a credibility anchor for the negative finds.

### Step 5: Audit → issues → fix bundles flow

If the user asks "file these and start fixing":

1. **Ship the audit doc as PR-1 (report-only, docs label).** Lands fast, nothing breaks.
2. **File one GitHub issue per P0 item.** Apply ≥1 standard label at creation (per the project's labeling convention). Title each issue so the symptom is grep-able from the dashboard ("[P0] /actions A1 tile renders ~930 but live cohort=94").
3. **Bundle fixes into 2–4 PRs**, NOT one mega-PR and NOT N mini-PRs:
   - Bundle by *kind*: "wiring fixes", "jargon cleanup", "copy edits" — each PR has a coherent review story
   - Each fix PR closes its bundle of issues; format as one keyword per issue (`Closes #X. Closes #Y.`) — see `gh-squash-merge-closes-only-one-issue` for why comma-list silently fails
4. **Merge each PR as CI lands**; rebase the next branch off updated main. The audit doc PR is the spine; fix PRs are spokes.
5. **Tracker entry** captures the whole arc (audit + N fix PRs + N issues) under one tracker key — saves future-you reconstructing the timeline.

## Verification

The audit's value is verifiable post-hoc:

- **For each P0 fact-check item**: BQ probe in the report should reproduce the "Actual" column today.
- **For each fact-check fix PR**: byte-grep the deployed build tarball (per `deploy-from-stale-worktree-silent-rollback` Step 2) for the fingerprint string from the fix.
- **For each language fix**: the literal string from the audit's "Suggestion" column should appear in the deployed build.
- **Cluster coverage**: sum of pages-audited across cluster reports should equal the surface area discovered in Step 1. Any page silently missing means a cluster split was wrong.

## Example

**Setup** (university-admissions propensity dashboard, 2026-05-07): 14 user-facing routes, BigQuery payload table `<project>.<dataset>.dashboard_payloads_v1`, PRODUCT.md defining "leadership skim + counselors act" two-audience voice.

**Cluster split:**
- A: `/`, `/monitor`, `/drivers` (executive headline + driver detail)
- B: `/actions`, `/nudges`, `/pipeline-health` (operational worklists)
- C: `/library/*`, `/intelligence`, `/student-explorer` (rigor + per-row)

**Round 1 (parallel agents, ~15 min each):** three subagents returned per-cluster reports. Each found a mix of fact-check and language issues. Synthesis pulled the highest-impact items into a TL;DR.

**Findings highlights:**
- 5 metric tiles rendering hardcoded literals 2.5–10× off live cohort sizes (template never wired to existing baked payload)
- 9,747 / ~325 / N=~8,043 hero stats unverifiable
- "162 input signals" stale (current model has 170)
- `/nudges` redirected to non-existent `#students` anchor (silent no-op)
- AI Insights LLM mislabeled `funnel_stage='In Process'` as "submitted applications" + inverted mix-shift mechanism direction
- "Avg SHAP" / "sentinel" / "Propensity breakdown" / "Base value" jargon on counselor-most-clicked surfaces
- "/library/diagnostics" page-title vs sidebar 3-way naming split

**Workflow:** audit doc PR (report-only) → 9 issues filed (with labels) → 3 fix bundle PRs (wiring / jargon / copy+prompt) → tracker entry. All 5 PRs merged the same day. 2 sister skills extracted (`template-hardcoded-literal-vs-existing-payload`, `gh-squash-merge-closes-only-one-issue`).

**Total time:** ~3 hours from "review this dashboard" to deployed-and-verified fixes.

## Notes

- **Cluster size matters.** 3 clusters × ~5 pages each is the sweet spot for a 14-page dashboard. 4 clusters × 3-4 pages works for 12-16 pages. 5+ clusters means each agent has too thin a brief and the cross-cluster synthesis becomes a coordination problem.

- **Subagent prompts MUST include the output schema.** A natural-language "audit this for numbers and language" produces unstructured prose that doesn't synthesize cleanly. The Element / Source / Claimed / Actual / Verdict table is the synthesis-enabler.

- **Don't dispatch a 4th "synthesize across clusters" subagent.** Synthesis benefits from knowing the user's voice + product priorities; it's a main-context task, not a delegate-able one. The 3 cluster agents do the data-lifting; the main agent does the editorial judgment.

- **Always include a "What's solid" section.** Pure-negative reports erode trust in the audit's calibration. Naming what works ("baked KPIs match BQ exactly", "diagnostics floor logic correct", "AI Insights numbers are arithmetically right — only the labels are wrong") makes the negative finds credible.

- **Sister skills in this toolkit:**
  - `template-hardcoded-literal-vs-existing-payload` — diagnostic for the specific "consumer-not-wired-to-producer" pattern (the most common fact-check fix-shape produced by this audit; extracted FROM this audit)
  - `dashboard-metric-label-vs-sql-definition` — for any single metric whose definition is questioned during the audit
  - `gh-squash-merge-closes-only-one-issue` — operational gotcha when filing multi-issue close PRs (extracted FROM this audit)

- **Related skills outside this toolkit:**
  - `client-readiness-pass-on-rendered-html` — for single-page sanitize-for-client work (different shape; use that one for single-page client polish)
  - `overnight-review-client-delivery` (in [`wan-huiyan/overnight-workflows`](https://github.com/wan-huiyan/overnight-workflows)) — autonomous overnight version when the client deliverable already exists and needs polish, not when you're auditing a live product
  - `deploy-from-stale-worktree-silent-rollback` — when shipping the fix bundles to production

- **The audit doc itself is reusable** as the source-of-truth for follow-up work. Reference it in tracker entries, in fix-PR bodies, in handoff docs. Keep it editable: add a "Status update" footnote at the top after each fix PR merges so future readers see what's closed vs open.

- **AI-generated content (LLM narrative blocks, generated insights) is the highest fact-check risk.** The numbers usually compute correctly; the labels and mechanism descriptions are where hallucination lands. Always include LLM-generated payloads in the audit even if they live on pages outside their immediate route (e.g. `ai_insights_cards` baked into a separate payload but rendered on /overview).

## References

- [`template-hardcoded-literal-vs-existing-payload`](../template-hardcoded-literal-vs-existing-payload/SKILL.md) — fix-shape for the most common fact-check finding produced by this audit
- [`dashboard-metric-label-vs-sql-definition`](../dashboard-metric-label-vs-sql-definition/SKILL.md) — single-metric depth audit, sister to this whole-product audit
- [`gh-squash-merge-closes-only-one-issue`](../gh-squash-merge-closes-only-one-issue/SKILL.md) — operational gotcha when running the audit→issues→bundles flow
- [`wan-huiyan/overnight-workflows`](https://github.com/wan-huiyan/overnight-workflows) — autonomous overnight versions of polish (`overnight-review-client-delivery`) and ah-ha discovery (`overnight-insight-discovery`)