frontend-audit

# frontend-audit A reusable design verification loop. The user only needs to: 1. Drop goal PNGs into `design/` 2. Have a dev server running That's it. The assistant — using this skill — handles every other piece, including authoring the region descriptors. ## When to invoke - The user shares a goal PNG and asks to match the UI to it ("match this design", "iterate on this mock", "make it look like the goal") - The user calls out visual drift in a previous UI iteration ("the button is too white", "the leaves are too high") - The user asks to "validate" or "check" that the implementation matches the design ## Bootstrap — first use in a project Check whether the current working directory has `.frontend-audit.json`: ```bash test -f .frontend-audit.json && echo "configured" || echo "needs-bootstrap" ``` **If `needs-bootstrap`:** the whole setup runs eagerly in one shot — the user types `/frontend-audit` once and Claude provisions every dependency. The user should not have to think about Python, OmniParser, Playwright, or any other internal tool. ### Step A — gather config (one AskUserQuestion call, three questions) Call AskUserQuestion **exactly once with all three of these questions in the same call** — don't infer answers from the project layout, the user wants the choice: 1. **Dev server URL** — first option `http://localhost:3000` labelled `(Recommended)`, with `Other` available for custom values. 2. **Goal-image directory** — first option `design/` labelled `(Recommended)`. Offer `design/`, `mocks/`, `screenshots/` as quick options. 3. **Install everything now?** — first option `Yes, install with <pkg-manager>` labelled `(Recommended)`. Detect the lockfile (`bun.lock` → bun, `pnpm-lock.yaml` → pnpm, `yarn.lock` → yarn, else npm). Mention that "install everything" means JS deps (pngjs, playwright, chromium) **and** the OmniParser ML region detector — about 3GB on disk one-time. ### Step B — install JS dependencies On `Yes` to install: run `bun add -d pngjs jpeg-js playwright && bunx playwright install chromium` (substituting the detected package manager — `pngjs` decodes PNG, `jpeg-js` decodes JPEG so the user can drop either format into `design/`). On `Skip`: tell the user they'll need to install before continuing and stop. ### Step C — check for Python and offer to install if missing OmniParser (the ML region detector) requires Python 3.10+. Run: ```bash command -v python3 >/dev/null 2>&1 && python3 --version ``` If absent OR the reported version is `< 3.10`: - Call AskUserQuestion: "Python 3.10+ isn't installed. Install it now? (recommended; needed for high-quality region detection — without it, the skill falls back to a simpler edge-detection mode that still works but produces less semantic region names)." - Quick options: `Yes, install Python` (recommended), `Skip — use the no-Python fallback`. - On `Yes`: detect OS and run the right command: - macOS with brew: `brew install python@3.12` - macOS without brew: ask the user to install brew first, or fall back to the python.org installer link - Linux Debian/Ubuntu: `sudo apt install -y python3.12 python3.12-venv` - Linux Fedora/RHEL: `sudo dnf install -y python3.12` - Windows: send the user to `https://www.python.org/downloads/` (Claude can't reliably install Python on Windows from CLI) - On `Skip`: set `_skipOmniParser: true` in the config so Step D doesn't run, and inform the user that future PNG region authoring will use the `--programmatic` (Sobel + CCL) path. ### Step D — provision OmniParser eagerly If Python is available and the user didn't skip: ```bash bun ~/.claude/skills/frontend-audit/scripts/bootstrap-regions.mjs --setup-only ``` This creates the Python venv at `~/.claude/skills/frontend-audit/.venv/` and installs OmniParser's full dep stack (transformers, torch, ultralytics, easyocr, etc.) and triggers the model-weight download. Total ~3GB on disk, one-time. The script prints progress; relay it to the user so they know it's working. On failure (e.g. pip download blocked, disk full): tell the user the error and offer to retry, OR fall back to `--programmatic` mode for region authoring. ### Step E — write the config and confirm Write `.frontend-audit.json`: ```json { "devUrl": "http://localhost:3000", "designDir": "design" } ``` Add `design/_debug/` to `.gitignore` (idempotent). Tell the user setup is complete and they can now drop PNGs into `design/` and re-run `/frontend-audit` to start auditing. **Do not** drop a `regions.json` — regions are per-PNG and the assistant generates them in Step 0 below. **If `configured`:** read `.frontend-audit.json` so subsequent script invocations use the project's URL and paths. Skip bootstrap entirely. ## The workflow (run in order, every iteration) ### Step 0 — discover which PNGs need regions ```bash bun ~/.claude/skills/frontend-audit/scripts/discover.mjs ``` `discover.mjs` walks `designDir` (from `.frontend-audit.json`, default `design/`), hashes each PNG with SHA-256, and reports per-PNG status: - **`new`** — no sibling regions file. Regions must be generated. - **`stale`** — regions file exists but its `_meta.sourceHash` doesn't match the current PNG bytes. The image was replaced; regions are out of date. Regions must be regenerated. - **`up-to-date`** — hash matches. Skip. - **`unhashed`** — regions file exists with no `_meta` block (legacy / hand-authored). Treated as up-to-date but flagged; use `--stamp` to record the hash. The script always exits `0` on a successful inventory — `new` and `stale` are work items, not failures. Parse the printed table to decide next steps. **For each `new` or `stale` PNG**, the assistant must do the following: #### Optional shortcut — auto-bootstrap the regions Before hand-authoring, the assistant **may** invoke `bootstrap-regions.mjs` to seed the regions file with detected elements. This skips the slowest part of the manual flow (placing 20+ bboxes by eye) but never skips the human-quality pass that follows. ```bash # Default: ML-based, semantic labels. ~10s per PNG once the venv is warm. bun ~/.claude/skills/frontend-audit/scripts/bootstrap-regions.mjs --image=design/<name>.png # Fallback: pure-JS Sobel + connected-components, no Python deps. <1s per PNG. bun ~/.claude/skills/frontend-audit/scripts/bootstrap-regions.mjs --image=design/<name>.png --programmatic ``` **Default (OmniParser).** Calls `microsoft/OmniParser-v2.0` (YOLOv8 UI-icon detector + Florence-2 captioner + easyocr) via a Python subprocess. First run installs a ~3GB venv at `~/.claude/skills/frontend-audit/.venv/` and downloads ~600MB of model weights to the HF cache; subsequent runs are ~10s each on MPS, ~15–30s on CPU. Returns regions with semantic labels like `your-recent-orders`, `order-4000257090`, `view-all-orders`. License note: icon_detect is AGPL — bounded design-time exposure only, but worth knowing if your project has an AGPL policy. **Fallback (`--programmatic`).** Pure-JavaScript path using the same Sobel + connected-component primitives the audit gate uses. Runs in <1s, requires no Python, generates regions named by grid position + dimensions (`shape-mid-center-494-427`). The assistant renames them to something semantic in step 1 below. **When the default fails** (no Python, install errors, model download blocked) the script exits with a clear message pointing at the three fallback paths. There's no auto-fallback to programmatic — the choice is explicit. **Then, regardless of how the regions arrived in the file:** 1. Use the **Read tool** on the PNG — it renders the image back as visible content because the assistant is multimodal. 2. Identify ~10–25 key visual components in the image. For each, write a region descriptor with fractional coordinates (`x`, `y`, `w`, `h` as fractions of image width/height — 0..1). _If you bootstrapped, the regions already exist; your job in this step is to spot what's MISSING — bootstrap typically catches text and icons but misses card/panel containers, decorative strips, and background surfaces._ 3. Name regions semantically by what the element IS, not where. Use kebab-case. _If you bootstrapped, replace generic names like `shape-mid-center-494-427` (programmatic) or imperfect ML labels like `icon-region` with what the element actually is (`card-1-bg`, `status-pill-shipped`)._ Examples of good names: - Surfaces: `page-bg`, `main-card`, `sidebar`, `footer-strip` - Interactive: `primary-cta`, `secondary-btn`, `ghost-btn`, `submit-pill` - Text: `hero-title`, `section-eyebrow`, `body-paragraph`, `subtitle` - Decoration: `avatar-bg`, `status-icon`, `badge` 4. Pad rects a few pixels around the element edge so the shape inspector has flat fill to scan (don't size rects tight to text). 5. **Add `locator` fields** for every region that maps to a distinct interactive element (button, pill, chip, list-row, badge). Decorative or text-only regions (`page-bg`, headings) can skip the locator. Locators enable the BLOCKING gate in Step 6 to inspect live computed styles instead of sampling re-rendered pixels. A `locator` is one of these shapes: - `{ "testid": "ship-now-btn" }` — best when the codebase uses `data-testid` attributes - `{ "role": "button", "name": "Ship now" }` — ARIA role + accessible name (works regardless of class names) - `{ "text": "Ship now" }` — exact visible text (last semantic resort) - `{ "css": "main > div:nth-of-type(2) > button:first-of-type" }` — structural CSS path with NO class names (fallback when nothing semantic is available) **Prefer not to author locators by hand — use `bind-selectors.mjs` (Step 0.5 below) to generate them from the live DOM automatically.** Hand-authored class-based selectors are the documented brittleness vector this skill is fighting; the bind step generates locators that survive Tailwind class refactors, Svelte scoped-style mangling, and CSS Modules hash suffixes. 6. Write the result to `design/<name>.regions.json` (sibling to the PNG). The file should look like: ```json { "page-bg": { "x": 0.02, "y": 0.02, "w": 0.02, "h": 0.02 }, "primary-cta": { "x": 0.08, "y": 0.41, "w": 0.1, "h": 0.04, "locator": { "role": "button", "name": "Ship now" } } } ``` You don't need to write the `_meta` block — that gets stamped in the next step. The legacy `selector` field is still accepted and treated equivalently once resolved; new files should use `locator`. 7. **Bind locators to the live DOM:** ```bash bun ~/.claude/skills/frontend-audit/scripts/bind-selectors.mjs \ --regions=design/<name>.regions.json --url=<dev-path> ``` This boots Playwright, walks each region that has a `selector` (or `locator`), inspects the live DOM, and rewrites it with the most stable locator shape it can find — preferring testid → role+name → text → structural CSS path. Rect-only regions (with no selector) are skipped by default; pass `--include-rect` to also bind them via `elementsFromPoint` (speculative; review every binding). The script validates every emitted locator resolves to exactly one element before writing. 8. Run `bun ~/.claude/skills/frontend-audit/scripts/discover.mjs --stamp` once after writing all new/regenerated regions. This adds the `_meta.sourceHash` so future `discover` runs detect content changes. **Region-picking guidance:** - Always include `page-bg` — a tiny patch near the top-left corner. This anchors the shadow probe. - For each major card / panel / surface, include a region covering the whole element (radius/border/shadow probes need this). - For each interactive element (button, pill, chip, toggle), include a region covering the whole element (bg-mode probe needs the surrounding context). - For each text label / heading / accent eyebrow, include a region around the text. - For each icon avatar / decorative element with a distinct fill, include a region. **Verification:** after writing the regions file, run sample-colors with `--debug`: ```bash bun ~/.claude/skills/frontend-audit/scripts/sample-colors.mjs design/<name>.png --debug ``` This writes `design/_debug/<region>.png` crops for each region. The assistant should Read a few of those crops to verify regions landed on the right glyph. If a region missed (e.g. you placed a button rect on adjacent whitespace), refine the JSON and re-run. **Handling stale PNGs:** when `discover` reports `stale`, the existing regions file describes a different image. Two options: - **Replace** (default): regenerate the regions from scratch. The new regions overwrite the old when the assistant writes the file. Common case — the user updated the design. - **Adapt**: if the image changed only slightly (e.g. minor tweak to a button) and most regions still apply, the assistant can open the old regions file, adjust only the affected coordinates, and re-stamp. Use sparingly — it's easy to leave drift behind. The user should never have to touch the regions file — but they can if they want to. ### Step 1 — sample dominant colors from the goal ```bash bun ~/.claude/skills/frontend-audit/scripts/sample-colors.mjs <goal.png> --debug ``` Histogram clustering (5-bit-per-channel quantization + merge by RGB distance) returns the **top-3 dominant clusters per region** with hex, RGB, area %, and saturation. Why histogram, not "darkest pixel": a swatch over "Delivery by …" contains pill-bg + white text + AA edges. Darkest-1% returns the AA boundary (darker and less saturated than the real fill). Histogram returns the true pill color AND the text color in one pass. For accent labels with low pixel-area (e.g. a brick-red "NEXT CHARGE DATE" eyebrow): pick the cluster with **highest saturation**, not highest area. The sampler prints both. ### Step 2 — inspect shape tokens (radius, border, bg-mode, shadow) ```bash bun ~/.claude/skills/frontend-audit/scripts/inspect-shape.mjs <goal.png> # Or for one region: bun ~/.claude/skills/frontend-audit/scripts/inspect-shape.mjs <goal.png> --region=<name> ``` Per region: - **r-tl / r-tr / r-bl / r-br** — corner radius in goal-image pixels. Map to Tailwind: 12 ≈ `rounded-xl`, 16 ≈ `rounded-2xl`. - **bg-mode** — `ghost (Δ2)` means the rect fill matches its surrounding container → button is transparent in the goal (no `bg-…` class). `filled (Δ40 vs #fbfaf8)` means rect has its own fill. Catches the "white-filled button where goal wants outline" regression. - **border** — detected hex if ≥40% of edge samples differ from inner fill. Reports confidence % and RGB Δ. Hairline like `border-[#…]/15` reads as low confidence + small Δ; solid border reads ≥90% confidence. - **shadow↓ / shadow→** — page-bg luma minus strip-just-outside luma. Threshold `*` = 4. ### Step 3 — inspect the live render's computed styles ```bash bun ~/.claude/skills/frontend-audit/scripts/check.mjs <url-path> --selector='<css>' ``` Boots headless Chromium against the configured `devUrl`, runs `getComputedStyle()` on matched elements, and dumps: color, bg, border, radius, shadow, padding, margin, font, size, weight, letter-spacing, transform, width, height. This is the only reliable way to confirm what your Tailwind classes actually rendered. ### Step 4 — apply Tailwind changes - Sampled hex codes go in via arbitrary values: `text-[#9d573d]`, `bg-[#e1e0d7]`. Don't reach for `text-emerald-800` or other palette names unless you've confirmed they match the sampled hex. - Radius/shadow: use the class that makes intent obvious (`rounded-xl`, `rounded-2xl`, `shadow-soft`). ### Step 5 — snap the live render WITH `--goal`, then READ EVERY DIFF **This step is the whole point of the skill. It is not optional.** **Always pass `--goal=<path-to-goal.png>`** — `snap.mjs` will then auto-generate **four** diff PNGs: the full side-by-side AND three band zooms (header, middle, footer). Without `--goal`, you'll have only the bare snap, and you cannot reliably remember the goal's structure well enough to catch what's missing. ```bash # One command: snap + auto-diff (full + 3 band zooms). Always include --goal. bun ~/.claude/skills/frontend-audit/scripts/snap.mjs <url-path> \ --selector='<css>' \ --out=design/_debug/current.png \ --goal=design/<name>.png ```