judge-ui

name	judge-ui
description	Drive a web UI through a defined flow, capture stage screenshots + a structured read-back, and have an independent vision-judge sub-agent render a PASS / FAIL / UNCERTAIN verdict cross-checked against the read-back — turning "look at the screen" UAT from a human task into an agent-judged one, with a mandatory fall-back to human escalation on low confidence. Use when a UAT step needs someone to drive a browser and judge what renders ("does the screen look right?"), or when user-uat's Visual tier delegates here. Project specifics (server bring-up, auth, testids) come from a project adapter. Invoke as "/judge-ui [flow-or-spec] [--adapter <name>] [--dry-run]".
user-invocable	true

UI Judge

Judging doctrine: invariants, archetypes, and the judge↔advisor spectrum live in _shared/judge-core.md — this skill instantiates them (it is the §10 reference implementation for separate-vision-judge + read-back + escalate).

The generic engine for the visual tier of UAT. It drives a real browser through a flow, captures a screenshot at each stage, and dispatches an independent vision-judge sub-agent that views the pixels, cross-checks them against a structured read-back (API JSON / DB query), and returns a per-stage verdict with evidence.

The point: "did the screen render correctly?" has always been a human's job. A browser driver plus a vision model can now do most of it — but only safely if the judgment is disciplined. This skill is that discipline. It is project-agnostic: everything project-specific (how to start the app, how to reach an authed state, which testids to click) comes from a project adapter (e.g. toybox's /uat-ui). Without an adapter it runs an inline flow spec you supply.

When to use / not

• Use: a UI flow needs driving + visual judgment — a parent dashboard renders a list, a wizard advances through screens, a form's result must look right. Or /user-uat --ui routed a visual-tier step here. Or a project adapter calls you with a named flow.
• Don't use for things a still frame can't judge: animation/motion, audio/sfx, real-device input, or subjective "does it feel right" — those stay human (a screenshot is one frame; it cannot see a countdown tick or hear a sound). Don't use it to refine a fuzzy UAT (that is /review-uat) or to run shell-only checks (that is /user-uat's mechanical tier).

Invocation

/judge-ui --adapter toybox --flow room-import   # run a named flow from a project adapter
/judge-ui <path-to-flow-spec>                    # run an inline flow spec (no adapter)
/judge-ui --adapter toybox --flow room-import --dry-run   # print the plan + the stages; drive nothing

The flow-spec contract (what a flow must provide)

A flow is the parametrized input the engine drives. A project adapter supplies the first three; the caller or the adapter supplies the rest.

1. Bring-up — how to get the app running + reachable: start command(s), a readiness probe, and the base URL. Skipped if the app is already up. Usually project-specific → adapter.
2. Reach state — how to get from cold to the flow's starting UI state: auth (login / PIN), seed fixtures, feature-flag values. Usually project-specific → adapter.
3. Read-back handle — how to read the ground truth out of band (an authed API call, a DB query) so the judge can cross-check pixels against data. Adapter provides the auth.
4. Stages — an ordered list. Each stage = an action (navigate / click / fill), a screenshot target path, optional mechanical DOM asserts (testid present, heading text, row count, URL), and an optional read-back call.
5. Rubric — per-stage natural-language pass criteria for the vision judge, in observable terms ("review table shows 7 rows; first row name is editable; a Create button is present"). Borrow these verbatim from a Verify (human) block if /review-uat produced one.

The run loop

1. Ensure the app is up. Run the adapter's bring-up (or verify an already-running instance with the readiness probe). For Windows / port-collision / parallel-session hazards, defer to the adapter — it owns the safe bring-up + teardown.
2. Drive the flow. Prefer a parametrized Playwright spec (it gives DOM asserts and screenshots in one pass; see toybox's frontend/playwright/room-import.spec.ts as the reference shape). Run the mechanical DOM/API asserts FIRST at each stage — they're cheap, deterministic, and they ground the screenshot (a green DOM assert + a screenshot beats a screenshot alone). Capture each stage's screenshot to the agreed path and call the read-back. A mechanical assert failure stops the flow and is itself a FAIL (no vision call needed).
3. Dispatch the vision-judge sub-agent. Hand a separate agent (Agent tool) the stage screenshots + the read-back JSON + the rubric + the flow's intent. It must (a) view each image, (b) cross-check what it sees against the read-back, (c) return a per-stage verdict PASS / FAIL / UNCERTAIN with the concrete evidence it used. The judge is independent eyes — never the orchestrator grading its own driving.
4. Resolve the verdict.
   • Every stage PASS and pixels agree with the read-back → PASS.
   • Any stage FAIL (with evidence) → FAIL.
   • Any stage UNCERTAIN, low confidence, or pixels disagree with the read-back → escalate to Human — degrade to today's behavior, present the evidence, ask the one-line question. Never auto-PASS through uncertainty.
5. Write the verdict doc (the durable artifact) and return a structured result to the caller. Doc structure — all six sections always present, even on an early-exit run: scope, environment, per-stage "what the judge saw" (legible observations, not vibes), the read-back summary, findings, verdict. On a fail-fast (a mechanical-gate FAIL that never reached the vision step), still emit every section — mark the unreached ones explicitly ("what the judge saw: not invoked — mechanical gate failed at "; "read-back: not reached — flow stopped") rather than dropping them, and capture a screenshot of the failing state to cite as evidence. Last line is exactly VERDICT: PASS / VERDICT: FAIL / VERDICT: ESCALATE.

The honesty invariants (do not skip — these are why it's safe)

A vision-judge is an agent grading an agent-rendered surface — the same trap operator UAT exists to prevent (agents grading agent work codify regressions: toybox G2 / the audit-wire- shape rule). These five rules keep it honest:

1. Read-back cross-check is mandatory. The judge may not PASS on pixels alone — every PASS must be corroborated by the API/DB read-back. A pretty screen over a wrong DB is a FAIL.
2. Low confidence → Human, not PASS. Uncertainty degrades to escalation. The engine's job is to remove the easy visual checks from the human's plate, not to fabricate confidence.
3. Mechanical asserts gate the judge. Deterministic DOM/API checks run first and fail fast; the vision call only judges what they couldn't.
4. The judge is a separate sub-agent. Independent eyes. The orchestrator that drove the browser does not also render the verdict.
5. Evidence on every verdict. Each PASS/FAIL cites the screenshot path and the read-back value. A verdict with no evidence is a defect (inherited from /user-uat).

Side-effecting flows (form submits, real writes, sends) inherit /user-uat's safety gate: tag them side-effectful, confirm before running, prefer an isolated fixture/DB. The adapter should run against an isolated store, never the operator's real data.

Reuse — don't reimplement

/build-step --ui already owns app lifecycle + capture primitives: dev/.claude/skills/build-step/scripts/capture_evidence.py (headless Chromium, full-page screenshots, console/HAR, project-specific --exercise-cmd) plus Windows-aware readiness-polling and taskkill/port-kill teardown. Reuse those primitives for capture/teardown rather than rebuilding the Windows signal-handling. What build-step lacks — and what this skill adds — is the judgment: a structured PASS/FAIL verdict against an acceptance rubric. Capture there; judge here.

Relationship to other skills

• /user-uat — its Visual tier (--ui) delegates here. user-uat owns the partition + the terse report; judge-ui owns the drive + the vision verdict. The UNCERTAIN → Human fallback feeds straight into user-uat's Needs you section.
• /review-uat — emits the Verify (visual — vision-judge) blocks this skill consumes; its rubric is the per-stage pass criteria. Refine → judge.
• /build-step --ui — the capture/lifecycle primitives to reuse (above).
• Project adapters (e.g. toybox /uat-ui) — supply bring-up + auth + the flow library; this engine is the project-agnostic half they call.