verify-before-land

name

verify-before-land

description

Prove a change is interpretable AND actually works at the consumer side before landing. Extract the deliverable's surface deterministically from source (no editorializing), enumerate its state matrix, hand the raw extraction and — when a UI can render — real screenshots to blind multi-persona agents, judge against pre-written ground truth, and block on FAIL. This is the procedure behind the iron rule "green only once consumed." Use AFTER a UI- or contract-facing change and BEFORE landing. Trigger on "zero-knowledge verification", "blind test", "verify before land", "consumer-side verification", "UI verify".

Verify before land

Verify that every user- and consumer-facing element a change introduces is correctly interpretable by someone who has never seen this product, and that it actually works where it is consumed — not only where the library is defined. This is the concrete procedure behind the iron rule in CLAUDE.md: a change is green only once consumer-side integration passes.

The exact extraction commands (which files hold routes, which symbol is the API single-source-of-truth, how to launch the running product) are project-specific — see the project CLAUDE.md. This skill is the method; the project supplies the verbs.

Three layers

A complete verification runs all three. If a layer is unavailable, record it as an explicit gap — never relabel a skipped layer as "out of scope."

Layer 0 — Consumer integration (the gating layer)

The library being green proves nothing on its own. Find every consumer of the change (the screen, the endpoint, the other package that imports it) and prove the change works there: the published API resolves, aliases / tokens resolve, the real screen does not break, the request round-trips. A library that no consumer exercises is unverified. If the change has no consumer yet, building the consumer is part of this verification, not a "next PoC."

Layer 1 — Structural (always available)

Extract the deliverable's surface deterministically from source — no agent, no human editorial judgment — and feed the raw extraction to blind agents.

Layer 2 — Visual (when the product can render)

Launch the running product, capture actual screenshots, and present the images to blind agents. This tests what Layer 1 cannot: visual hierarchy, spacing, color, theme compatibility, information density, where the eye is drawn.

Layer 1: Structural extraction

Method

Run a deterministic transformation that parses source and emits raw JSON. No agent is involved in the extraction — the output is a mechanical transform of the source, not a description. Template literals are reproduced with ${...} placeholders intact; a conditional value carries its condition. Ugly is fine, accuracy matters.

Extract the surfaces a never-seen user or consumer must interpret. Depending on what the product exposes, that typically means some of:

• API contract — the single source of truth for paths (whatever the product names it), each endpoint's method, the request/response DTO shape. A consumer reads this; it must be self-explanatory.
• Routes & navigation — route paths and their titles/labels; what each view is named.
• Actions & labels — button / command / menu titles, their grouping, their enablement conditions.
• Tables & lists — column headers, the value expression per cell, the empty state.
• Feedback — toast / notification / error templates (exact text, with ${...} intact) and the actions they offer.
• Forms — field labels, placeholders, validation messages.

Feed the raw extraction directly to the blind agents. Do NOT restructure, filter, or annotate it.

State matrix (fold every domain state in)

A surface is only as verified as its least-common state. Enumerate the full matrix: cross each domain entity's states (every status / variant a value can take) with each place it renders. For every cell, confirm there is a defined label/badge, a defined action set, a defined empty/zero state, and a sensible fallback (no silent ?, no opening a non-existent ref). The gaps this surfaces — a status with no label, a view with no empty state, a missing welcome state — are themselves findings to fix before landing.

What this tests / does not test

Tests: label clarity, structural logic, naming consistency, information completeness, state-matrix coverage. Does not test: visual hierarchy, rendering fidelity, color, spacing, real-scale density — that is Layer 2.

Layer 2: Visual capture

Build and launch the product (for this template, make fixture bootstraps a deterministic sandbox and make dev runs the stack), populate it with the fixture's real data, and capture each meaningful state as an image. Capture multiple states, including the empty / zero state and the error / busy state, not only the happy path. Present each screenshot to the blind agents as an image (Claude agents read images) and ask the same interpretability questions.

When the environment cannot render (headless CI, remote SSH), state explicitly: "Layer 2 (visual capture) was not performed — visual rendering, spatial relationships, and density are unverified." That is a gap, not a design choice.

Blind testing procedure

Agents

Run 3 concurrent agents with distinct personas, e.g.:

• Newcomer — competent with the platform, never seen this product.
• Designer — a UX evaluator, not a domain expert.
• Power user — evaluates it as a replacement for what they use today.

Give each the raw extraction (Layer 1) and/or the screenshots (Layer 2). Add no explanatory context — the agent must work from exactly the information a real user or consumer would have.

Output per agent

{
  "elements": { "<id>": { "does": "...", "confidence": 1-5, "confusion": "..." } },
  "overallConfusion": ["..."],
  "redundancies": ["..."]
}

Judge

A fourth agent compares the responses against ground truth:

• PASS — all 3 correct
• PARTIAL — 1–2 uncertain
• FAIL — majority wrong, or anyone confidently wrong

Ground truth

Write it before running the test. It states what each element actually does — not what you wish it did — and is not adjusted after seeing results.

Blocking criteria

• Any Layer 0 consumer that does not actually work = red, fix before landing. Report the change as red until consumer-side integration passes.
• Any FAIL = fix before landing.
• Any confident misinterpretation = fix before landing.
• A state-matrix gap (unhandled status, missing empty state) = fix before landing.
• A Layer 2 gap = note it in the commit message and schedule the visual pass; do not silently drop it.

Output

A verification record in reports/<purpose>/README.md: the ground truth, the extraction, the per-agent results, the judge verdict, and any gaps with their fixes. Deliver the fixes via the isolate-and-integrate / fix-as-poc flow.