# vlmkit
<p align="center"><img src="docs/assets/readme-hero.webp" width="100%" alt="The same web page on a phone, a tablet and a desktop monitor; a magnifying glass over the desktop finds overlapping text outlined in red, while a robot and a person tick off a checklist beside them." /></p>
Deterministic verification toolkit for frontend work: markup gates
(broken-page scan, copy fidelity, responsive boundaries, keyboard
operability, scripted flows), visual regression over time, design
audits, and repair tools. Built for humans and coding agents alike —
every failing gate prints a machine-parsable fix list. Everything is
key-free unless marked `[key]`.
Requires Node 24+.
## Quickstart — visual analysis with zero context
```bash
npm install -D @mizchi/vlmkit
npx playwright install chromium # once
# 1. find concrete visual defects on the page you're working on
# (no reference, no config, no API key — 3 viewports at once)
npx vlmkit check integrity http://localhost:3000/
# -> verdict: DEFECTS … [page-overflow-x] @768: … main > p (right edge 924px)
# [text-collision] @1280: "Total: €1,240" overlaps "Refunds: €80"
# fix what it names, re-run until: verdict: CLEAN
# long-polling page: choose an earlier navigation milestone
npx vlmkit check integrity http://localhost:3000/ --wait-until domcontentloaded
# 2. track what your edits change visually (1st run = baseline, then diffs)
npx vlmkit snapshot http://localhost:3000/ --output .vlmkit/snapshots
# -> desktop 13.11% … mobile 6.57% + diff heatmaps next to the report
# intended? npx vlmkit snapshot approve --output .vlmkit/snapshots
```
From there: `check copy --manifest` (exact copy), `check breakpoints
--sweep` (responsive boundaries), `check interactions` (keyboard
operability), `verify markup --target design.png` (match a design) —
the full task-routing table, done-condition recipes, and agent
integration (MCP server + `markup-assist` skill) live in
**[`docs/markup-assist.md`](./docs/markup-assist.md)**.
## Demos
**[mizchi.github.io/vlmkit/demos/](https://mizchi.github.io/vlmkit/demos/)** — every feature on a page with a
known defect: the command, its whole output, and screenshots from the same run. Most pages are this
repository's own test fixtures, so each command works from a clone after `pnpm build`.
<!-- demos:start — generated by examples/demos/render.mjs -->
| | Demo | Command |
|---|---|---|
| <a href="https://mizchi.github.io/vlmkit/demos/integrity/"><img src="examples/demos/integrity/shot-1.webp" width="220" alt="" /></a> | <img src="docs/assets/icons/page-check.webp" width="20" height="20" alt="Is the page broken?" /> [A pricing page that breaks on smaller screens](https://mizchi.github.io/vlmkit/demos/integrity/) | `check integrity` |
| <a href="https://mizchi.github.io/vlmkit/demos/copy/"><img src="examples/demos/copy/shot-1.webp" width="220" alt="" /></a> | <img src="docs/assets/icons/page-check.webp" width="20" height="20" alt="Is the page broken?" /> [Copy that drifted from the spec](https://mizchi.github.io/vlmkit/demos/copy/) | `check copy --manifest` |
| <a href="https://mizchi.github.io/vlmkit/demos/responsive/"><img src="examples/demos/responsive/shot-1.webp" width="220" alt="" /></a> | <img src="docs/assets/icons/page-check.webp" width="20" height="20" alt="Is the page broken?" /> [A card grid that needs 826px, switched on at 768px](https://mizchi.github.io/vlmkit/demos/responsive/) | `check responsive` |
| <a href="https://mizchi.github.io/vlmkit/demos/interactions/"><img src="examples/demos/interactions/shot-1.webp" width="220" alt="" /></a> | <img src="docs/assets/icons/keyboard.webp" width="20" height="20" alt="Can it be operated?" /> [A Save button that does nothing on Enter](https://mizchi.github.io/vlmkit/demos/interactions/) | `check interactions` |
| <a href="https://mizchi.github.io/vlmkit/demos/focus-order/"><img src="examples/demos/focus-order/shot-1.webp" width="220" alt="" /></a> | <img src="docs/assets/icons/keyboard.webp" width="20" height="20" alt="Can it be operated?" /> [A toolbar whose Tab order runs backwards](https://mizchi.github.io/vlmkit/demos/focus-order/) | `check a11y focus` |
| <a href="https://mizchi.github.io/vlmkit/demos/grounding/"><img src="examples/demos/grounding/evidence-1.webp" width="220" alt="" /></a> | <img src="docs/assets/icons/keyboard.webp" width="20" height="20" alt="Can it be operated?" /> [A checkout button an agent would miss](https://mizchi.github.io/vlmkit/demos/grounding/) | `check grounding --mark` |
| <a href="https://mizchi.github.io/vlmkit/demos/contrast/"><img src="examples/demos/contrast/shot-1.webp" width="220" alt="" /></a> | <img src="docs/assets/icons/contrast.webp" width="20" height="20" alt="Can everyone read it?" /> [Pale text that fails WCAG contrast](https://mizchi.github.io/vlmkit/demos/contrast/) | `check a11y contrast` |
| <a href="https://mizchi.github.io/vlmkit/demos/a11y-tree/"><img src="examples/demos/a11y-tree/evidence-1.webp" width="220" alt="" /></a> | <img src="docs/assets/icons/contrast.webp" width="20" height="20" alt="Can everyone read it?" /> [An app with no DOM to read](https://mizchi.github.io/vlmkit/demos/a11y-tree/) | `scan a11y → check a11y tree` |
| <a href="https://mizchi.github.io/vlmkit/demos/composition/"><img src="examples/demos/composition/shot-1.webp" width="220" alt="" /></a> | <img src="docs/assets/icons/layout-grid.webp" width="20" height="20" alt="Does it look designed?" /> [Headings that sit closer to the section above](https://mizchi.github.io/vlmkit/demos/composition/) | `check composition` |
| <a href="https://mizchi.github.io/vlmkit/demos/color/"><img src="examples/demos/color/shot-1.webp" width="220" alt="" /></a> | <img src="docs/assets/icons/layout-grid.webp" width="20" height="20" alt="Does it look designed?" /> [Links you can only tell apart by colour](https://mizchi.github.io/vlmkit/demos/color/) | `check color` |
| <a href="https://mizchi.github.io/vlmkit/demos/tokens/"><img src="examples/demos/tokens/shot-1.webp" width="220" alt="" /></a> | <img src="docs/assets/icons/layout-grid.webp" width="20" height="20" alt="Does it look designed?" /> [Spacing and radii off the design scale](https://mizchi.github.io/vlmkit/demos/tokens/) | `check tokens` |
| <a href="https://mizchi.github.io/vlmkit/demos/diff/"><img src="examples/demos/diff/evidence-1.webp" width="220" alt="" /></a> | <img src="docs/assets/icons/compare.webp" width="20" height="20" alt="What changed?" /> [A header that grew 48px and pushed everything down](https://mizchi.github.io/vlmkit/demos/diff/) | `diff html` |
| <a href="https://mizchi.github.io/vlmkit/demos/i18n/"><img src="examples/demos/i18n/shot-1.webp" width="220" alt="" /></a> | <img src="docs/assets/icons/compare.webp" width="20" height="20" alt="What changed?" /> [Buttons that overflow when the copy gets longer](https://mizchi.github.io/vlmkit/demos/i18n/) | `stress i18n` |
| <a href="https://mizchi.github.io/vlmkit/demos/zoom/"><img src="examples/demos/zoom/view.webp" width="220" alt="" /></a> | <img src="docs/assets/icons/zoom.webp" width="20" height="20" alt="Vision models" /> [Letting a vision model zoom into the original](https://mizchi.github.io/vlmkit/demos/zoom/) | `@mizchi/vlmkit-ai/zoom.ts` |
<!-- demos:end -->
Six whole sites built by agents, each beside the log of how it was judged, are at
[/sites/](https://mizchi.github.io/vlmkit/sites/).
## Setup
```bash
npm install -D @mizchi/vlmkit # or -g
npx playwright install chromium # once
```
vlmkit declares `playwright` as a peer and `@playwright/test` as an optional
peer, reusing the versions already selected by the project instead of
installing a second browser build.
- **Coding agents (MCP)** — `.mcp.json`:
`{ "mcpServers": { "vlmkit": { "command": "npx", "args": ["-y", "@mizchi/vlmkit", "mcp"] } } }`
- **Agent skill** — install the automatic `vlmkit` router once with APM or the
open `skills` CLI (commands below), then ask for the result in natural language.
- **API keys** — only for `[key]` features (`heal markup`,
`check copy --vlm`, fix-loop): `OPENROUTER_API_KEY` /
`GEMINI_API_KEY` / `ANTHROPIC_API_KEY`.
- **Snapshot / CI config** — `vlmkit.config.json` (routes, thresholds)
via `vlmkit workflow init`.
Full setup detail: [`docs/configuration.md`](./docs/configuration.md).
### Install agent skills
Recommended: install the `vlmkit` skill package once. You do not choose a
specialized skill; the agent classifies each frontend request, loads the
matching bundled workflow, runs its deterministic gates, fixes failures, and
reruns to green.
```bash
# Install or update APM (macOS / Linux)
curl -sSL https://aka.ms/apm-unix | sh
# Install vlmkit with APM
apm install mizchi/vlmkit
# Or install with the skills CLI (APM is not required)
npx skills add mizchi/vlmkit
```
Inside Claude Code the same package installs as a plugin — no shell, and
`/plugin marketplace update` picks up new workflows:
```
/plugin marketplace add mizchi/vlmkit
/plugin install vlmkit@vlmkit
```
The APM bootstrap above is the [official recommended installer](https://microsoft.github.io/apm/getting-started/installation/)
and resolves the latest binary for the current platform. It avoids depending on
a stale package-manager formula.
Then ask naturally: “implement this mock,” “check this page's responsive and
keyboard behavior,” or “turn this story into stable Playwright tests.” There is
no skill-selection step in the normal workflow. On first use, the agent detects
the repository's package manager, reuses or adds `@mizchi/vlmkit` locally, and
installs Chromium only if a selected gate reports it missing.
All three installers expose one visible `vlmkit` skill. The 13 specialized
workflows are internal resources bundled under that entry, so the agent selects
them without adding 13 separate skills or copying the vlmkit source repository.
See the [agent skill catalog](./.claude/skills/README.md) for all 13
specialized skills, grouped by general verification, UI creation, test
generation, comparison and monitoring, and evaluation and hardening.
Explanation and figures (animations, D2 diagrams and slides)
live in [mizchi/explainer](https://github.com/mizchi/explainer).
## When to use what
`vlmkit --help` prints this same map. Everything is key-free unless
marked `[key]`.
| You want to… | Reach for |
|---|---|
| <img src="docs/assets/icons/page-check.webp" width="20" height="20" alt="" /> Check the page you just wrote/edited, no reference | `check integrity` (broken-page scan, 3 viewports) · `check copy --manifest` (copy present, visibly, verbatim) · `check layout --contract` · `scan scroll` / `scan handlers` |
| <img src="docs/assets/icons/responsive.webp" width="20" height="20" alt="" /> Verify behavior, not pixels | `check breakpoints --sweep` · `check responsive` (every width, shrunk to breakpoint + declaration) · `check interactions` · `check scroll` / `check animation` · `verify flow --flow` |
| <img src="docs/assets/icons/pointer.webp" width="20" height="20" alt="" /> Drive the page from a screenshot (computer use) | `check grounding` — action map in screenshot px, which element a click at each point actually reaches, `--mark` for a numbered overlay |
| <img src="docs/assets/icons/target.webp" width="20" height="20" alt="" /> Match a target design | `verify markup --target` (done verdict + fix list) · `build page` / `build component` · `scan mock` (normalize @2x exports) |
| <img src="docs/assets/icons/history.webp" width="20" height="20" alt="" /> Track changes over time (VRT) | `snapshot` (baseline → per-viewport diff → `snapshot approve`) · `watch` · `diff-pr` · `baseline` |
| <img src="docs/assets/icons/compare.webp" width="20" height="20" alt="" /> Compare two versions | `diff html\|png\|elements` · `migration compare` · `check equivalence --region` |
| <img src="docs/assets/icons/layout-grid.webp" width="20" height="20" alt="" /> Audit design quality | `check tokens\|theme\|palette` · `check design` (is the page consistent with itself — one button style or six) · `check a11y contrast\|touch\|focus` · `check perf` · `check drift` · `stress i18n\|media` |
| <img src="docs/assets/icons/image-check.webp" width="20" height="20" alt="" /> Vet an image asset before it enters a slot | `check asset --slot --expect-transparent --against-bg --page-palette` |
| <img src="docs/assets/icons/wrench.webp" width="20" height="20" alt="" /> Repair | `heal selector` (dead selector) · `heal markup` `[key]` (LLM auto-fix from a kickback) |
| <img src="docs/assets/icons/stack.webp" width="20" height="20" alt="" /> Run gates over a whole site | `batch --gate "check integrity" "routes/**/*.html"` (parallel, `--shard i/n`, per-job timing) · `gates run` (same, from one reviewed config) |
| <img src="docs/assets/icons/bell-off.webp" width="20" height="20" alt="" /> Audit what has been silenced | `gates suppressions` — every suppression with reason, owner, expiry; expired ones stop applying |
| <img src="docs/assets/icons/plug.webp" width="20" height="20" alt="" /> Wire into agents / pipelines | `mcp` (gate tools over stdio) · `contract` · `workflow` · `markup-loop` · `api` · `bench` · `skill` |
Task-routing recipes and done-condition sets:
[`docs/markup-assist.md`](./docs/markup-assist.md).
## With a key: TypeSafe Jev
Four tools ask [TypeSafe Jev](https://docs.typesafe.ai) typed questions — a `box`, a yes/no (`noul`),
a `choice` — with `TYPESAFEAI_API_KEY`. The model says **what** is there and **where**; the
values come from the pixels and the page. It is never asked to name a measurement.
Jev is opt-in. Where a command works without it, `--jev` turns it on (`diff png --jev`,
`inspect motion --jev`). `inspect locate`, `inspect claims` and a `describe` click have no other
engine, so they need the key and nothing else.
| Feature | Command | Jev answers | vlmkit measures | Measured |
|---|---|---|---|---|
| Classify each diff region | `diff png --jev` | A `choice` per region: text, colour, position or presence | The regions themselves, from the pixel diff | Text 14/14, colour 18/18, presence 17/17; 1–6px shifts 11/16, every miss marked `uncertain` |
| Find an element from a description | `inspect locate`, `click` with `describe` | Is it shown (`noul`), a `box`, then a `box` again on a magnified crop | The crop, cut from the original pixels; the point, in click coordinates | 94% of targets hit; 0 of 85 decoys accepted |
| Check statements about a screen | `inspect claims --forbid/--expect` | One `noul` per statement, all in one request | Nothing: it advises and never gates | 54 of 55 planted problems caught; 0 false alarms on 43 clean pages |
| Is an animation visibly broken? | `inspect motion --jev` | Per frame: is each target shown, its `box`, a `noul` per forbidden statement | Frames on the held clock; jumps, snap-backs and easing fitted from the boxes; with `--selector`, geometry from the page | 7/7 broken caught, 7/7 intact passed (deterministic gates: 3/7); ~$0.002 a page. Without `--jev`, from `--selector` alone and no key: 6/7 and 7/7 (text left on text needs Jev) |
Asked to name an easing, Jev was at chance (7 of 21); fitted from its boxes, 21 of 21. Reports:
[region kinds](./docs/reports/2026-10-16-jev-region-kind.md),
[locate](./docs/reports/2026-10-16-jev-locate.md),
[claims](./docs/reports/2026-10-17-jev-claims.md),
[easing](./docs/reports/2026-10-18-jev-easing.md),
[motion](./docs/reports/2026-10-18-jev-motion.md).
## The workspace
```mermaid
flowchart BT
subgraph L0["layer 0"]
p7["judge"]
end
subgraph L1["layer 1"]
p4["core"]
end
subgraph L2["layer 2"]
p1["ai"]
p2["animation-eval"]
p3["capture"]
end
subgraph L3["layer 3"]
p5["generate"]
p6["heal"]
p8["markup"]
p10["plan"]
end
subgraph L4["layer 4"]
p9["mcp"]
end
subgraph L5["layer 5"]
p0["vlmkit (cli)"]
end
p0 --> p1
p0 --> p2
p0 --> p3
p0 --> p4
p0 --> p5
p0 --> p6
p0 --> p7
p0 --> p8
p0 --> p9
p0 --> p10
p1 --> p4
p1 --> p7
p2 --> p4
p2 --> p7
p3 --> p4
p4 --> p7
p5 --> p1
p6 --> p1
p6 --> p3
p6 --> p4
p8 --> p1
p8 --> p2
p8 --> p3
p8 --> p4
p8 --> p7
p9 --> p4
p9 --> p8
p10 --> p1
```
Eleven packages, layer by layer from `judge` (no dependencies) to the CLI; an arrow points from a
package to one it depends on. Generated from the manifests by `scripts/workspace-map.mjs`,
and a test fails when this block drifts from them.
## Examples
| | Example | What it shows |
|---|---|---|
| <img src="docs/assets/icons/gallery.webp" width="24" height="24" alt="" /> | [Feature demos](./examples/demos/) | One page per feature: a page with a known defect, the command that finds it, its output and screenshots |
| <img src="docs/assets/icons/stack.webp" width="24" height="24" alt="" /> | [Demo sites](./examples/sites/) | Six sites built by agents with vlmkit's gates, each published beside the log of how it was judged |
| <img src="docs/assets/icons/component.webp" width="24" height="24" alt="" /> | [Component-focused VRT](./examples/story-gallery/) | `check story` on one component instead of a whole page, with a gallery to copy |
| <img src="docs/assets/icons/target.webp" width="24" height="24" alt="" /> | [Markup loop project](./examples/markup-loop-project/) | A minimal standalone project that reproduces the drop-in markup loop |
| <img src="docs/assets/icons/clipboard.webp" width="24" height="24" alt="" /> | [Markup VRT eval](./examples/markup-vrt-eval/) | A dogfood harness that measures the Playwright workflow while a new screen is built |
| <img src="docs/assets/icons/puzzle.webp" width="24" height="24" alt="" /> | [Gate plugin](./examples/gate-plugin/) | A project that adds two gates of its own through `vlmkit.config.json` |
| <img src="docs/assets/icons/card.webp" width="24" height="24" alt="" /> | [Solitaire](./examples/solitaire/) | Klondike in plain HTML/CSS/JS, the drag-and-drop and animation dogfood target |
| <img src="docs/assets/icons/page-check.webp" width="24" height="24" alt="" /> | [Intro page](./examples/vlmkit-intro-page/) | The dependency-free landing page, verified with vlmkit's own loop |
## Documentation
| Doc | Contents |
|---|---|
| [`docs/introduce.md`](./docs/introduce.md) | What is vlmkit? — narrative introduction assuming zero context |
| [`docs/markup-assist.md`](./docs/markup-assist.md) | Task-routing guide: which gate for which job, done-condition recipes, anti-gaming rules |
| [`docs/cli-reference.md`](./docs/cli-reference.md) | Complete command reference: all groups, examples, workflow/API/HTTP, architecture, project structure |
| [`docs/configuration.md`](./docs/configuration.md) | Install, MCP/skill setup, environment variables, snapshot/CI config, agent skills catalog |
| [`packages/vlmkit-mcp/README.md`](./packages/vlmkit-mcp/README.md) | MCP tool table |
| [`docs/knowledge.md`](./docs/knowledge.md) · [`docs/reports/`](./docs/reports/) | Accumulated evaluation findings and dated experiment reports |
| [`CHANGELOG.md`](./CHANGELOG.md) | Release notes, including the 0.4.x → current command mapping |
## License
MIT