// Reference for the `vrt` CLI — Visual Regression Testing combined with accessibility (a11y) semantic verification. Use when running `vrt-test` / `vrt` (Playwright VRT) / `vrt-update` / `vrt compare` / `vrt snapshot`, configuring the fix-loop CSS challenge benchmark, or picking a VLM model for diff analysis.
Closed-loop CSS auto-repair. Given a fixture with a known regression (one CSS property or one selector block removed), iterate with a VLM that proposes the missing fix from the diff screenshot, apply it, and re-run until the diff falls below a threshold. Currently scoped to the CSS-challenge fixture set in `src/experiments/css-challenge/`; adapting to an arbitrary repo requires writing a fixture entry. Use when measuring whether a VLM model can recover a known regression, not for production self-healing.
Run vrt diff in a stateful loop where each run is compared against the previous run's persisted summary, surfacing a `⚠ REGRESSION` banner when the majority of viewports get worse. Designed for periodic CI gates (per-PR or scheduled) where you want "did this change make things worse" as a binary signal, not a one-shot snapshot. Stores summary at `.vrt/last-diff-for-agent.json` by default.
Compare two rendered pages (URL pairs or local HTML) and produce a structured Markdown report tailored for coding agents — pixel diff per viewport, per-section diffRatio, computed-style diff split into universal vs. breakpoint-gated, and worst-viewport screenshot paths. Use when the agent just made a UI change and needs to know whether it altered visible output, where it altered it, and which CSS properties drove the change. Works on a single HTML/URL pair without baseline state.
Evaluate whether a framework / CSS-library / build-system swap (Tailwind → vanilla CSS, reset-css switch, bundler swap, etc.) produced a visually-equivalent result. Three modes — compare (deterministic pixel + CSD), blind (agent runs without baseline reference), subagent (dispatched-agent verification) — let the caller pick the rigour level. Use when the diff is large by construction (the markup was rewritten) and a flat pixel diff would drown the actual regressions in noise.
Five DOM/pixel-based signal tools for markup work — turn a screenshot + HTML scaffold into a converging pixel diff (build component), crop a page screenshot into per-component PNGs (scan component), audit hard-coded values against a design-token scale (check tokens), verify theme parity (check theme — light vs dark render), and stress-test layout under inflated text (stress i18n). All five are pure DOM + Playwright + pixel processing — **no VLM / no API key required**. The agent supplies the markup reasoning; the tool surfaces the signal. Use when you've authored HTML/CSS and want signal back without paying VLM latency or cost.
Improve a developer-facing tool (CLI, library, agent harness) by running closed-loop validation with disposable subagents, treating their friction as the deliverable, fixing the friction, and re-running. Use when the user wants to evaluate or harden a tool whose value depends on whether an agent can use it — vrt-like signal loops, agent SDKs, CLI ergonomics, prompt scaffolding, IDE integrations. The loop produces both a stronger tool and a written record (per-run reports + tracked issues) of what was learned and why.
| name | vrt |
| description | Reference for the `vrt` CLI — Visual Regression Testing combined with accessibility (a11y) semantic verification. Use when running `vrt-test` / `vrt` (Playwright VRT) / `vrt-update` / `vrt compare` / `vrt snapshot`, configuring the fix-loop CSS challenge benchmark, or picking a VLM model for diff analysis. |
A quality assurance tool for coding agents that combines Visual Regression Testing (VRT) with accessibility semantics verification.
Automatically verifies that changes are visually and semantically (a11y) as intended, running a loop to detect and repair regressions.
All commands run from the project root. See docs/api-design.md for API design details.
just vrt-test # Unit tests
just vrt # Playwright VRT
just vrt-update # Update snapshots
just css-challenge # Single CSS deletion challenge (LLM recovery)
just css-bench --trials 30 # Benchmark (detection rate measurement)
just css-bench --fixture dashboard --backend crater # Specify fixture/backend
just css-bench-all # All fixtures at once
just css-report # Analysis report of accumulated data
just migration-compare before.html after.html # 2-file comparison
just migration-reset # Reset CSS comparison (normalize vs others)
just migration-tailwind # Tailwind → vanilla CSS
Breakpoints are auto-discovered from CSS, generating boundary ±1px + random sample viewports.
just vrt-demo # Basic VRT demo (kitty graphics)
just vrt-demo-fix # Fix loop demo
just vrt-demo-multi # Multi-scenario
just vrt-demo-multistep # Multi-step
┌─────────────────────────────────────────────┐
│ 1. Create baseline │
│ just vrt-update │
└─────────┬───────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────┐
│ 2. Make code changes │
│ - State intent clearly in commit message │
│ (feat: / fix: / style: / refactor: / │
│ a11y: / deps:) │
└─────────┬───────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────┐
│ 3. just vrt │
└─────────┬───────────────────────────────────┘
│
┌────┴────────────────┐
│ │
PASS FAIL/ESCALATE
│ │
▼ ▼
┌──────────┐ ┌─────────────────────┐
│ 4a. │ │ 4b. Identify issue │
│ Done │ │ → Fix code │
│ │ │ → Return to 3 │
└──────────┘ └─────────────────────┘
Change ─→ 3 tracks run in parallel:
Track 1: Diff Intent — git diff + commit message → infer change intent
Track 2: Visual Diff — pixel comparison → heatmap → region classification
Track 3: A11y Diff — a11y tree diff → semantic change detection
→ Cross-Validation (cross-reference all 3):
| Visual | A11y | Intent | → Verdict |
|--------|-------|---------|----------------------|
| None | None | any | APPROVE (no change) |
| Yes | Yes | match | APPROVE (as expected) |
| Yes | Yes | none | ESCALATE (unclear intent) |
| Yes | None | style | APPROVE (visual only) |
| Yes | None | refac | ESCALATE (unintended) |
| None | Yes | a11y | APPROVE (a11y improvement) |
| None | Yes | other | REJECT (semantics broken) |
| any | regr | any | REJECT (a11y regression) |
→ Quality Gate:
- Whiteout detection (blank white screen)
- Error state detection (red warning display)
- Empty content detection
- A11y regression (lost label, removed landmark)
| code | Meaning |
|---|---|
| 0 | PASS — no change, or all approved |
| 1 | FAIL — rejected changes, or quality error |
escalate returns exit 0 but emits warnings.
The verification pipeline infers change intent from the commit message. When intent is correctly inferred, expected visual changes are auto-approved.
feat: add dark mode toggle → visual + a11y additions expected
fix: fix mobile layout breakage → only fix target should change
refactor: extract utility functions → no visual/a11y changes expected
style: change button color blue→green → visual change, no a11y change expected
a11y: add labels to form → a11y change, minimal visual change expected
deps: update to React 19 → no visual/a11y changes expected
VRT verify also inspects the A11y tree simultaneously. The following are detected:
label-missing)img-alt-missing)landmark-changed)node-removed)role-changed)If any of these are detected during refactoring, semantics are likely broken.
├── SKILL.md ← This file
├── Taskfile.pkl # Task runner (pkfire)
├── Spec.pkl / Test.pkl # Specs + smoke gate (pkspec)
├── package.json
├── playwright.config.ts # Playwright config for VRT
├── e2e/
│ └── vrt-capture.spec.ts # Screenshot + a11y collection
├── fixtures/ # Test fixtures
├── src/
│ ├── vrt-cli.ts # CLI entry point
│ ├── cli.ts # CLI helpers
│ ├── types.ts # All type definitions
│ ├── playwright-analyzer.ts # Playwright output analysis
│ ├── playwright-helper.ts # Playwright helpers
│ ├── dep-graph.ts # Dependency tree (TS/MoonBit/Rust)
│ ├── heatmap.ts # Pixel comparison + heatmap
│ ├── visual-semantic.ts # Visual Semantic Diff classification
│ ├── a11y-semantic.ts # A11y tree diff + quality checks
│ ├── cross-validation.ts # Visual x A11y x Intent cross-reference
│ ├── intent.ts # Diff → change intent inference
│ ├── quality.ts # Quality gate
│ ├── reasoning.ts # Change reason inference
│ ├── expectation.ts # Expectation matching
│ ├── introspect.ts # Spec generation/verification
│ ├── goal-runner.ts # Goal-driven execution
│ ├── llm-client.ts # LLM provider
│ ├── agent.ts # 5-stage verification loop
│ ├── demo.ts # VRT demo
│ ├── demo-fix-loop.ts # Fix loop demo
│ ├── demo-scenarios.ts # Multi-scenario demo
│ └── demo-multistep.ts # Multi-step demo
└── test-results/ # Execution results (gitignore)
| Issue | Solution |
|---|---|
| Font rendering diffs | Adjust pixelmatch threshold (heatmap.ts) |
| A11y tree is null | Wait for page render completion (adjust waitFor) |
| Everything becomes ESCALATE | Add prefix to commit message (feat:/fix:/style: etc.) |