| name | site-test |
| description | Visual compliance testing tools for comparing a website clone against reference recordings. Use to run tests, generate visual diffs, and analyze differences. |
Visual Compliance Testing
These CLI tools compare your clone against reference recordings to identify visual differences.
Commands
site-test
Run a full visual compliance test against a recording.
site-test <recording_folder> <url>
Arguments:
recording_folder - Path to the recording directory (e.g., ./recordings/0)url - URL of your dev server (e.g., http://localhost:5173)
Options:
--output-dir <path> - Where to write the test report (default: auto-generated)--screenplay <path> - Custom screenplay file (auto-discovered by default)
What it does:
- Reads the screenplay.json from the recording
- Executes each step (navigation, clicks, etc.) via browser automation
- Takes screenshots at assertion points
- Compares screenshots against reference images
- Generates a report with diffs
Output structure:
{domain}_{timestamp}_test/
├── execution-log.json # Detailed step-by-step log
├── summary.json # Pass/fail with step counts and duration
├── asserts/
│ ├── 0/
│ │ ├── screenshot.png # Captured screenshot
│ │ └── diff.png # Visual diff (if differences found)
│ └── ...
└── recording/ # Copy of reference data
Example:
site-test ./recordings/0 http://localhost:5173
site-test ./recordings/0 http://localhost:5173 --output-dir ./test-results
site-test-diff
Generate a visual diff between two screenshots.
site-test-diff <reference_screenshot> <subject_screenshot> <output_path>
Arguments:
reference_screenshot - Path to the reference image (from recording)subject_screenshot - Path to the current implementation screenshotoutput_path - Where to write the diff image
Options:
--no-dynamic-detection - Pure SSIM-based diff without LLM analysis--skip-verification - Stage 1 only (detect regions but skip classification)
What it does:
- Computes structural similarity (SSIM) between the two images
- Stage 1: Uses Gemini to detect dynamic content regions (timestamps, ads, etc.)
- Stage 2: Classifies each difference as structural (red) or content change (blue)
- Outputs a color-coded diff image
Expected image dimensions: 1280x720
Example:
site-test-diff ./recordings/0/screenshots/0/screenshot.png ./my-screenshot.png ./diff.png
site-test-diff ./recordings/0/screenshots/0/screenshot.png ./my-screenshot.png ./diff.png --no-dynamic-detection
lookatdiff
AI-powered analysis of visual differences using Gemini.
lookatdiff <subject> <diff> <actual>
Arguments:
subject - Screenshot of the current implementationdiff - The diff image (from site-test-diff)actual - The reference screenshot (target)
Options:
-q, --question <text> - Ask a specific question about the differences
What it does:
- Sends all three images to Gemini for multi-modal analysis
- Identifies what needs to be fixed based on red-highlighted differences
- Groups changes by UI section (header, sidebar, content, footer)
- Flags suspicious blue/cyan areas that might be incorrectly marked
Example:
lookatdiff ./my-screenshot.png ./diff.png ./recordings/0/screenshots/0/screenshot.png
lookatdiff ./my-screenshot.png ./diff.png ./reference.png -q "What CSS changes would fix the header?"
Workflow
A typical testing iteration:
npm run dev &
site-test ./recordings/0 http://localhost:5173
cat ./*_test/summary.json
lookatdiff ./capture.png ./diff.png ./reference.png
site-test ./recordings/0 http://localhost:5173
Diff Color Coding
| Color | Meaning | Action |
|---|
| Red | Structural differences (layout, missing elements) | Must fix |
| Blue/Cyan | Dynamic content changes (same structure, different text) | Can ignore |
Two-Stage Detection
- Stage 1 - Region detection: Gemini identifies dynamic content areas (timestamps, counters, personalized content, ads, carousels)
- Stage 2 - Difference verification: Each flagged region is classified as:
CONTENT_CHANGE - Same structure, different content (blue, ignore)STRUCTURAL_CHANGE - Layout or size changed (red, must fix)MISSING - Element missing entirely (red, must fix)
