Visual Regression Testing
Overview
Visual regression testing detects unintended changes to the appearance of your UI by comparing screenshots against approved baselines. It catches CSS regressions, layout shifts, font changes, color modifications, and other visual issues that functional tests miss because they don't assert on how things look.
When Visual Testing Matters
| Scenario | Why |
|---|
| Design systems / component libraries | Every component must match the design spec exactly |
| CSS refactors | Changing shared styles can break distant pages |
| Dependency upgrades | Font, icon, or framework updates can shift layouts |
| Theme / dark mode | Multiple visual variants to verify |
| Cross-browser support | Same code renders differently across browsers |
| Responsive breakpoints | Layout changes at mobile, tablet, desktop widths |
Cross-Platform Tools
| Tool | Integration | Approach | Pricing |
|---|
| Chromatic | Storybook | Cloud-hosted visual snapshots of every story | Free tier + paid |
| Percy (BrowserStack) | Any framework | Cloud-hosted cross-browser visual diffs | Paid |
| BackstopJS | Standalone | Local headless CSS regression with reference images | Free / open source |
| Playwright | Playwright tests | Built-in toHaveScreenshot() with pixel comparison | Free / open source |
| Applitools Eyes | Any framework | AI-powered visual testing (layout, content, strict modes) | Paid |
Chromatic
Overview
Chromatic is a cloud-based visual testing service built by the Storybook maintainers. It captures a snapshot of every Storybook story on every commit and detects visual changes with a review/approve workflow.
Setup with Storybook
npm install --save-dev chromatic
Running Chromatic
npx chromatic --project-token=<your-project-token>
npx chromatic --project-token=<your-project-token> --only-changed
npx chromatic --project-token=<your-project-token> --auto-accept-changes
CI Integration (GitHub Actions)
name: Chromatic
on: push
jobs:
chromatic:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: actions/setup-node@v4
with:
node-version: 20
- run: npm ci
- uses: chromaui/action@latest
with:
projectToken: ${{ secrets.CHROMATIC_PROJECT_TOKEN }}
onlyChanged: true
exitZeroOnChanges: true
exitOnceUploaded: true
Configuring Stories for Visual Testing
import { Card } from "./Card";
export default {
title: "Components/Card",
component: Card,
parameters: {
chromatic: {
viewports: [320, 768, 1200],
delay: 300,
diffThreshold: 0.063,
},
},
};
export const Default = {
args: { title: "Card Title", body: "Card content goes here." },
};
export const Loading = {
args: { loading: true },
parameters: {
chromatic: {
pauseAnimationAtEnd: true,
},
},
};
export const WithTooltip = {
: { : },
: {
: { : },
},
};
Percy (BrowserStack)
Overview
Percy captures snapshots at multiple browser/viewport combinations in the cloud. It integrates with Playwright, Cypress, Selenium, Storybook, and static sites.
Playwright + Percy
npm install --save-dev @percy/cli @percy/playwright
import { test } from "@playwright/test";
import percySnapshot from "@percy/playwright";
test.describe("Homepage visual tests", () => {
test("homepage renders correctly", async ({ page }) => {
await page.goto("/");
await page.waitForLoadState("networkidle");
await percySnapshot(page, "Homepage");
});
test("homepage with modal open", async ({ page }) => {
await page.goto("/");
await page.click("button#open-modal");
await page.waitForSelector("[role='dialog']");
await percySnapshot(page, "Homepage - Modal Open");
});
test("responsive layouts", async ({ page }) => {
await page.goto("/");
await percySnapshot(page, , {
: [, , ],
});
});
});
Running Percy
npx percy exec -- npx playwright test tests/visual/
npx percy storybook ./storybook-static
npx storybook build -o storybook-static
npx percy storybook ./storybook-static
Percy CI Configuration
name: Visual Tests
on: [push, pull_request]
jobs:
visual-test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
- run: npm ci
- run: npx playwright install --with-deps
- run: npx percy exec -- npx playwright test tests/visual/
env:
PERCY_TOKEN: ${{ secrets.PERCY_TOKEN }}
BackstopJS
Overview
BackstopJS is a free, open-source visual regression tool that runs headless Chrome/Firefox locally. It compares test screenshots against reference images and generates an HTML diff report.
Configuration
{
"id": "my-project",
"viewports": [
{ "label": "phone", "width": 320, "height": 480 },
{ "label": "tablet", "width": 768, "height": 1024 },
{ "label": "desktop", "width": 1280, "height": 800 }
],
"scenarios": [
{
"label": "Homepage"
BackstopJS Workflow
npm install -g backstopjs
backstop reference
backstop test
backstop approve
CI Integration
jobs:
visual-test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
- run: npm ci
- run: npm start &
- run: npx wait-on http://localhost:3000
- run: npx backstopjs test --config backstop.json
- uses: actions/upload-artifact@v4
if: always()
with:
name: backstop-report
path: tests/visual/backstop_data/html_report/
Playwright Screenshot Assertions
Overview
Playwright has built-in visual comparison via toHaveScreenshot() and toMatchSnapshot(). No external service required — baselines are stored in the repository.
Basic Screenshot Comparison
import { test, expect } from "@playwright/test";
test.describe("Visual regression tests", () => {
test("homepage matches baseline", async ({ page }) => {
await page.goto("/");
await page.waitForLoadState("networkidle");
await expect(page).toHaveScreenshot("homepage.png", {
fullPage: true,
maxDiffPixels: 100,
});
});
test("navigation component matches baseline", async ({ page }) => {
await page.goto("/");
const nav = page.locator("nav.main-nav");
await expect(nav).toHaveScreenshot("navigation.png");
});
test("responsive layouts match baselines", async ({ page }) => {
await page.goto("/");
( width [, , ]) {
page.({ width, : });
(page).(, {
: ,
});
}
});
(, ({ page }) => {
page.({ : });
page.();
(page).(, {
: ,
});
});
});
Playwright Visual Test Configuration
import { defineConfig } from "@playwright/test";
export default defineConfig({
testDir: "tests/visual",
expect: {
toHaveScreenshot: {
maxDiffPixels: 50,
maxDiffPixelRatio: 0.01,
threshold: 0.2,
animations: "disabled",
},
},
projects: [
{
name: "visual-chromium",
use: {
browserName: "chromium",
viewport: { width: 1280, height: 720 },
},
},
],
});
Updating Baselines
npx playwright test tests/visual/
npx playwright test tests/visual/ --update-snapshots
npx playwright test tests/visual/pages.spec.js -g "homepage" --update-snapshots
Applitools Eyes
Overview
Applitools Eyes uses AI-powered visual comparison with multiple match levels: Strict (pixel-level), Layout (structure only), Content (text/images), and Dynamic (ignores dynamic regions). This significantly reduces false positives compared to pixel-based tools.
Playwright + Applitools
import { test } from "@playwright/test";
import {
Eyes,
ClassicRunner,
Configuration,
BatchInfo,
MatchLevel,
} from "@applitools/eyes-playwright";
let eyes;
let runner;
test.beforeAll(() => {
runner = new ClassicRunner();
});
test.beforeEach(async ({ page }) => {
eyes = new Eyes(runner);
const config = new Configuration();
config.setBatch(new BatchInfo("My App Visual Tests"));
config.setMatchLevel(MatchLevel.Strict);
eyes.setConfiguration(config);
await eyes.open(page, "My App", test.info().title, { width: 1280, height: 720 });
});
test.afterEach(async () => {
await eyes.close(false);
});
test.afterAll( () => {
results = runner.();
.(, results.());
});
(, ({ page }) => {
page.();
eyes.(, {
: ,
: ,
});
});
(, ({ page }) => {
page.();
eyes.(, {
: ,
: ,
});
});
Baseline Management
Strategy: Git-Stored Baselines (Playwright, BackstopJS)
project/
tests/
visual/
pages.spec.js
pages.spec.js-snapshots/ # Auto-generated by Playwright
homepage-chromium-linux.png # Platform-specific baselines
homepage-chromium-darwin.png
navigation-chromium-linux.png
- Commit baselines to git so all developers share the same references.
- Use platform-specific baseline names (Playwright does this automatically).
- Update baselines via
--update-snapshots after intentional changes.
- Review baseline diffs in PR reviews (GitHub renders image diffs).
Strategy: Cloud-Hosted Baselines (Chromatic, Percy, Applitools)
- Baselines are stored in the cloud service, not in git.
- Approval happens through the service's web UI.
- Branch-based baselines: each branch gets its own baseline context.
- Auto-accept on merge to main (approved PR changes become the new baseline).
Handling Flaky Screenshots
Common Causes and Solutions
| Cause | Solution |
|---|
| Animations / transitions | Disable CSS animations: animations: "disabled" in Playwright; pauseAnimationAtEnd in Chromatic |
| Font loading | Wait for fonts: await page.waitForLoadState("networkidle") or use document.fonts.ready |
| Dynamic timestamps | Hide or mask dynamic elements: removeSelectors (BackstopJS) or exclude() (axe) |
| Caret / cursor blinking | Click away from input fields before screenshot |
| Lazy-loaded images | Scroll to trigger lazy loading or wait for specific selectors |
| Scrollbar differences | Hide scrollbars via CSS: *::-webkit-scrollbar { display: none; } |
| Anti-aliasing differences | Use maxDiffPixels or threshold to allow minor sub-pixel differences |
| System font rendering | Run visual tests in Docker for consistent rendering, or use platform-specific baselines |
Playwright Anti-Flake Pattern
test("stable visual comparison", async ({ page }) => {
await page.goto("/");
await page.waitForLoadState("networkidle");
await page.evaluate(() => document.fonts.ready);
await page.waitForSelector("[data-loaded='true']");
await page.evaluate(() => {
document.querySelectorAll("[data-dynamic]").forEach((el) => {
el.style.visibility = "hidden";
});
});
await page.evaluate(() => {
const style = document.createElement("style");
style.textContent = `
*, *::before, *::after {
animation-duration: 0s !important;
transition-duration: 0s !important;
}
`;
document..(style);
});
(page).(, {
: ,
: ,
});
});
Best Practices
General
- Visual tests complement functional tests — they catch what
expect(text).toBe("Hello") cannot.
- Start with component-level visual tests (Storybook + Chromatic) before full-page screenshots.
- Test at multiple viewport widths to catch responsive breakpoint regressions.
- Test light and dark themes as separate visual baselines.
Baseline Management
- Commit git-stored baselines alongside the code changes that caused them.
- Require visual review approval in PRs before merging (Chromatic and Percy support this as GitHub checks).
- Update baselines intentionally — never auto-approve in CI without human review.
- Use platform-specific baselines if tests run on different OSes (font rendering differs).
Reducing False Positives
- Disable CSS animations and transitions during visual tests.
- Wait for network idle, font loading, and lazy-loaded content before capturing.
- Mask or hide dynamic content (timestamps, random avatars, ads).
- Use AI-powered tools (Applitools) for content-heavy pages where pixel comparison is too brittle.
- Set a reasonable diff threshold — zero tolerance leads to flaky tests.
CI Integration
- Run visual tests on every PR to catch regressions early.
- Use cloud services (Chromatic, Percy) for cross-browser visual testing — local tools only test one browser.
- Store visual diff reports as CI artifacts for review.
- Gate merging on visual approval for design system and component library repositories.
Workflow
- Developers update baselines locally after intentional changes.
- Reviewers verify visual diffs alongside code diffs in PRs.
- Designers can participate in Chromatic/Percy review workflows for design approval.
- Schedule periodic full-suite visual runs to catch environment-level drift (OS/browser updates).
