automated-a11y-testing-axe-core

# Automated A11y Testing: axe-core Engine Implements automated accessibility testing using the axe-core engine across the entire testing pyramid: unit tests (jest-axe), end-to-end tests (@axe-core/playwright), and CI/CD integration (pa11y-ci). This skill covers configuration, rule tags, assertions, baselines for regression detection, and common violation patterns. Load when setting up accessibility testing, adding A11y checks to test suites, or integrating continuous accessibility validation. ## TL;DR Checklist - [ ] Install axe-core dependencies: `npm install -D axe-core jest-axe @axe-core/playwright pa11y-ci` - [ ] Add jest-axe matcher in Jest setup: `expect.extend(toHaveNoViolations)` - [ ] Create unit test for each component: scan and assert `toHaveNoViolations()` - [ ] Configure Playwright E2E with AxeBuilder: specify WCAG rules (wcag2a, wcag21aa, wcag22aa) - [ ] Set up pa11y-ci for multi-URL scanning with GitHub Actions - [ ] Establish baseline for violations in main branch - [ ] Configure CI to fail on new accessibility violations - [ ] Document false positives and tag for exclusion --- ## When to Use Use this skill when: - Adding accessibility testing to a new project's test suite - Setting up CI/CD pipeline with accessibility gates - Implementing testing pyramid accessibility coverage (unit → E2E → integration) - Establishing regression detection for accessibility violations - Scanning multiple URLs or pages systematically - Documenting and excluding false positive violations - Reporting accessibility metrics in dashboards --- ## When NOT to Use Avoid this skill for: - Manual accessibility testing (keyboard nav, screen reader testing — those require human testing) - Designing accessibility fixes (use WCAG fundamentals, keyboard-navigation, semantic-HTML skills) - Advanced semantic testing beyond automated rules (axe catches ~30% of issues) - Testing PDF/document accessibility (different tools required) - Testing third-party embedded widgets (accessibility varies by widget) --- ## Core Workflow 1. **Install Dependencies** — Add axe-core, jest-axe, @axe-core/playwright, pa11y-ci to dev dependencies. Verify versions support your Jest/Playwright versions. 2. **Configure Rule Set** — Choose rule set (wcag2a, wcag21aa, wcag22aa) based on compliance target. Tag configuration determines which violations are caught. AA is standard baseline. 3. **Unit Test Components** — Create jest-axe tests for each component. Arrange component state, scan with axe, assert no violations. Test multiple states (hover, disabled, error, loading). 4. **E2E Page Tests** — Add Playwright tests with AxeBuilder. Test full pages, not just components. Specify which rules to check (wcag21aa standard). 5. **Establish Baseline** — Run full test suite, capture violations that are expected/acceptable. Create baseline snapshot or JSON report. 6. **Configure CI Gate** — Set up GitHub Actions (or CI tool) to run pa11y-ci on all URLs. Compare against baseline. Fail pipeline on new violations. 7. **Document Exclusions** — For false positives, tag with `data-testid` or CSS selector for exclusion. Document why exclusion exists. Review exclusions in sprint planning. 8. **Monitor and Report** — Track A11y test results over time. Report violations metrics (active violations, fixed in last sprint, remediation backlog). Use dashboard to track progress. --- ## Testing Pyramid for Accessibility ``` ▲ │ Integration Tests (multi-page flows) │ pa11y-ci scanning all URLs │ │ E2E Tests (AxeBuilder on full pages) │ @axe-core/playwright scanning user interactions │ │ Unit Tests (jest-axe on components) │ Each component scanned in isolation │ └──────────────────────────────────── ``` **Goal**: Catch accessibility violations as early as possible in development. --- ## Implementation Patterns ### Pattern 1: Jest Unit Test with jest-axe ```typescript import { render, screen } from '@testing-library/react'; import { axe, toHaveNoViolations } from 'jest-axe'; // Register jest-axe matchers expect.extend(toHaveNoViolations); describe('AccessibleButton Component', () => { // Test 1: Component renders without a11y violations it('should not have accessibility violations in default state', async () => { const { container } = render( <button aria-label="Add item"> <svg aria-hidden="true">+</svg> </button> ); const results = await axe(container); expect(results).toHaveNoViolations(); }); // Test 2: Disabled state it('should not have violations when disabled', async () => { const { container } = render( <button disabled aria-label="Add item"> <svg aria-hidden="true">+</svg> </button> ); const results = await axe(container); expect(results).toHaveNoViolations(); }); // Test 3: With error state it('should not have violations with error state', async () => { const { container } = render( <button aria-label="Add item" aria-invalid="true"> <svg aria-hidden="true">!</svg> </button> ); const results = await axe(container); expect(results).toHaveNoViolations(); }); // Test 4: Specific rule check (focus-visible) it('should have visible focus indicator', async () => { const { container } = render( <button aria-label="Add item">+</button> ); const results = await axe(container, { rules: { 'focus-visible': { enabled: true }, }, }); expect(results).toHaveNoViolations(); }); }); // Form field with label describe('AccessibleFormField Component', () => { it('should have proper label association', async () => { const { container } = render( <div> <label htmlFor="email-input">Email Address</label> <input id="email-input" type="email" aria-required="true" placeholder="you@example.com" /> </div> ); const results = await axe(container); expect(results).toHaveNoViolations(); }); it('should have accessible error messages', async () => { const errorId = 'email-error'; const { container } = render( <div> <label htmlFor="email-input">Email Address</label> <input id="email-input" type="email" aria-invalid="true" aria-describedby={errorId} /> <p id={errorId} role="alert"> Please enter a valid email </p> </div> ); const results = await axe(container); expect(results).toHaveNoViolations(); }); }); ``` **Key patterns:** - Use `await axe(container)` to scan the component - Multiple tests for different states (default, disabled, error, loading) - Use `rules` option to enable/disable specific checks - Always include jest-axe extend in setup file for project-wide use ### Pattern 2: Playwright E2E with AxeBuilder ```typescript import { test, expect } from '@playwright/test'; import { injectAxe, checkA11y } from 'axe-playwright'; test.describe('HomePage Accessibility', () => { // Full page scan test('should not have a11y violations on homepage', async ({ page }) => { await page.goto('http://localhost:3000'); // Inject axe-core into page await injectAxe(page); // Check a11y violations (default: wcag2aa standard) await checkA11y(page, null, { rules: { 'color-contrast': { enabled: true }, 'heading-order': { enabled: true }, }, }); }); // Scan specific region (exclude certain elements) test('should scan main content area only', async ({ page }) => { await page.goto('http://localhost:3000/dashboard'); await injectAxe(page); // Scan only main, exclude sidebar await checkA11y(page, 'main', { exclude: ['aside', '[data-testid="cookie-banner"]'], }); }); // Interactive scan (test after user actions) test('should have no violations after opening dialog', async ({ page }) => { await page.goto('http://localhost:3000'); await injectAxe(page); // User opens dialog await page.click('button[aria-label="Open settings"]'); await page.waitForSelector('[role="dialog"]'); // Scan updated DOM await checkA11y(page, '[role="dialog"]', { rules: { 'focus-visible': { enabled: true }, 'button-name': { enabled: true }, }, }); }); // Scan multiple states test('should check form validation states', async ({ page }) => { await page.goto('http://localhost:3000/form'); await injectAxe(page); // Scan initial state await checkA11y(page, 'form'); // Fill invalid email, trigger validation await page.fill('input[type="email"]', 'invalid'); await page.click('button[type="submit"]'); await page.waitForSelector('[role="alert"]'); // Scan error state await checkA11y(page, 'form', { rules: { 'color-contrast': { enabled: true }, // Ensure error color passes contrast }, }); }); }); // Using AxeBuilder (alternative API) test('alternative AxeBuilder API', async ({ page }) => { await page.goto('http://localhost:3000'); const results = await new AxeBuilder({ page }) .withTags(['wcag21aa', 'wcag22aa']) .exclude('[data-testid="third-party-widget"]') .analyze(); expect(results.violations).toEqual([]); }); ``` **Key patterns:** - Use `injectAxe(page)` then `checkA11y(page)` or use AxeBuilder - Test after user interactions (clicks, form fills, dialogs) - Exclude third-party widgets or known false positives - Specify rules based on compliance target (wcag21aa, wcag22aa) - Test multiple pages and user flows ### Pattern 3: pa11y-ci for CI/CD Integration ```json // .pa11yci.json — Configuration for multi-URL scanning { "runners": ["axe"], "standard": "WCAG2AA", "timeout": 10000, "wait": 500, "chromeLaunchConfig": { "args": ["--disable-gpu", "--no-sandbox"] }, "urls": [ "http://localhost:3000/", "http://localhost:3000/about", "http://localhost:3000/contact", "http://localhost:3000/products", "http://localhost:3000/checkout", "http://localhost:3000/account" ], "headless": true, "includeNotices": false, "includeWarnings": false, "suppressStderr": false, "strict": true, "bail": true, "reporters": ["json", "csv"], "outputDir": "./a11y-reports" } ``` ```bash # Run pa11y-ci locally pa11y-ci --config .pa11yci.json # Run against live staging URL pa11y-ci --config .pa11yci.json --base-url https://staging.example.com # Generate baseline (first run, commit as reference) pa11y-ci --config .pa11yci.json --save-baseline # Compare against baseline (fail if new violations) pa11y-ci --config .pa11yci.json --check-baseline ``` **GitHub Actions Workflow:** ```yaml name: Accessibility Tests on: pull_request: push: branches: [main] jobs: a11y: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - name: Setup Node uses: actions/setup-node@v3 with: node-version: 18 cache: npm - name: Install dependencies run: npm ci - name: Build application run: npm run build - name: Start dev server run: npm run dev & env: CI: true - name: Wait for server run: npx wait-on http://localhost:3000 - name: Run jest-axe tests run: npm run test:a11y - name: Run pa11y-ci run: npx pa11y-ci --config .pa11yci.json --check-baseline - name: Upload a11y report if: always() uses: actions/upload-artifact@v3 with: name: a11y-reports path: a11y-reports/ - name: Comment PR with results if: always() uses: actions/github-script@v6 with: script: | const fs = require('fs'); const report = JSON.parse(fs.readFileSync('a11y-reports/results.json')); const violations = report.results.reduce((sum, r) => sum + r.violations.length, 0); github.rest.issues.createComment({ issue_number: context.issue.number, owner: context.repo.owner, repo: context.repo.repo, body: `🔍 Accessibility Test Results\n\nViolations: ${violations}\n\n[View detailed report](https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }})` }); ``` ### Pattern 4: Baseline Management and Exclusions ```typescript // jest.setup.js - Configure jest-axe with default rules import { axe, toHaveNoViolations } from 'jest-axe'; expect.extend(toHaveNoViolations); // Define globally excluded rules and elements