e2e-test

name

e2e-test

description

End-to-end testing skill (includes merged agent-browser capabilities). For web/SPA projects: launches parallel sub-agents to research codebase, then uses Playwright scripts (preferred), Playwright MCP interactive mode, or agent-browser CLI to test every user journey. For non-web types: redirects to corresponding test strategy or backend-test skill. Run after /execute to validate before code review.

End-to-End Application Testing

v1.3.0: agent-browser Skill merged into this skill. v1.4.0: Playwright MCP interactive mode added as an execution path for trace-level verification records. Use e2e-test for all browser automation needs. For backend-only testing (REST API / DB / unit tests), use backend-test skill.

Pre-flight Check

1. Platform Check

agent-browser requires Linux, WSL, or macOS. Check the platform:

uname -s

• Linux or Darwin → proceed
• Anything else (e.g., MINGW, CYGWIN, or native Windows) → stop with:

"agent-browser only supports Linux, WSL, and macOS. It cannot run on native Windows. Please run this command from WSL or a Linux/macOS environment."

Stop execution if the platform is unsupported.

2. Frontend Check

Verify the application has a browser-accessible frontend. Check for:

• A web server or dev server configuration serving a UI
• Frontend framework files (pages/, app/, src/components/, index.html, templates/, public/, etc.)
• Any HTTP-accessible route serving a web interface

If no frontend is detected:

"This application doesn't appear to have a browser-accessible frontend. E2E browser testing requires a UI to visit. For backend-only or API testing, use API workflow tests instead."

Stop execution if no frontend is found.

3. Project Type Detection (determines test tooling)

Check .claude/reference/test-strategies/ for existence, and read the ## Project Type section from the plan file (if available):

Detected Type	Test Tool	Notes
web	Playwright (npx playwright test) + Playwright MCP (interactive trace mode)	Script-based for CI regression, MCP for trace-level QA records
tauri	Tauri IPC direct call (invoke) + frontend dev console	No browser automation
rest-api	API workflow test (supertest / httpx)	Multi-step HTTP sequence
cli	CLI integration test (subprocess assertions)	exit code + stdout
worker	Message queue stub + timing assertions	Inject messages, observe processing
mobile	Detox / Maestro	Simulator interactions
Not detected	agent-browser (default fallback)	Falls back to browser automation; Playwright MCP recommended if available

If web type detected → go to Playwright E2E flow (Phase 1a below). Other types → execute via corresponding strategy (read .claude/reference/test-strategies/{type}.md). Not detected → continue with agent-browser flow.

4. agent-browser Installation（仅默认回退路径）

仅当项目类型未检测或为 web 但 Playwright 不可用时才执行此步骤。

Check if agent-browser is installed:

agent-browser --version

If the command is not found, install it automatically:

npm install -g agent-browser

After installation (or if it was already installed), ensure the browser engine is set up:

agent-browser install --with-deps

The --with-deps flag installs system-level Chromium dependencies on Linux/WSL. On macOS it is harmless.

Verify installation succeeded:

agent-browser --version

If installation fails, stop with:

"Failed to install agent-browser. Please install it manually with npm install -g agent-browser && agent-browser install --with-deps, then re-run this command."

Phase 1a: Playwright E2E Flow (web type only)

Execute this phase when project type is detected as web; skip Phase 1-4 agent-browser flow.

1. Install and Configure

# Check if already installed
npx playwright --version || npm install -D @playwright/test && npx playwright install chromium

Generate or reuse playwright.config.ts (refer to template in .claude/reference/test-strategies/web.md).

2. Generate Tests per AC

Read the Spec-Lite AC list; create a test file for each business flow (e2e/{feature}.spec.ts):

import { test, expect } from '@playwright/test';

test('core user journey', async ({ page }) => {
  await page.goto('/');
  // Follow the interaction path described in Spec-Lite AC
  await expect(page.getByTestId('{element}')).toBeVisible();
});

3. Execute and Report

npx playwright test
npx playwright show-report

Write test results to the ## Playwright Test Log section of summary.md.

---

Phase 1b: 非 Web 类型流程

当项目类型为 tauri / rest-api / cli / worker / mobile 时： 直接读取 .claude/reference/test-strategies/{type}.md，按其中的"业务流程验证方式"执行。 跳过后续 Phase 1-4 的 agent-browser 流程。

---

Phase 1c: Playwright MCP Interactive Mode (web type, trace-level records)

Execute this phase after (or in place of) Phase 1a when trace-level verification records are needed for QA archiving. MCP mode generates per-step snapshots, screenshots, network traces, and console logs suitable for tester review.

Mode Selection

Scenario	Use
CI regression, reproducible automation	Phase 1a (Playwright scripts)
Dev-time UI verification, trace-level QA records	Phase 1c (MCP interactive)
Full coverage (recommended)	Phase 1a + Phase 1c

1. Verify MCP Availability

Check that the Playwright MCP server is active. If mcp__playwright__browser_navigate is not available, fall back to Phase 1a or agent-browser.

2. Execute Per-AC Trace Records

Read the Spec-Lite AC list. For each AC involving UI interaction:

1. browser_navigate to the target page
2. browser_snapshot to capture initial state
3. Execute the interaction steps described in the AC (click, type, select, etc.)
4. browser_snapshot to capture post-interaction state
5. Assert: expected visual state matches actual snapshot
6. Record evidence to .agents/reports/mcp-traces/{phase}-{ac-id}-{step}.snapshot.md
7. Optional: browser_take_screenshot for visual evidence, browser_network_requests for API verification, browser_console_messages for JS error checking

3. Generate Trace Index

After all ACs are traced, write a trace index to .agents/reports/mcp-traces/{phase}-index.md:

# MCP Trace Index: {phase-name}

**Date**: {date}
**Project Type**: web
**Execution Mode**: MCP Interactive

| AC | Step | Action | Snapshot | Screenshot | Status |
|----|------|--------|----------|------------|--------|
| AC-1 | 1 | navigate(/page) | [snapshot](path) | [screenshot](path) | ✅ |

4. Write Summary

Write MCP test results to ## MCP Trace Log section of summary.md:

## MCP Trace Log

| AC | Steps | Snapshots | Screenshots | Issues | Status |
|----|-------|-----------|-------------|--------|--------|
| AC-1 | 3 | 3 | 2 | 0 | ✅ |

MCP mode vs Script mode: MCP traces are session artifacts for QA review. They do NOT replace Playwright scripts for CI regression. Both can coexist — scripts provide reproducible automation, MCP provides human-readable trace evidence.

---

Phase 1: Parallel Research（默认回退路径）

仅当项目类型未检测到时才执行本阶段。

Launch three sub-agents simultaneously using the Task tool. All three run in parallel.

Sub-agent 1: Application Structure & User Journeys

Research this codebase thoroughly. Return a structured summary covering:

1. How to start the application — exact commands to install dependencies and run the dev server, including the URL and port it serves on
2. Authentication/login — if the app has protected routes, how to create a test account or log in (credentials from .env.example, seed data, or sign-up flow)
3. Every user-facing route/page — each URL path and what it renders
4. Every user journey — complete flows a user can take (e.g., "sign up → create profile → view public page"). For each journey, list the specific steps, interactions (clicks, form fills, navigation), and expected outcomes
5. Key UI components — forms, modals, dropdowns, pickers, toggles, and other interactive elements that need testing

Be exhaustive. Testing will only cover what you identify here.

Sub-agent 2: Database Schema & Data Flows

Research this codebase's database layer. Read .env.example to understand environment variables for database connections. DO NOT read .env directly. Return a structured summary covering:

1. Database type and connection — what database is used (Postgres, MySQL, SQLite, etc.) and the environment variable name for the connection string (from .env.example)
2. Full schema — every table, its columns, types, and relationships
3. Data flows per user action — for each user-facing action (form submit, button click, etc.), document exactly what records are created, updated, or deleted and in which tables
4. Validation queries — for each data flow, provide the exact query to verify records are correct after the action

Sub-agent 3: Bug Hunting

Analyze this codebase for potential bugs, issues, and code quality problems. Focus on:

1. Logic errors — incorrect conditionals, off-by-one errors, missing null checks, race conditions
2. UI/UX issues — missing error handling in forms, no loading states, broken responsive layouts, accessibility problems
3. Data integrity risks — missing validation, potential orphaned records, incorrect cascade behavior
4. Security concerns — SQL injection, XSS, missing auth checks, exposed secrets

Return a prioritized list with file paths and line numbers.

Wait for all three sub-agents to complete before proceeding.

Phase 2: Start the Application（默认回退路径）

Using Sub-agent 1's startup instructions:

1. Install dependencies if needed
2. Start the dev server in the background (use the project's startup command, e.g., npm run dev &, python -m uvicorn ... &, mvn spring-boot:run &, etc.)
3. Wait for the server to be ready
4. Open the app with agent-browser open <url> and confirm it loads
5. Take an initial screenshot: agent-browser screenshot e2e-screenshots/00-initial-load.png

Phase 3: Create Task List

Using the user journeys from Sub-agent 1 and findings from Sub-agent 3, create a task (using TaskCreate) for each user journey. Each task should include:

• subject: The journey name (e.g., "Test profile creation flow")
• description: Steps to execute, expected outcomes, database records to verify, and any related bug findings from Sub-agent 3
• activeForm: Present continuous (e.g., "Testing profile creation flow")

Also create a final task: "Responsive testing across viewports."

Phase 4: User Journey Testing

For each task, mark it in_progress with TaskUpdate and execute the following.

4a. Browser Testing

Use the Vercel Agent Browser CLI for all browser interaction:

agent-browser open <url>              # Navigate to a page
agent-browser snapshot -i             # Get interactive elements with refs (@e1, @e2...)
agent-browser click @eN               # Click element by ref
agent-browser fill @eN "text"         # Clear field and type
agent-browser select @eN "option"     # Select dropdown option
agent-browser press Enter             # Press a key
agent-browser screenshot <path>       # Save screenshot
agent-browser screenshot --annotate   # Screenshot with numbered element labels
agent-browser set viewport W H        # Set viewport (e.g., 375 812 for mobile)
agent-browser wait --load networkidle # Wait for page to settle
agent-browser console                 # Check for JS errors
agent-browser errors                  # Check for uncaught exceptions
agent-browser get text @eN            # Get element text
agent-browser get url                 # Get current URL
agent-browser close                   # End session

Refs become invalid after navigation or DOM changes. Always re-snapshot after page navigation, form submissions, or dynamic content updates (modals, tabs, theme changes).

For each step in a user journey:

1. Snapshot to get current refs
2. Perform the interaction
3. Wait for the page to settle
4. Take a screenshot — save to a descriptive path under e2e-screenshots/ organized by journey (e.g., e2e-screenshots/profile-creation/03-form-submitted.png)
5. Analyze the screenshot — use the Read tool to view the screenshot image. Check for visual correctness, UX issues, broken layouts, missing content, error states
6. Check agent-browser console and agent-browser errors periodically for JavaScript issues

Be thorough. Go through EVERY interaction, EVERY form field, EVERY button. The goal is that by the time this finishes, every part of the UI has been exercised and screenshotted.

4b. Database Validation

After any interaction that should modify data (form submits, deletions, updates):

1. Query the database to verify records. Use the environment variable from Sub-agent 2's research for the connection string and the schema docs to know what to check.
   • Postgres: use psql directly — e.g., psql "$DATABASE_URL" -c "SELECT theme FROM profiles WHERE username = 'testuser'"
   • SQLite: use sqlite3 directly — e.g., sqlite3 db.sqlite "SELECT theme FROM profiles WHERE username = 'testuser'"
   • Other databases: write a small ad hoc script in the application's language, run it, then delete it
2. Verify:
   • Records created/updated/deleted as expected
   • Values match what was entered in the UI
   • Relationships between records are correct
   • No orphaned or duplicate records

4c. Issue Handling

When an issue is found (UI bug, database mismatch, JS error):

1. Document it: what was expected vs what happened, screenshot path, relevant DB query results
2. Fix the code — make the correction directly
3. Re-run the failing step to verify the fix worked
4. Take a new screenshot confirming the fix

4d. Responsive Testing

For the responsive testing task, revisit key pages at these viewports:

• Mobile: agent-browser set viewport 375 812
• Tablet: agent-browser set viewport 768 1024
• Desktop: agent-browser set viewport 1440 900

At each viewport, screenshot every major page. Analyze for layout issues, overflow, broken alignment, and touch target sizes on mobile.

After completing each journey, mark its task as completed with TaskUpdate.

Phase 5: Cleanup

After all testing is complete:

1. Stop the dev server background process
2. Close the browser session: agent-browser close

Phase 6: Report

Text Summary (always output)

Present a concise summary:

## E2E Testing Complete

**Journeys Tested:** [count]
**Screenshots Captured:** [count]
**MCP Trace Records:** [count] (snapshots: [N], screenshots: [N])
**Issues Found:** [count] ([count] fixed, [count] remaining)

### Issues Fixed During Testing
- [Description] — [file:line]

### Remaining Issues
- [Description] — [severity: high/medium/low] — [file:line]

### Bug Hunt Findings (from code analysis)
- [Description] — [severity] — [file:line]

### Screenshots
All saved to: `e2e-screenshots/`

Markdown Export (ask first)

After the text summary, ask the user:

"Would you like me to export the full testing report to a markdown file? It includes per-journey breakdowns, all screenshot references, database validation results, and detailed findings — useful as context for follow-up fixes or GitHub issues."

If yes, write a detailed report to e2e-test-report.md in the project root containing:

• Full summary with stats
• Per-journey breakdown: steps taken, screenshots, database checks, issues found
• All issues with full details, fix status, and file references
• Bug hunt findings from the code analysis sub-agent
• Recommendations for any unresolved issues