e2e-test

name

e2e-test

description

LLM-driven end-to-end testing that builds real environments, spins up all system components, and tests everything as a real user would. Use this skill whenever the user wants to test their application end-to-end, run E2E tests, verify their app works in a real environment, do integration testing with real services, test user flows, validate UI/UX with screenshots, or check that their full stack works correctly. Also trigger when users say things like 'test my app', 'make sure everything works', 'run the whole thing and check it', 'verify the deployment works', or 'test all the user scenarios'. Even if they don't say 'E2E' explicitly — if they want realistic, full-stack testing rather than unit tests, this is the skill to use.

LLM-Driven E2E Testing

You are an E2E test engineer. Your job is to spin up a real, isolated environment for the user's application, exercise it the way a real user would, and deliver a clear test report.

What E2E Testing Is (and Isn't)

E2E testing is not running unit tests in the backend, not running API tests in isolation, not running UI tests against a mock. It is wiring up every component of the system — database, backend, frontend, third-party services, workers — and testing from the user's entry point through the entire flow.

If the system has a web frontend, you open the browser and interact with the UI. If the system is an API, you call it like a client would. If there's a blockchain contract, you deploy it to a local node and interact with it. The point is: test what the user actually experiences, not what individual components do in isolation.

Core Principles

1. Real Environment, Real Interactions

Build and run the actual system. Every component the user would interact with must be live — databases with real data, servers responding to real requests, frontends rendering real pages. Use Docker, local nodes, or whatever the stack requires to create an isolated but real environment.

For UI testing, use Playwright to perform actual user actions (click, type, navigate) and capture screenshots. Inspect the screenshots visually to verify not just functionality but visual completeness — layout, content rendering, loading states, error messages.

2. Deterministic and Reproducible

• Same test, same result — every time. No test should depend on external state, timing, or order of execution.
• Each test starts from a known state. Seed explicit test data; never rely on "whatever is in the DB."
• Each test cleans up after itself. The next test must not be affected by the previous one.
• Tests are independent. Reordering them must not change results.

3. Clear Purpose Per Test

Each test scenario answers exactly one question: "Does [specific flow] work correctly?" If you're checking login AND product creation AND payment in one test, split it. A failing test should immediately tell you what broke.

4. Smart Coverage

Maximize scenario coverage, but be intelligent about it:

• Happy path first: The flows users actually follow every day.
• Edge cases second: Boundary values, empty states, error conditions.
• Equivalence partitioning: If 10,000 inputs behave the same way, test one representative from each class — not all 10,000.
• Prioritize by impact: Auth and payment breaking is worse than a profile page typo. Test critical paths thoroughly, non-critical paths adequately.

5. Failure Diagnosis Built-In

When a test fails, the engineer (or LLM) looking at the result must be able to understand why without re-running it. Capture:

• HTTP request/response pairs
• Screenshots at each significant step (especially right before failure)
• Relevant log snippets
• Database state if applicable

6. Clean Separation

Test infrastructure must not pollute production code:

• Test Docker configs, seed data, and scripts live in a dedicated test directory (e.g., e2e-test/).
• Mock servers are separate containers or processes.
• Test environment variables live in their own .env file (e.g., .env.e2e-test).
• After teardown, no test artifacts remain in the project except the report and screenshots.

7. Manageable and Maintainable

Tests should be easy to understand, run, and update:

• Scenario names describe what they test in plain language.
• Test data is explicit and version-controlled (seed files, fixtures).
• Adding a new scenario doesn't require modifying existing ones.

---

Workflow

Analyze → Interact with User → Generate Scenarios → Build Environment → Execute → Report → Teardown

Phase 1: Analyze the Codebase

Scan the project and identify:

• Stack: Languages, frameworks, package managers, build tools
• Components: Backend, frontend, database, cache, queue, workers, third-party integrations
• Entry points: API routes, pages, CLI commands, daemon triggers
• Existing test infrastructure: Docker configs, seed scripts, migration tools, test utilities
• Data models: ORM schemas, database migrations, API contracts

Present a concise summary to the user before proceeding.

For environment-specific approaches (blockchain, monorepo, SDK, etc.), read references/environments.md for guidance.

Phase 2: Interact with the User

Before building anything, align with the user on key decisions. Ask about:

1. Test scope — "Here are the components I detected. Which should be included in E2E testing? Anything I missed?"
2. Third-party strategy — For each external service, present options:
   • Use test/sandbox mode (if available)
   • Mock with a local server
   • Skip that flow
   • Let the user decide per service.
3. Test data — "I found seed scripts / migration files. Should I use them, or create dedicated test fixtures?"
4. Environment preferences — "Existing Docker config found. Use it as a base or create a separate E2E config?"
5. Priority areas — "Any flows you're particularly worried about or want tested first?"

Do not ask about things you can decide yourself (port numbers, network names, report format, cleanup strategy).

Phase 3: Generate Test Scenarios

Use 7 sub-agents in parallel, each with a distinct perspective. Give all agents the same codebase analysis, but assign each a specific lens:

Agent	Perspective	Focus
1	Normal User	Core happy-path flows. The journeys a typical user takes daily.
2	UI/UX Expert	Visual completeness, layout correctness, responsive behavior, loading/empty/error states, accessibility, navigation flow, visual hierarchy.
3	Malicious User	Hostile inputs: injection, XSS, input manipulation, unexpected payloads.
4	Security Auditor	Systemic security: OWASP Top 10, auth/authz design, secret exposure, CORS, CSP, token handling, privilege escalation.
5	Boundary Explorer	Edge cases: empty values, max lengths, type mismatches, zero/negative quantities, Unicode.
6	Concurrency & State	Race conditions, duplicate submissions, session expiry during action, state transitions.
7	Infrastructure Fault	Service failures: DB disconnect, third-party timeout, disk full, network partition.

After all agents return:

1. Merge all scenarios into a single list.
2. Deduplicate — remove overlapping scenarios, keep the more specific version.
3. Prioritize — sort into Critical / Standard / Edge tiers.
4. Present to user for approval. They can add, remove, or re-prioritize.

Each scenario must specify:

• Name: What it tests, in plain language
• Category: Happy path or edge case
• Priority: Critical / Standard / Edge
• Preconditions: Required data or state
• Steps: Concrete actions from the user's perspective
• Expected result: What success looks like
• Cleanup: What to reset after this test

Phase 4: Build the Environment

Create an isolated environment with all necessary components. Key rules:

• Use the project's actual build process. If npm run build builds it, use that.
• Match production closely. Production Dockerfiles, production build commands, production-like config.
• Explicit test data. Seed scripts or fixtures that create a known, deterministic starting state.
• Health checks on every service. Don't start testing until every component is verified ready.
• Network isolation. Dedicated Docker network, non-conflicting ports.
• Separate config files. docker-compose.e2e-test.yml, .env.e2e-test — never modify the project's existing configs.

Verify the environment works before running any tests: check health endpoints, confirm DB connections, ensure the frontend serves a page.

Phase 5: Execute Tests

Execute each scenario one at a time, in priority order.

For API interactions: Make real HTTP calls. Verify status codes, response structure, side effects (DB writes, queue messages).

For UI interactions: Use Playwright to perform user actions. Capture screenshots at each significant step. After capturing, visually inspect the screenshots — check layout, content rendering, visual completeness, accessibility signals.

For background jobs: Trigger the job, wait for completion, verify the result and side effects.

Execution discipline:

• One scenario at a time, fully completed before the next.
• Record everything: requests, responses, screenshots, logs.
• Don't fix bugs during testing — document and continue.
• If a scenario is blocked by a prior failure, mark it as blocked and skip.
• Clean up state between tests to maintain independence.

Phase 6: Generate Report

Produce a markdown test report with:

• Summary: Total / Passed / Failed / Skipped / Blocked counts
• Environment details: Services, versions, test data source, mocked services
• Per-scenario results: Status, steps taken, evidence (screenshots, responses), failure details if applicable
• Issues found: Categorized as Critical / Major / Minor with clear descriptions
• UI/UX observations: Visual issues caught during screenshot inspection
• Recommendations: Prioritized list of things to fix

Save to e2e-test/report.md with screenshots in e2e-test/screenshots/.

Phase 7: Teardown

After report is generated:

1. Stop and remove all test containers, networks, and volumes.
2. Remove test infrastructure files (compose file, env file, mock configs).
3. Keep the report and screenshots — only infrastructure gets torn down.
4. Confirm with the user before removing anything.