run-e2e-tests

Run E2E (end-to-end) tests to verify complete user workflows work correctly. ## When to Use This Skill - Before submitting a PR with significant changes - After modifying environment discovery, creation, or selection logic - Before a release to validate full workflows - When user reports a workflow is broken **Note:** Run smoke tests first. If smoke tests fail, E2E tests will also fail. ## Quick Reference | Action | Command | | ----------------- | -------------------------------------------------------------- | | Run all E2E tests | `npm run compile && npm run compile-tests && npm run e2e-test` | | Run specific test | `npm run e2e-test -- --grep "discovers"` | | Debug in VS Code | Debug panel → "E2E Tests" → F5 | ## How E2E Tests Work Unlike unit tests (mocked) and smoke tests (quick checks), E2E tests: 1. Launch a real VS Code instance with the extension 2. Exercise complete user workflows via the real API 3. Verify end-to-end behavior (discovery → selection → execution) They take longer (1-3 minutes) but catch integration issues. ## Workflow ### Step 1: Compile and Run ```bash npm run compile && npm run compile-tests && npm run e2e-test ``` ### Step 2: Interpret Results **Pass:** ``` E2E: Environment Discovery ✓ Can trigger environment refresh ✓ Discovers at least one environment ✓ Environments have required properties ✓ Can get global environments 4 passing (45s) ``` **Fail:** Check error message and see Debugging section. ## Debugging Failures | Error | Cause | Fix | | ---------------------------- | ---------------------- | ------------------------------------------- | | `No environments discovered` | Python not installed | Install Python, verify it's on PATH | | `Extension not found` | Build failed | Run `npm run compile` | | `API not available` | Activation error | Debug with F5, check Debug Console | | `Timeout exceeded` | Slow operation or hang | Increase timeout or check for blocking code | For detailed debugging: Debug panel → "E2E Tests" → F5 ## Prerequisites E2E tests have system requirements: - **Python installed** - At least one Python interpreter must be discoverable - **Extension builds** - Run `npm run compile` before tests - **CI needs webpack build** - Run `npm run compile` (webpack) before tests, not just `npm run compile-tests` (tsc) ## Adding New E2E Tests Create files in `src/test/e2e/` with pattern `*.e2e.test.ts`: ```typescript import * as assert from 'assert'; import * as vscode from 'vscode'; import { waitForCondition } from '../testUtils'; import { ENVS_EXTENSION_ID } from '../constants'; suite('E2E: [Workflow Name]', function () { this.timeout(120_000); // 2 minutes let api: ExtensionApi; suiteSetup(async function () { const extension = vscode.extensions.getExtension(ENVS_EXTENSION_ID); assert.ok(extension, 'Extension not found'); if (!extension.isActive) await extension.activate(); api = extension.exports; }); test('[Test description]', async function () { // Use real API (flat structure, not nested!) // api.getEnvironments(), not api.environments.getEnvironments() await waitForCondition( async () => (await api.getEnvironments('all')).length > 0, 60_000, 'No environments found', ); }); }); ``` ## Test Files | File | Purpose | | ----------------------------------------------- | ------------------------------------ | | `src/test/e2e/environmentDiscovery.e2e.test.ts` | Discovery workflow tests | | `src/test/e2e/index.ts` | Test runner entry point | | `src/test/testUtils.ts` | Utilities (`waitForCondition`, etc.) | ## Notes - E2E tests are slower than smoke tests (expect 1-3 minutes) - They may create/modify files - cleanup happens in `suiteTeardown` - First run downloads VS Code (~100MB, cached in `.vscode-test/`) - For more details on E2E tests and how they compare to other test types, refer to the project's testing documentation.