wpt-update

# WPT Update ## Context WPT (Web Platform Tests) is an open-source test suite for assessing runtime conformance with Web standards. workerd keeps configuration under `src/wpt` because some tests are not applicable or cover behavior workerd does not yet implement. A request to bump or update WPT includes both the dependency update and triage unless the user explicitly requests a bump-only change. ## Scope Allowed edits are dependency metadata produced by the updater, WPT config files under `src/wpt`, and the WPT patches described in [WPT patches](#wpt-patches). Do not fix runtime conformance bugs during a WPT update. If correct triage requires runtime, harness, build-rule, upstream, or other dependency patch changes, report the required follow-up but don't make changes unless the user expands the scope. ## WPT patches workerd applies the patches under `patches/wpt` to the WPT archive. They are listed in the `patches` of the `wpt` entry in `build/deps/shared_deps.jsonc`. Unlike patches to other dependencies, maintaining these is in scope for every WPT update: they target WPT's own files, so upstream changes can stop them from applying or leave them incomplete. If most WPT targets fail during analysis with `Build did NOT complete successfully`, run `bazel build` on a single suite to see the root error. A patch that no longer applies to the new archive reports `CONTENT_DOES_NOT_MATCH_TARGET`. Rebase it onto the new upstream file and keep its intent. If you add, remove, or rename a patch, update the patch list and regenerate `build/deps/gen/shared_deps.MODULE.bazel` with `./build/deps/update-deps.py wpt`, which keeps the pinned `freeze_version`. Bazel deletes the extracted repository when a patch fails, but the archive remains in the repository cache at `$(bazel info repository_cache)/content_addressable/sha256/<sha256>/file`, using the `sha256` values from the old and new dependency metadata. Extract the old and new archives into separate empty directories under `$TMPDIR` to rebase patches and to compare the mapped WPT directories with `diff -rq`. The `serve` patch keeps `tools/serve/commands.json` from requesting a virtualenv. `tools/wpt/wpt.py` asserts that any command declaring `requirements` or `conditional_requirements` uses a virtualenv, so the patch must also remove any such keys that upstream adds to the `serve` entry. Describe each patch change and the upstream commit that required it in the final report. ## Preflight 1. Determine whether workerd is standalone or a submodule and read all applicable `AGENTS.md` files. 2. Inspect the worktree and preserve unrelated changes. 3. Load `ts-style` before editing TypeScript config files. ## Bumping WPT From the workerd root, run: ```bash ./build/deps/update_wpt.py ``` The updater selects the newest published `wpt-*` release from `cloudflare/workerd-tools`. A normal update changes only: * `build/deps/shared_deps.jsonc` * `build/deps/gen/shared_deps.MODULE.bazel` Verify that the new tag appears consistently in `freeze_version`, `strip_prefix`, and the archive URL, and that a new SHA-256 was generated. Do not hand-edit generated dependency output. The updater writes the manifest before downloading and regenerating metadata. If it fails, do not continue with inconsistent files. Resolve the failure, rerun the updater, and verify both files. The generic dependency updater may mention an unrelated available release because `workerd-tools` contains several tools. Determine success from the selected WPT tag and resulting diff. If the updater reports that WPT is current, stop for a bump-only request. Keep the dependency changes separate from triage changes. If commits were explicitly requested, commit the dependency update before moving on. Otherwise, continue without committing. ## Triaging a WPT update For a triage-only request, first verify that a pin change exists relative to the agreed base. Read the old and new `wpt-<sha>` values from the dependency diff. The suffix identifies the corresponding upstream WPT commit. Use pinned SHAs for all upstream research. Do not inspect files from upstream `main`. GitHub's compare API truncates large WPT updates, so use the complete diff when necessary: ```text https://github.com/web-platform-tests/wpt/compare/<old-sha>...<new-sha>.diff ``` Filter upstream changes to the WPT directories mapped by `src/wpt/BUILD.bazel`. For an individual failure, inspect the changed file, its commits, the associated pull request, and any linked specification change. ### Running WPT The commands below show the correct repository labels. Add all flags required by the active test-hygiene instructions. For standalone workerd, run from the workerd root: ```bash bazel test //src/wpt/... --test_size_filters= ``` For a submodule checkout, run from the parent repository root: ```bash bazel test @workerd//src/wpt/... --test_size_filters= ``` Clearing `test_size_filters` ensures that enormous WPT targets are included. The complete pattern runs default and all-autogates variants plus ESLint and TypeScript checks. If most WPT targets fail during analysis, see [WPT patches](#wpt-patches). Use an affected suite target while iterating. Include the required `@` suffix. For standalone workerd: ```bash bazel test //src/wpt:<suite>@ \ --test_size_filters= \ --test_output=errors ``` For a submodule checkout: ```bash bazel test @workerd//src/wpt:<suite>@ \ --test_size_filters= \ --test_output=errors ``` Do not use test-case filter arguments. Resolve the log root with `bazel info bazel-testlogs` or use the path printed by Bazel. Parent-repository logs are nested beneath the external workerd repository. Search logs first for these messages instead of reading them in full: ```text Missing test configuration unexpectedly failed unexpectedly succeeded Please update the test config Test file ... not found ``` ### Handling test suite-level changes A suite is a discovered WPT JavaScript file. Its config key is the exact, case-sensitive path relative to the `wpt_directory` declared in `src/wpt/BUILD.bazel`. Discovery excludes non-JavaScript files and JavaScript files whose paths contain `.tentative.` or `/tentative/`. Config keys MUST remain in ascending order as enforced by ESLint's `sort-keys` rule. Compare the complete raw key: punctuation affects ordering, so `-` sorts before `.`. Do not place a new key based only on visual grouping. 1. Check for renames or moves before treating files as additions and deletions. Transfer existing configuration across genuine renames. 2. Remove config keys for deleted suites. 3. Add each new suite as a sorted blank entry so its behavior can be observed: ```typescript 'path/to/example.any.js': {}, ``` Discovery includes JavaScript helper files unless their immediate parent is `resources`. Helpers outside those directories may need explicit omitted configuration even when they register no tests. Re-run the complete WPT set after reconciling suite keys. Repeat until there are no missing or stale configuration errors. After every config edit, run the aggregate `//src/wpt:wpt-all@tsproject@eslint` target with the repository prefix required by the checkout. Do not rely only on suite-specific ESLint targets. ### Handling test changes A subtest is an individual named `test()` or `promise_test()` within a suite. 1. Check whether removed and added names represent a renamed subtest. 2. When an expected failure now passes, determine whether workerd improved or upstream renamed, removed, or weakened the test before removing the expectation. 3. Investigate every new or newly failing subtest before classifying it. Re-run affected suites while iterating, then rerun the complete WPT set until there are no unexpected failures or successes. ### Analyzing failing tests For each unexpected result: 1. Read the exact subtest failure and stack. 2. Inspect the upstream file at the new pinned SHA. 3. Identify the commit and specification change that introduced it. 4. Inspect the relevant workerd implementation or dependency. 5. Decide whether the test is relevant, unsafe to run, or inapplicable. 6. Record the current cause and any out-of-scope fix direction. 7. Compare default and all-autogates behavior before using one shared entry. Do not classify an uncertain result merely to make the target pass. Runtime fixes are out of scope; explain the likely fix in the report instead. ### META and load failures META scripts and top-level code execute before subtests are registered. An `expectedFailures` entry cannot classify a missing resource or evaluation error at this stage. Partial `disabledTests` or `omittedTests` arrays cannot avoid it either. Whole-file `disabledTests: true` and `omittedTests: true` skip evaluation before META imports. Use either only when its classification is independently correct. Do not omit a relevant suite merely because harness support is missing. ### Classifications * `expectedFailures`: relevant subtests execute but currently fail assertions because workerd is non-conformant. * `disabledTests`: relevant tests cannot run reliably or safely, such as hangs, crashes, state corruption, or harness limitations. * `omittedTests`: tests are not applicable to workerd and are excluded from coverage. Each property accepts `true` for a whole suite or an array of exact strings and regular expressions. Prefer exact subtest names. Copy non-ASCII names with `\u` escapes when they contain decomposed or otherwise non-NFC text, because copying them through a terminal or editor can normalize them into different strings. Use a regex only when it is narrow, anchored where practical, and checked against every intended name. Do not use broad regexes or whole-suite `true` merely to make a target pass. Every classification requires a `comment` describing the current technical reason. Source comments must describe current behavior, not the investigation or history that produced the entry. Commit messages and reports may include the additional evidence and history. The harness reports unmatched `expectedFailures`, but a regex can still hide partially stale cases. It does not report stale disabled or omitted array entries. Audit those manually against renamed and removed upstream subtests. ### Verification Before completion: 1. Remove temporary `only` settings. 2. Confirm there are no missing or stale suite keys. 3. Confirm there are no unexpected failures or successes. 4. Manually audit changed disabled and omitted selectors. 5. Run the complete WPT target set using the applicable checkout command. 6. Require Bazel to report `Build completed successfully`. 7. Confirm the final `Executed N out of N tests: N tests pass` summary covers every selected target. 8. Do not treat an interrupted or partially analyzed invocation as success. 9. Run formatting and whitespace checks from the workerd root: ```bash python3 tools/cross/format.py --check git git diff --check ``` Running `python3 tools/cross/format.py git` without `--check` applies the formatting to the reported files. Rerun the affected targets afterwards. ### Continuous improvement Re-read the SKILL.md file. If there are any details that weren't clear or that you had to look up separately, edit the file to include them. Information that concerns how the WPT test framework works within workerd is valuable, but refrain from adding info about specific situations you encountered during a WPT update, as this is transient and not likely to be relevant to future WPT updates. ### Final report Report: * The old and new WPT tags and pinned upstream comparison URL. * WPT patch changes and the upstream commits that required them. * Added, removed, and renamed suites. * Expectations added or removed. * Each classification and its durable technical reason. * Relevant upstream and workerd source locations. * Proposed out-of-scope fixes and unresolved items. ### Git handoff Keep the generated dependency bump and WPT triage in separate commits.