roll-playwright

# Roll playwright-go to a new Playwright version Roll the client from the currently-pinned version to a target version (e.g. `1.61.0`). Work autonomously; only stop for the decision points called out in **When to ask the user**. The job isn't done when the PR opens — it's done when CI is green (Step 11), so expect to iterate on cross-browser/cross-OS failures that don't reproduce locally. ## What a roll actually is playwright-go vendors `microsoft/playwright` as a git submodule (`playwright/`) pinned to a tag. The public Go API (`generated-*.go`) is generated from the submodule's `docs/src/api/*.md` by `playwright/utils/doclint/generateGoApi.js`. Those `.md` files gate each API by language with a `* langs:` line; **the Go client only sees an API if `go` is in that line.** `patches/main.patch` is a diff applied onto the upstream submodule that (a) adds the Go generator and (b) adds `go` to the `* langs:` of every API the Go client exposes. So a roll = bump the version, re-apply the patch onto the new submodule, add `go` to the langs of any **newly-added** APIs we want, regenerate, build, test. ## Inputs - Target version, e.g. `1.61.0` (the `v` is optional). If not given, default to the latest stable from `gh release list --repo microsoft/playwright --limit 5` and confirm with the user. - Current version is read from `run.go` (`const playwrightCliVersion`). ## Prerequisites (check, install if missing) `gh` (authenticated), `node`, `go`, `gofumpt` (`go install mvdan.cc/gofumpt@latest`), `jq`. The submodule must be initialized: `git submodule update --init`. ## Step 1 — Discover what changed Run the helper, which prints the upstream docs diff, every new `* since: vNEW` API with its `langs:` line, and the sibling roll PRs: ```bash bash .claude/skills/roll-playwright/find-changes.sh <NEW_VERSION> ``` **Required: locate the actual roll PR in all three sibling clients** (python, java, dotnet) before porting — you need all three as references, not just one. They are the source of truth for which new APIs are real client surface, their exact method/param names, types, defaults, optionality, AND which upstream PRs were skipped. The helper prints candidates; for each repo open the right PR and confirm it's the one that ports the API surface (not a no-op bump): ```bash gh pr view <num> --repo microsoft/playwright-python # body enumerates ported APIs gh pr view <num> --repo microsoft/playwright-java # body links each upstream PR ported + what was skipped gh pr view <num> --repo microsoft/playwright-dotnet ``` If a repo's helper line is missing or looks like a no-op, dig with the queries in `references/finding-changes.md` until you have a real roll PR for **each** of the three. Use python as the primary shape reference (closest to Go), java for the per-upstream-PR breakdown, dotnet for option/enum shapes. > **Critical:** the heavy API port usually lands in the **`-beta` / `-alpha`** roll PR, not the final `X.Y.0` PR (which is often a one-line version bump — e.g. python v1.60.0 final roll was +1/−1, the beta roll +2194/−226). Read the beta/feature roll for the real changes. Build a checklist of new/changed APIs to expose in Go, AND a checklist of the **new tests** each sibling added (you'll port these in Step 6b). The Go client deliberately omits some upstream APIs (e.g. some Electron/Android/JS-internal surface) — match python's choices unless there's reason not to. > **Watch for return-type naming:** new return-object types carry a `* alias:` in the docs (e.g. `* alias: VirtualCredential`, `* alias: WebStorageItem`). The Go generator does NOT honor it by default and falls back to a generic method-derived name (`Create`, `Get`, `Item`). If you see a generic/duplicate generated struct name, add a guarded special-case in `generateGoApi.js` `generateNameDefault` (scoped by `parent.name`) mapping it to the documented alias — see `references/patch-editing.md`. > **Watch for pure getters:** a new no-arg getter (e.g. `Page.LocalStorage()`, `BrowserContext.Credentials()`) generates with a trailing `error` return unless its method name is in `methodNoErrArray` in `generateGoApi.js`. Add it there so it matches sibling getters like `Clock()`/`Mouse()`. ## Step 2 — Bump the version 1. Edit `run.go`: set `const playwrightCliVersion = "<NEW_VERSION>"`. 2. Download the new driver (also advances browser versions for the README step): ```bash go run scripts/install-browsers/main.go ``` ## Step 3 — Apply the patch onto the new submodule ```bash bash scripts/apply-patch.sh ``` This checks out the new tag in `playwright/`, creates branch `playwright-build`, and runs `git apply --3way patches/*`. Two outcomes: - **Clean apply** → continue to Step 4. - **Merge conflicts** (`.rej` files, or `<<<<<<<` markers in `playwright/docs/src/api/*.md`) → upstream edited a line the patch also touched. Resolve them: keep the upstream change AND re-add `go` to the `* langs:` line. Then `cd playwright && git add -A && git commit -am "apply patch" && cd ..`. ## Step 4 — Add `go` to new APIs (the real work) For each new API from your Step 1 checklist that you want in Go, edit the matching block in `playwright/docs/src/api/class-*.md` (and `params.md`) so its `* langs:` line includes `go`. The patch in Step 5 is regenerated from these edits. ```bash cd playwright git reset --soft HEAD~1 # un-commit the applied patch so edits fold into one diff (keeps working tree) # ...edit docs/src/api/*.md: add `go` to the `* langs:` lines of the new APIs... git add -A && git commit -m "apply patch" cd .. ``` Rules for editing langs (see `references/patch-editing.md` for details and examples): - `* langs: js, python, csharp` → `* langs: js, python, csharp, go` (append, comma+space). - A Go-only addition uses `* langs: go`. - Keep edits surgical — only touch lines for APIs you're intentionally exposing. - Mirror the python client's method/param naming and optionality; the Go generator turns these docs into `generated-*.go`. ## Step 5 — Regenerate the patch and the Go API ```bash bash scripts/update-patch.sh # regenerates patches/main.patch from the playwright-build branch go generate ./... # runs generate-api.sh: re-applies patch, runs the JS generator, gofumpt ``` `go generate` rewrites `generated-enums.go`, `generated-interfaces.go`, `generated-structs.go`. Review the diff — it should reflect exactly the APIs you added. > **Gotcha:** `generate-api.sh` ends with `git submodule update`, which resets `playwright/` back to the parent repo's *currently pinned* (old) commit. After generation you MUST bump the gitlink yourself: `(cd playwright && git checkout v<NEW_VERSION>)`, then confirm with `git submodule status playwright`. Otherwise the committed submodule pointer still references the old version. Do this as part of Step 9. ## Step 6 — Hand-written wiring The generator produces interfaces/structs, but most new APIs need a hand-written implementation in the corresponding `*.go` file (e.g. a new `Page` method goes in `page.go`, often with a channel send). Use the sibling clients' impl (python `_impl/*.py`, java/dotnet `Core`/`impl`) as the behavior reference. Look at a previous roll commit for the pattern: ```bash git show $(git log --oneline --grep="[Rr]oll to" -1 --format=%H) --stat # files a past roll touched ``` New thin wrapper classes (not channel owners) follow `clock.go`: a struct holding the parent `*pageImpl`/`*browserContextImpl`, constructed in the parent's `newPage`/`newBrowserContext`, exposed via a getter, methods call `parent.channel.Send(...)`. Wire the getter field into the parent struct too. Watch for **protocol behavior changes** (not just new APIs) — these don't show as build errors but break at runtime. Check the sibling roll PR bodies and the upstream `validator.ts` diff (`gh api .../compare/vOLD...vNEW`) for renamed channel methods, split methods, or changed result shapes. Example from v1.61: assertion `expect` stopped returning `{matches}` and now throws a protocol error carrying `errorDetails` — every assertion path had to be updated. **Grep for duplicated logic** (e.g. there were TWO expect call-sites: `locatorImpl.expect` and `pageAssertionsImpl.expectOnFrame`) so you fix all of them. ### Step 6b — Port EVERY sibling test (required) **Mandatory and exhaustive: every test python, java, AND dotnet added in their roll PR must have a Go counterpart in `tests/`.** Do not cherry-pick — enumerate all of them and port each, following Go conventions (`BeforeEach(t)`, `require`, `server.EMPTY_PAGE`, per-browser branches via `isChromium`/`isFirefox`/`isWebKit`). This is both for coverage parity and because **a sibling test frequently exposes a feature that is broken or missing in Go** — porting it is how you find that. > Real example (v1.61): java's `getByTestIdWithCommaSeparatedTestIdAttributesShouldMatchAny` had no Go equivalent. Porting it revealed that comma-separated `SetTestIdAttribute` produced an invalid selector in Go (the `encodeTestIdAttributeName` quoting was missing). The test caught a real bug — the impl had to be fixed, not just the test added. Build a complete checklist. List every test function each sibling added, then tick off the Go equivalent: ```bash # 1. List the test FILES each sibling roll added/changed: for repo in microsoft/playwright-python microsoft/playwright-java microsoft/playwright-dotnet; do echo "## $repo"; gh pr view <num> --repo $repo --json files --jq '.files[].path' | grep -iE 'test|spec' done # 2. Extract the test FUNCTION NAMES added (so nothing is missed): gh pr diff <pyNum> --repo microsoft/playwright-python | grep -E '^\+\s*(async )?def test_' gh pr diff <javaNum> --repo microsoft/playwright-java | grep -E '^\+\s*(public )?void ' gh pr diff <netNum> --repo microsoft/playwright-dotnet | grep -iE '^\+\s*public async Task|\[(Playwright)?Test\]' # 3. For each, read the body and port the cases: gh pr diff <num> --repo <repo> ``` For each new API, dedupe across the three (they overlap heavily) and port the **union** of their cases — e.g. python tests session-storage clear separately, java tests local/session independence; port both. If a sibling test can't be ported (feature intentionally not in Go, or needs an unavailable fixture like an HTTPS server), `log` it explicitly with the reason rather than silently dropping it. These tests are also your verification that the impl works — run them across all three browsers in Step 7 (a test green on chromium may fail on firefox/webkit due to engine-specific error strings or behavior). ## Step 7 — Build, lint, test ```bash gofumpt -l -w . go build ./... go vet ./... golangci-lint run ./... # CI's Lint job; fails on errcheck/staticcheck/gofumpt # Run the NEW tests on every browser — CI is a 3 OS x 3 browser matrix and # engine-specific behavior (error strings, etc.) differs. A pass on chromium # alone is NOT enough. for b in chromium firefox webkit; do BROWSER=$b HEADLESS=1 go test -race ./tests/ -run '<NewTests>'; done go test -race ./... # full suite at least once ``` Lint commonly fails on test helpers: unchecked `defer x.Close()`/`Stop()` (errcheck → add `//nolint:errcheck`), deprecated `WaitForTimeout` (staticcheck → use `time.Sleep`), and gofumpt alignment. Run `golangci-lint run` locally; the CI Lint job is fast but blocking. Cross-browser is not optional: a v1.61 client-cert assertion passed on chromium but failed on firefox (`SSL_ERROR_UNKNOWN`), webkit-linux/macos (`Certificate is required`), and webkit-windows (`Failure when receiving data`) — each emits a different error string. Match any known variant rather than one browser's wording. The `verify_type_generation` CI job runs `go generate` and fails if it produces a diff — so committed `generated-*.go` must exactly match a fresh generation. Run `go generate ./...` once more and confirm `git diff --ignore-submodules` is clean for the generated files. ## Step 8 — Update README versions Already handled inside `scripts/generate-api.sh` (it runs `update-readme-versions/main.go`), but if you ran steps manually, run `go run scripts/update-readme-versions/main.go` to refresh the browser-version badges. ## Step 9 — Verify parity against the sibling clients Before committing, run the parity verifier. It cross-checks this roll's diff against what python/java/dotnet did, so you catch anything missed: a new `* since: vNEW` API the siblings expose but you didn't add `go` to, an API you exposed that no sibling did, or a sibling test you haven't ported. ```bash bash .claude/skills/roll-playwright/verify-parity.sh <NEW_VERSION> ``` It reports three buckets: 1. **New upstream APIs not in Go** — each `* since: vNEW` block whose `* langs:` still lacks `go`. For each, decide: intentionally omitted (matches a sibling skipping it) or a miss to fix in `patches/main.patch`. 2. **Sibling tests added this roll** — the test files AND the individual test function names from all three clients, so you can tick off that each has a Go counterpart in `tests/` (Step 6b). 3. **Go API symbols added vs sibling API surface** — a rough name-level diff to spot a method you exposed that no sibling did (often a naming mistake) or vice-versa. Treat each flagged item as needing an explicit decision: fix it, or note why Go intentionally differs. The goal is that any remaining difference from python/java/dotnet is deliberate and explainable. ## Step 10 — Commit & PR Branch name should start with `roll/` (CI triggers on `roll/*`). Match the existing commit style: ```bash # Bump the submodule gitlink to the new tag (generate-api.sh reset it — see Step 5 gotcha): (cd playwright && git checkout v<NEW_VERSION>) git submodule status playwright # confirm it shows v<NEW_VERSION> git diff --submodule=log -- playwright # confirm the pointer moves OLD -> NEW git checkout -b roll/v<NEW_VERSION> git add -A && git commit -m "chore: roll to Playwright v<NEW_VERSION>" git push -u origin roll/v<NEW_VERSION> gh pr create --repo mxschmitt/playwright-go --base main \ --title "chore: roll to Playwright v<NEW_VERSION>" --body "<summary of new APIs + behavior changes>" ``` Create the PR when the local checks (Step 7/9) pass — don't wait to be asked. Summarize the new APIs and any behavior changes in the body, and **always end the body with this footer** so it's obvious at a glance the PR was skill-generated rather than hand-rolled (write the body to a temp file and pass `--body-file` if it's easier than escaping a multi-paragraph `--body` string): ``` --- 🤖 Rolled with the [`roll-playwright`](.claude/skills/roll-playwright/SKILL.md) skill. Sibling references used: python <PR_URL>, java <PR_URL>, dotnet <PR_URL> verify-parity.sh (Step 9): <clean | N items flagged and how each was resolved> ``` Fill in the three sibling PR URLs found in Step 1 — if any was unavailable or skipped, say so explicitly on that line rather than omitting it (this is also a forcing function: don't skip the Step 1 requirement to check all three). Then drive it to green (Step 11). ## Step 11 — Drive CI to green (loop until all checks pass) The roll is **not done when the PR opens — it's done when CI is green.** A roll PR runs a large matrix (3 OS × 3 browsers × 2 Go versions) plus Lint, verify (type generation), and test-examples. Local chromium-green is **not** sufficient — most roll failures surface only on firefox/webkit or on Windows. Loop autonomously until every check passes: 1. **Wait for the run to finish** (don't poll in a tight loop — block on it): ```bash # Find the run for the branch's head commit, then watch it to completion: RUN=$(gh run list --repo mxschmitt/playwright-go --branch roll/v<NEW_VERSION> \ --limit 1 --json databaseId --jq '.[0].databaseId') gh run watch "$RUN" --repo mxschmitt/playwright-go --exit-status # (exits non-zero if the run failed; `gh pr checks <num> --watch` is an alternative.) ``` 2. **List results and read every failure's log** (the failed step, not the whole log): ```bash gh pr checks <num> --repo mxschmitt/playwright-go gh run view --repo mxschmitt/playwright-go --job <jobId> --log-failed \ | grep -iE 'FAIL|Error:|\.go:[0-9]+|panic' ``` 3. **Triage each failure** the same way as Step 6 / "behavior changes": is it my code, an engine/OS-specific string (match any known variant), a lint issue, or an intended upstream change? Note the OS+browser — a failure on only webkit-windows is still a failure. 4. **Fix, re-verify locally on the affected browser(s), and re-push** (amend the relevant commit to keep history clean; `--force-with-lease`). Re-run from step 1. 5. Repeat until `gh pr checks` shows **0 fail** and no relevant pending. - **Flakiness.io is a reporter, not a merge gate** — it shows "fail" only because a test job it reports on failed; it goes green when the tests do. Don't chase it directly. - A genuinely flaky (not roll-caused) test that fails once may pass on re-run; confirm it's unrelated to your changes before re-running rather than fixing. The repo uses `flakiness-go` (no auto-retry).