playwright

name	playwright
description	Browser/data automation via run_playwright_code: tool contract, parallel dispatch, expect() rules, retry strategy classes, and result interpretation.

Playwright Skill

Tool: run_playwright_code

One-shot TypeScript Playwright execution for browser automation and HTTP/API probing.

Inputs

Param	Required	Description
code	yes	Complete TS test using @playwright/test
save_as	no	Base filename, saved as tests/<save_as>.spec.ts
timeout	no	Max seconds (1..600), default per-call timeout

Output Contract — read these every time

Field	Meaning	Use it for
ok	execution_ok && data_ok	Quick "did the whole thing work" check
execution_ok	Process exited 0	Was there a crash?
data_ok	No data-failure warnings	Did we actually get useful data?
warnings	Machine-readable caveats	What went wrong (HTTP error, empty results, JSON parse, etc.)
exit_code, stdout, stderr	Raw process output	Debugging
timed_out	Hit timeout	Retry with larger timeout or different source
setup_error	Runtime broken	Stop — do not retry; report to user

Hard Rules

1. Never claim retrieval success when execution_ok=true but data_ok=false. The process ran but returned nothing useful.
2. Always check warnings before responding. Common blockers:
   • http_4xx_or_5xx_detected → endpoint failed
   • json_parse_failure_detected → response wasn't JSON
   • empty_collection_detected / no_results_detected → 0 results
   • assertion_received_false → an expect() failed
3. setup_error=true → stop and report. Do NOT retry; the runtime is broken.

---

Pre-Flight Decision Tree

Before writing TS code, answer:

1. What kind of source?

   • JSON / RSS / XML API → use request fixture (Class A — fast, no JS)
   • Rendered HTML page (search results, articles) → use page fixture (Class B)
   • Both available → start with Class A

2. Does this need parallel calls?

   • Multiple independent sources for same topic → batch all in one response
   • Sequential dependency (URL from search → article read) → emit search alone first

3. Is 0 results a valid outcome?

   • Yes (search queries, optional fields) → NO expect() on counts
   • No (required schema fields, mandatory HTTP 200) → use expect()

---

Parallel Dispatch

Multiple run_playwright_code calls without inter-dependencies must be issued in the same response. This is the single biggest budget multiplier.

Batch (parallel)

Scenario	Action
Shotgun search across sources	All run_playwright_code calls in one response
Read multiple articles (URLs known)	All reads in one response
API + page scrape, same topic	Both in one response
Search + known static endpoint (e.g. Wikipedia API)	Both in one response

Sequence (separate calls)

Scenario	Why
Article read using URL from prior search	URL not known until search returns
Test B's code contains a placeholder filled from A's stdout	Hard data dependency

Inside One Tool, Not Two

Multi-step API flows (fetch IDs → fetch details with those IDs) belong inside a single run_playwright_code test using sequential await calls.

---

Code Patterns

1. Prefer request for machine-readable sources

JSON, RSS, XML APIs are faster and more reliable via request than via page.

2. expect() Usage Rule

Use expect() when failure = hard error:

• Non-2xx HTTP on a mandatory endpoint
• Missing required schema fields (e.g. body.items must be an array)

Do NOT use expect() when 0/empty is a valid outcome:

• Result counts in search queries (results.length > 0 will crash on empty searches)
• Body length on article reads (paywalls return short bodies legitimately)
• Optional fields

A failed expect() crashes the test → execution_ok=false → masks the real "source returned nothing" signal.

3. Always print structured JSON to stdout

Downstream reasoning uses stdout parsing. Format:

console.log(JSON.stringify({ source: 'name', count: results.length, results }, null, 2));

Example A — API with required structure (use expect())

import { test, expect } from '@playwright/test';

test('fetch API', async ({ request }) => {
  const res = await request.get('https://example.com/api/items', { timeout: 15000 });
  expect(res.ok()).toBeTruthy();              // 2xx is mandatory
  const body = await res.json();
  expect(Array.isArray(body.items)).toBeTruthy();   // schema check
  // NO expect on body.items.length — 0 results is valid
  console.log(JSON.stringify({ source: 'api', count: body.items.length, items: body.items.slice(0, 5) }, null, 2));
});

Example B — Search/scrape (NO count assertions)

import { test } from '@playwright/test';

test('search', async ({ page }) => {
  await page.goto('https://example.com/search?q=TOPIC', { waitUntil: 'domcontentloaded', timeout: 20000 });
  await page.waitForSelector('.result', { timeout: 10000 }).catch(() => {});
  const results = await page.$$eval('.result', els => els.slice(0, 10).map(el => ({
    title: el.querySelector('h2')?.textContent?.trim() || '',
    url: el.querySelector('a')?.getAttribute('href') || ''
  })).filter(r => r.title && r.url));
  console.log(JSON.stringify({ source: 'example', count: results.length, results }, null, 2));
});

---

Strategy Classes & Retry

When a class fails, switch class — never retry the same class with tweaked args.

Class	Use when	Switch to on failure
A — request	JSON/RSS/API, fast, no JS needed	B (page) or C (alternative source)
B — page	Rendered UI, JS-heavy, dynamic content	A (find an API endpoint) or C
C — alternative source	Same data, different provider	Stop, report best-effort

Retry Discipline

• One failure per class — then switch.
• Two failures total → stop, synthesize with partial data, note gap.
• setup_error → stop immediately, report runtime issue.
• Never retry with same code + same args.

---

Result Interpretation Checklist

Before delivering an answer based on a tool result:

1. ☐ setup_error is false
2. ☐ timed_out is false
3. ☐ execution_ok is true
4. ☐ data_ok is true
5. ☐ warnings has no blocking entries (http_4xx_or_5xx_detected, assertion_received_false, no_results_detected, etc.)
6. ☐ stdout content is on-topic — not a CAPTCHA page, bot-detection interstitial, redirect loop, or foreign-language fallback. A CAPTCHA page can still set data_ok=true if stdout is non-empty — check the actual content.

If any item fails, retrieval is incomplete. Report it and switch class/source per retry rules above.

---

Large Output / Context Eviction

• If a previous tool result was evicted ([tool output cleared]), re-run the tool — do not infer missing values.
• If the result was saved to large_tool_results/, re-read with read_file(path, offset, limit).
• Never quote a stat or URL you cannot see in current context.

---

Common Failure Modes

Symptom	Likely cause	Fix
data_ok=false, no_results_detected	Search returned nothing	Switch source, NOT same source with tweaks
execution_ok=false, assertion_received_false	Bad expect() on count	Remove the count assertion
timed_out=true	Page slow / blocked	Try request fixture or different source
http_4xx_or_5xx_detected	API rejected	Wrong endpoint, missing auth, or rate-limited — switch source
Stdout has CAPTCHA HTML	Bot detection	Switch to API or different engine (see websearch skill)