playwright-site-investigator

name	playwright-site-investigator
description	Decision-tree workflow for Playwright-based site investigation. Use when a site's pytest test passes but Kodi playback is broken, a new site needs probing before a scraper is written, or a Cloudflare-blocked site needs investigation.

Playwright Site Investigator

Playwright is a dev/debug tool only. Never use it inside site modules — it is not available in the Kodi runtime. For site module fixes, use kodi-site-maintenance after this investigation is complete.

Which branch are you in?

Situation	Go to
Test passes but Kodi playback is broken	Branch 1
New site — no scraper exists yet	Branch 2
Site blocks urllib/requests (Cloudflare/JS)	Branch 3

---

Branch 1: Test passes, playback broken

The scraper runs without error and your test still passes against saved fixtures — but the URL it returns no longer plays in Kodi. The site changed its stream delivery; you need to find the new URL pattern.

Step 1 — Confirm the test still passes

.venv/bin/pytest tests/sites/test_<site>.py -v

Expected: all tests PASS. If tests are failing, stop — use kodi-site-maintenance instead.

Step 2 — Sniff the live stream URL

Navigate to a video page on the site in your browser and copy a video URL. Then run:

python scripts/playwright_sniff.py <video-page-url>

Playwright will launch Chromium headlessly, navigate to the page, click any visible play button, and intercept all network responses. Lines prefixed with [>>] are stream candidates.

Example output:

[*] Starting Playwright sniffer for: https://example.com/videos/12345
[*] Navigating...
[*] Found play button (button.vjs-big-play-button), clicking...
[*] Waiting 10s for more network activity...
[>>] Found Video/Stream URL: https://cdn.example.com/hls/12345/master.m3u8
[*] Finished.

The [>>] lines are what the site actually serves. Compare these to what your scraper currently extracts.

Step 3 — If sniff returns no [>>] lines

The stream URL was not captured via network interception. This usually means the player uses a blob URL, a DRM token, or renders the source only after a deeper interaction. Fall back to inspecting the rendered HTML:

python scripts/playwright_listing_probe.py --url <video-page-url> --selector "video,source,[data-src],[data-video-url]"

This prints a count of matched elements and a sample of link titles. Use it to confirm the selector is hitting the right cards. To inspect raw HTML attributes (src, data-src, etc.), open the page in a browser and use DevTools → Elements.

If neither tool returns a usable URL, the site likely requires a signed token or API call — check the browser's Network tab manually (DevTools → Network → filter by .m3u8 or .mp4).

Step 4 — Identify the delta

Compare the URL playwright_sniff.py found with what the current scraper extracts. Common causes of breakage:

Symptom	Likely cause
Scraper returns a page URL, not a stream	Domain change — site moved video hosting to a new CDN
Scraper returns a URL that 404s	API endpoint version bumped (e.g. /api/v2/ → /api/v3/)
Scraper returns a URL but Kodi says "invalid"	Token or hash parameter added to stream URL
Scraper returns nothing	New iframe redirect layer wrapping the player

Step 5 — Hand off

You now know what URL the site serves. To update the scraper to extract it, use the kodi-site-maintenance skill.

---

Branch 2: New site, no scraper yet

Use playwright_listing_probe.py to inspect the listing structure before writing any scraper code. Pass the main listing URL and adjust --selector to match the card container:

python scripts/playwright_listing_probe.py --url <listing-url> --selector ".video-item,.thumb,.item"  # adjust to the site's card selector

For the full new-site workflow (creating the scraper, registering it, writing tests), use kodi-site-maintenance.

---

Branch 3: Cloudflare / JS-rendered site

If utils.getHtml returns a Cloudflare challenge page or empty content, confirm Playwright can get through:

python scripts/playwright_sniff.py <site-listing-url>

If [>>] lines appear in the output, Playwright can reach the site and it requires flaresolverr or Playwright-based fetching in the scraper. For setup and integration details, use kodi-site-maintenance.