web-fetch

name	web-fetch
description	Internal support capability for routing public-web fetch requests through the strongest honest access path and classifying blocked or partial fetch responses without turning those tactics into a public workflow concept.

Web Fetch

This is a support capability, not a public workflow concept.

Users should still reach this through a public skill such as gather when a public URL needs stronger retrieval tactics than a plain direct fetch.

Runtime Contract

• prefer a host-provided web grant or direct public fetch first
• use domain-specific public APIs or URL transforms when direct fetch is blocked or known to be weaker
• keep public-web routing and response classification here so public skills do not duplicate blocked-site heuristics
• preserve retrieval method and confidence in any durable artifact that uses this support capability

Use the helpers. Resolve $SKILL_DIR per ../../shared/references/bootstrap-resolution.md, then run:

python3 "$SKILL_DIR/scripts/route_public_fetch.py" --url "https://www.reddit.com/r/python/"
python3 "$SKILL_DIR/scripts/classify_fetch_response.py" --path /tmp/fetch-response.html
python3 "$SKILL_DIR/scripts/acquire_public_url.py" --url "https://example.com/article"

Guardrails

• Do not present a routing recommendation as proof that the site is currently reachable.
• Do not hide external binary dependencies such as gh or yt-dlp; record them in the selected route.
• Use curl_cffi, defuddle, Patchright, and agent-browser as explicit fallback stages with trace records, not as invisible magic. Headless fallbacks stay read-only by default: impersonate browser-like fetches, render, extract text/HTML, and inspect network candidates for collection intent without clicking through auth, submitting forms, or solving challenges.
• Do not turn gather into a giant site-by-site cookbook when this support seam can carry the fetch tactics.

References

• references/runtime-contract.md
• references/routing-table.md
• scripts/route_public_fetch.py
• scripts/classify_fetch_response.py
• scripts/acquire_public_url.py