菜单
Fetch and extract readable text content from web pages. Auto-selects best tool with smart fallback. Use when the user wants to read articles, documentation, or scrape text content from web pages. Works on Reddit, StackOverflow, GitHub, docs sites, and more.
用 Codex 或 Claude 帮你安装 复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。
基于 SOC 职业分类
Draw diagrams as code from text using the D2 language (d2lang.com). Use when the user wants to create, edit, validate, or render architecture diagrams, flowcharts, sequence diagrams, ER diagrams, class diagrams, or any .d2 file. Triggers: draw a diagram, architecture diagram, visualize the system, render d2, .d2 file.
Search the web with automatic backend selection. Works out-of-the-box with public SearXNG. Optional Duck API for advanced filters. Supports images, news, videos.
Download videos, audio, subtitles, and clean paragraph-style transcripts from YouTube and any other yt-dlp supported site. Transcripts are saved to file by default. Use when asked to download this video, save this clip, rip audio, get subtitles, get transcript, or to troubleshoot yt-dlp/ffmpeg and formats/playlists.
Search YouTube for fresh, long, deep content — podcasts, lectures, and deep dives. Use when the user wants to find videos, look up a topic, get video recommendations, or asks about YouTube content. Defaults to longform (20min+), recent (18mo), high-relevance results with a combined recency+views+duration+relevance score.
Drive headless Chrome from the CLI for web scraping, screenshots, form filling, PDF export, accessibility audits, and browser smoke tests. Use when the user wants to automate browser interactions, scrape dynamic pages (JS-rendered content), take page screenshots or element screenshots, fill and submit web forms, export pages as PDF, run accessibility checks, or assert page state in CI/CD pipelines. Triggers on mentions of: browser automation, headless Chrome, web scraping, page screenshots, form automation, accessibility testing, browser testing, rodney.
Create and update concise AGENTS.md files using proven best practices. Use when starting a new AGENTS.md, refactoring an overly long or confusing one, adding missing build/test commands, or capturing gotchas and verification steps. Triggers on: AGENTS.md, agent instructions, project guide, CLAUDE.md, cursor rules, copilot instructions, agent onboarding.
| name | fetch-url |
| description | Fetch and extract readable text content from web pages. Auto-selects best tool with smart fallback. Use when the user wants to read articles, documentation, or scrape text content from web pages. Works on Reddit, StackOverflow, GitHub, docs sites, and more. |
| metadata | {"author":"skale","version":"2.6"} |
fetch-url "https://example.com" # That's it.
Extracts text from any webpage. Works globally after install.
Linux / macOS (bash):
cd ~/.pi/agent/skills/fetch-url && ./install.sh
Windows (cmd):
cd %USERPROFILE%\.pi\agent\skills\fetch-url && install.bat
Creates fetch-url in ~/.local/bin/ (%USERPROFILE%\.local\bin\ on Windows). Requires uv (auto-installed).
fetch-url --update # Manual update (git pull + uv sync)
fetch-url --selfcheck # Show version + last update date
Auto-updates in background every 7 days.
fetch-url "https://example.com" # Auto-selects best tool
fetch-url "https://reddit.com/r/python" # Redirects to old.reddit.com, uses w3m
fetch-url "https://news.ycombinator.com" # Uses w3m (free, local)
fetch-url "URL" --tool jina # Force specific tool
fetch-url "URL" -v # Verbose (shows tool + redirects)
| Option | Description |
|---|---|
--tool NAME | w3m, lynx, jina, markdown, chrome, chawan |
-v, --verbose | Show tool selection and redirects |
--no-clean | Keep empty lines |
--update | Update the skill now |
--selfcheck | Show version and last update |
Priority: free local tools first (w3m, lynx), then free APIs (jina, markdown), then chrome.
| Tool | Best For | Cost |
|---|---|---|
| w3m | Reddit, HN, simple HTML sites | Free, local |
| lynx | Wikipedia, text-heavy sites | Free, local |
| jina | Docs, blogs, GitHub, Medium | Free API |
| markdown | StackOverflow, fallback | Free API (50/day) |
| chrome | Cloudflare/JS-protected sites | Free, needs Chrome |
| Site | Strategy |
|---|---|
| Auto-redirects to old.reddit.com, uses w3m | |
| HN | w3m (clean output) |
| Wikipedia | jina (cleanest), lynx/w3m fallback |
| GitHub | jina (clean markdown) |
| StackOverflow | markdown (bypasses blocks) |
| Medium | jina |
| Cloudflare sites | Chrome headless |
# Credgoo for API tool
credgoo add FETCH_URL_BEARER
# Optional browsers for better fallback
brew install w3m lynx chawan
| Problem | Fix |
|---|---|
fetch-url: command not found | Add ~/.local/bin to PATH |
| gunzip error on GitHub | Use --tool jina |
| Empty result | Try --tool jina or --tool chrome |
| Dependency error | fetch-url --update |
See references/sites.md for tool rankings per site. See references/github.md for GitHub raw URL patterns. See references/troubleshooting.md for detailed fixes.