| name | fetch |
| description | Public web retrieval and clean extraction engine. Use whenever the user wants to fetch, download, inspect, clean, or save content from a public URL. Supports safe fetching of public web pages, extracts title/text/links, stores results locally, and keeps a job history. No credentials, no login flows, no cloud sync. |
Fetch
Turn public URLs into usable local content.
Core Philosophy
- Fetch only public web content.
- Prefer clean extracted text over noisy raw HTML.
- Save both the raw response and structured extraction locally.
- Keep a simple local job history so previous fetches are easy to inspect.
Runtime Requirements
- Python 3 must be available as
python3
- No external packages required
Safety Boundaries
- Public URLs only
- No login flows
- No cookies or browser automation
- No API keys or credentials
- No external uploads or cloud sync
- All fetched data is stored locally only
Local Storage
All data is stored under:
~/.openclaw/workspace/memory/fetch/jobs.json~/.openclaw/workspace/memory/fetch/pages/
Key Workflows
- Fetch URL:
fetch_url.py --url "https://example.com"
- Save cleaned output:
save_output.py --url "https://example.com" --title "Example"
- List history:
list_jobs.py
- Show job details:
show_job.py --id JOB-XXXX
Scripts
| Script | Purpose |
|---|
init_storage.py | Initialize local storage |
fetch_url.py | Fetch a public URL and extract content |
save_output.py | Save cleaned output with a custom title |
list_jobs.py | List previous fetch jobs |
show_job.py | Show one saved fetch job |
