| name | searxng-extract |
| description | Use for URL content extraction when you want a free, unlimited, local solution. Priority: searxng-extract (free, basic) → tavily-extract (advanced features). Use when: user wants page content without API keys, mentions "free", "local", "SearXNG", or searxng-docker-tavily-adapter stack is running. Falls back to tavily-extract if advanced features needed (JS rendering, query-focused chunking).
|
| compatibility | Requires searxng-docker-tavily-adapter Docker stack running on localhost:8000 |
| allowed-tools | Bash(curl *) |
searxng extract
Extract clean text from URLs via local SearXNG. No API key required.
Prerequisites
Requires searxng-docker-tavily-adapter running locally.
curl -s http://localhost:8000/health
If not running, ask user for permission to set up:
git clone https://github.com/vakovalskii/searxng-docker-tavily-adapter
cd searxng-docker-tavily-adapter
cp config.example.yaml config.yaml
python -c "import secrets; print(secrets.token_urlsafe(32))"
docker compose up -d
Or start existing stack: docker compose up -d
When to use
- Have specific URLs and want their content
- User mentions "free", "no API key", "local"
- Privacy-focused (local deployment)
When NOT to use
- Need JavaScript-rendered pages → use tavily-extract with
--extract-depth advanced
- Need query-focused chunking → use tavily-extract with
--query
- Docker unavailable → use tavily-extract
Quick start
curl -X POST "http://localhost:8000/search" \
-H "Content-Type: application/json" \
-d '{"query": "site:example.com", "max_results": 1, "include_raw_content": true}'
Note: SearXNG extracts via search + scraping. For direct URL, search for the specific URL.
Response
{
"results": [{
"url": "https://example.com/page",
"title": "Page Title",
"raw_content": "Full page text up to 2500 characters..."
}]
}
Configuration
Edit config.yaml in your adapter directory:
adapter:
scraper:
timeout: 10
max_content_length: 2500
Tips
- Content truncated to 2500 chars per page (configurable)
- Timeout is 10 seconds per page (configurable)
- No API key needed — completely free
See also
