searxng-extract

name	searxng-extract
description	Use for URL content extraction when you want a free, unlimited, local solution. Priority: searxng-extract (free, basic) → tavily-extract (advanced features). Use when: user wants page content without API keys, mentions "free", "local", "SearXNG", or searxng-docker-tavily-adapter stack is running. Falls back to tavily-extract if advanced features needed (JS rendering, query-focused chunking).
compatibility	Requires searxng-docker-tavily-adapter Docker stack running on localhost:8000
allowed-tools	Bash(curl *)

searxng extract

Extract clean text from URLs via local SearXNG. No API key required.

Prerequisites

Requires searxng-docker-tavily-adapter running locally.

# Check if running
curl -s http://localhost:8000/health

If not running, ask user for permission to set up:

# Clone and configure (with user permission)
git clone https://github.com/vakovalskii/searxng-docker-tavily-adapter
cd searxng-docker-tavily-adapter
cp config.example.yaml config.yaml
# Generate secret key and update config.yaml
python -c "import secrets; print(secrets.token_urlsafe(32))"
docker compose up -d

Or start existing stack: docker compose up -d

When to use

• Have specific URLs and want their content
• User mentions "free", "no API key", "local"
• Privacy-focused (local deployment)

When NOT to use

• Need JavaScript-rendered pages → use tavily-extract with --extract-depth advanced
• Need query-focused chunking → use tavily-extract with --query
• Docker unavailable → use tavily-extract

Quick start

curl -X POST "http://localhost:8000/search" \
  -H "Content-Type: application/json" \
  -d '{"query": "site:example.com", "max_results": 1, "include_raw_content": true}'

Note: SearXNG extracts via search + scraping. For direct URL, search for the specific URL.

Response

{
  "results": [{
    "url": "https://example.com/page",
    "title": "Page Title",
    "raw_content": "Full page text up to 2500 characters..."
  }]
}

Configuration

Edit config.yaml in your adapter directory:

adapter:
  scraper:
    timeout: 10              # Seconds per page
    max_content_length: 2500 # Max chars per page

Tips

• Content truncated to 2500 chars per page (configurable)
• Timeout is 10 seconds per page (configurable)
• No API key needed — completely free

See also

• searxng-search — search the web
• tavily-extract — original Tavily extract