web-search

name	web-search
description	Implement web search capabilities using the mun-os-web-dev-sdk. Use this skill when the user needs to search for real-time information from the web, retrieve up-to-date content beyond the knowledge cutoff, or find the latest news and data. Returns structured search results with URLs, snippets, and metadata.
license	MIT

Web Search Skill

This skill guides the implementation of web search functionality using the mun-os-web-dev-sdk package, enabling applications to search the web and retrieve current information.

Installation Path

Recommended Location: {project_path}/skills/web-search

Extract this skill package to the above path in your project.

Reference Scripts: Example test scripts are available in the {project_path}/skills/web-search/scripts/ directory for quick testing and reference. See {project_path}/skills/web-search/scripts/web_search.ts for a working example.

Overview

The Web Search skill allows you to build applications that can search the internet, retrieve current information, and access real-time data from web sources.

IMPORTANT: mun-os-web-dev-sdk MUST be used in backend code only. Never use it in client-side code.

Prerequisites

The mun-os-web-dev-sdk package is already installed. Import it as shown in the examples below.

CLI Usage (For Simple Tasks)

For simple web search queries, you can use the mun-os CLI instead of writing code. This is ideal for quick information retrieval, testing search functionality, or command-line automation.

Basic Web Search

# Simple search query
mun-os function --name "web_search" --args '{"query": "artificial intelligence"}'

# Using short options
mun-os function -n web_search -a '{"query": "latest tech news"}'

Search with Custom Parameters

# Limit number of results
mun-os function \
  -n web_search \
  -a '{"query": "machine learning", "num": 5}'

# Search with recency filter (results from last N days)
mun-os function \
  -n web_search \
  -a '{"query": "cryptocurrency news", "num": 10, "recency_days": 7}'

Save Search Results

# Save results to JSON file
mun-os function \
  -n web_search \
  -a '{"query": "climate change research", "num": 5}' \
  -o search_results.json

# Recent news with file output
mun-os function \
  -n web_search \
  -a '{"query": "AI breakthroughs", "num": 3, "recency_days": 1}' \
  -o ai_news.json

Advanced Search Examples

# Search for specific topics
mun-os function \
  -n web_search \
  -a '{"query": "quantum computing applications", "num": 8}' \
  -o quantum.json

# Find recent scientific papers
mun-os function \
  -n web_search \
  -a '{"query": "genomics research", "num": 5, "recency_days": 30}' \
  -o genomics.json

# Technology news from last 24 hours
mun-os function \
  -n web_search \
  -a '{"query": "tech industry updates", "recency_days": 1}' \
  -o today_tech.json

CLI Parameters

• --name, -n: Required - Function name (use "web_search")
• --args, -a: Required - JSON arguments object with:
  • query (string, required): Search keywords
  • num (number, optional): Number of results (default: 10)
  • recency_days (number, optional): Filter results from last N days
• --output, -o <path>: Optional - Output file path (JSON format)

Search Result Structure

Each result contains:

• url: Full URL of the result
• name: Title of the page
• snippet: Preview text/description
• host_name: Domain name
• rank: Result ranking
• date: Publication/update date
• favicon: Favicon URL

When to Use CLI vs SDK

Use CLI for:

• Quick information lookups
• Testing search queries
• Simple automation scripts
• One-off research tasks

Use SDK for:

• Dynamic search in applications
• Multi-step search workflows
• Custom result processing and filtering
• Production applications with complex logic

Search Result Type

Each search result is a SearchFunctionResultItem with the following structure:

interface SearchFunctionResultItem {
  url: string;          // Full URL of the result
  name: string;         // Title of the page
  snippet: string;      // Preview text/description
  host_name: string;    // Domain name
  rank: number;         // Result ranking
  date: string;         // Publication/update date
  favicon: string;      // Favicon URL
}

Basic Web Search

Simple Search Query

import SovereignEngine from 'mun-os-web-dev-sdk';

async function searchWeb(query) {
  const sovereign = await SovereignEngine.create();

  const results = await sovereign.functions.invoke('web_search', {
    query: query,
    num: 10
  });

  return results;
}

// Usage
const searchResults = await searchWeb('What is the capital of France?');
console.log('Search Results:', searchResults);

Search with Custom Result Count

import SovereignEngine from 'mun-os-web-dev-sdk';

async function searchWithLimit(query, numberOfResults) {
  const sovereign = await SovereignEngine.create();

  const results = await sovereign.functions.invoke('web_search', {
    query: query,
    num: numberOfResults
  });

  return results;
}

// Usage - Get top 5 results
const topResults = await searchWithLimit('artificial intelligence news', 5);

// Usage - Get top 20 results
const moreResults = await searchWithLimit('JavaScript frameworks', 20);

Formatted Search Results

import SovereignEngine from 'mun-os-web-dev-sdk';

async function getFormattedResults(query) {
  const sovereign = await SovereignEngine.create();

  const results = await sovereign.functions.invoke('web_search', {
    query: query,
    num: 10
  });

  // Format results for display
  const formatted = results.map((item, index) => ({
    position: index + 1,
    title: item.name,
    url: item.url,
    description: item.snippet,
    domain: item.host_name,
    publishDate: item.date
  }));

  return formatted;
}

// Usage
const results = await getFormattedResults('climate change solutions');
results.forEach(result => {
  console.log(`${result.position}. ${result.title}`);
  console.log(`   ${result.url}`);
  console.log(`   ${result.description}`);
  console.log('');
});

Advanced Use Cases

Search with Result Processing

import SovereignEngine from 'mun-os-web-dev-sdk';

class SearchProcessor {
  constructor() {
    this.sovereign = null;
  }

  async initialize() {
    this.sovereign = await SovereignEngine.create();
  }

  async search(query, options = {}) {
    const {
      num = 10,
      filterDomain = null,
      minSnippetLength = 0
    } = options;

    const results = await this.sovereign.functions.invoke('web_search', {
      query: query,
      num: num
    });

    // Filter results
    let filtered = results;

    if (filterDomain) {
      filtered = filtered.filter(item => 
        item.host_name.includes(filterDomain)
      );
    }

    if (minSnippetLength > 0) {
      filtered = filtered.filter(item => 
        item.snippet.length >= minSnippetLength
      );
    }

    return filtered;
  }

  extractDomains(results) {
    return [...new Set(results.map(item => item.host_name))];
  }

  groupByDomain(results) {
    const grouped = {};
    
    results.forEach(item => {
      if (!grouped[item.host_name]) {
        grouped[item.host_name] = [];
      }
      grouped[item.host_name].push(item);
    });

    return grouped;
  }

  sortByDate(results, ascending = false) {
    return results.sort((a, b) => {
      const dateA = new Date(a.date);
      const dateB = new Date(b.date);
      return ascending ? dateA - dateB : dateB - dateA;
    });
  }
}

// Usage
const processor = new SearchProcessor();
await processor.initialize();

const results = await processor.search('machine learning tutorials', {
  num: 15,
  minSnippetLength: 50
});

console.log('Domains found:', processor.extractDomains(results));
console.log('Grouped by domain:', processor.groupByDomain(results));
console.log('Sorted by date:', processor.sortByDate(results));

News Search

import SovereignEngine from 'mun-os-web-dev-sdk';

async function searchNews(topic, timeframe = 'recent') {
  const sovereign = await SovereignEngine.create();

  // Add time-based keywords to query
  const timeKeywords = {
    recent: 'latest news',
    today: 'today news',
    week: 'this week news',
    month: 'this month news'
  };

  const query = `${topic} ${timeKeywords[timeframe] || timeKeywords.recent}`;

  const results = await sovereign.functions.invoke('web_search', {
    query: query,
    num: 10
  });

  // Sort by date (most recent first)
  const sortedResults = results.sort((a, b) => {
    return new Date(b.date) - new Date(a.date);
  });

  return sortedResults;
}

// Usage
const aiNews = await searchNews('artificial intelligence', 'today');
const techNews = await searchNews('technology', 'week');

console.log('Latest AI News:');
aiNews.forEach(item => {
  console.log(`${item.name} (${item.date})`);
  console.log(`${item.snippet}\n`);
});

Research Assistant

import SovereignEngine from 'mun-os-web-dev-sdk';

class ResearchAssistant {
  constructor() {
    this.sovereign = null;
  }

  async initialize() {
    this.sovereign = await SovereignEngine.create();
  }

  async researchTopic(topic, depth = 'standard') {
    const numResults = {
      quick: 5,
      standard: 10,
      deep: 20
    };

    const results = await this.sovereign.functions.invoke('web_search', {
      query: topic,
      num: numResults[depth] || 10
    });

    // Analyze results
    const analysis = {
      topic: topic,
      totalResults: results.length,
      sources: this.extractDomains(results),
      topResults: results.slice(0, 5).map(r => ({
        title: r.name,
        url: r.url,
        summary: r.snippet
      })),
      dateRange: this.getDateRange(results)
    };

    return analysis;
  }

  extractDomains(results) {
    const domains = {};
    results.forEach(item => {
      domains[item.host_name] = (domains[item.host_name] || 0) + 1;
    });
    return domains;
  }

  getDateRange(results) {
    const dates = results
      .map(r => new Date(r.date))
      .filter(d => !isNaN(d));

    if (dates.length === 0) return null;

    return {
      earliest: new Date(Math.min(...dates)),
      latest: new Date(Math.max(...dates))
    };
  }

  async compareTopics(topic1, topic2) {
    const [results1, results2] = await Promise.all([
      this.sovereign.functions.invoke('web_search', { query: topic1, num: 10 }),
      this.sovereign.functions.invoke('web_search', { query: topic2, num: 10 })
    ]);

    const domains1 = new Set(results1.map(r => r.host_name));
    const domains2 = new Set(results2.map(r => r.host_name));

    const commonDomains = [...domains1].filter(d => domains2.has(d));

    return {
      topic1: {
        name: topic1,
        results: results1.length,
        uniqueDomains: domains1.size
      },
      topic2: {
        name: topic2,
        results: results2.length,
        uniqueDomains: domains2.size
      },
      commonDomains: commonDomains
    };
  }
}

// Usage
const assistant = new ResearchAssistant();
await assistant.initialize();

const research = await assistant.researchTopic('quantum computing', 'deep');
console.log('Research Analysis:', research);

const comparison = await assistant.compareTopics(
  'renewable energy',
  'solar power'
);
console.log('Topic Comparison:', comparison);

Search Result Validation

import SovereignEngine from 'mun-os-web-dev-sdk';

async function validateSearchResults(query) {
  const sovereign = await SovereignEngine.create();

  const results = await sovereign.functions.invoke('web_search', {
    query: query,
    num: 10
  });

  // Validate and score results
  const validated = results.map(item => {
    let score = 0;
    let flags = [];

    // Check snippet quality
    if (item.snippet && item.snippet.length > 50) {
      score += 20;
    } else {
      flags.push('short_snippet');
    }

    // Check date availability
    if (item.date && item.date !== 'N/A') {
      score += 20;
    } else {
      flags.push('no_date');
    }

    // Check URL validity
    try {
      new URL(item.url);
      score += 20;
    } catch (e) {
      flags.push('invalid_url');
    }

    // Check domain quality (not perfect, but basic check)
    if (!item.host_name.includes('spam') && 
        !item.host_name.includes('ads')) {
      score += 20;
    } else {
      flags.push('suspicious_domain');
    }

    // Check title quality
    if (item.name && item.name.length > 10) {
      score += 20;
    } else {
      flags.push('short_title');
    }

    return {
      ...item,
      qualityScore: score,
      validationFlags: flags,
      isHighQuality: score >= 80
    };
  });

  // Sort by quality score
  return validated.sort((a, b) => b.qualityScore - a.qualityScore);
}

// Usage
const validated = await validateSearchResults('best programming practices');
console.log('High quality results:', 
  validated.filter(r => r.isHighQuality).length
);

Best Practices

1. Query Optimization

// Bad: Too vague
const bad = await searchWeb('information');

// Good: Specific and targeted
const good = await searchWeb('JavaScript async/await best practices 2024');

// Good: Include context
const goodWithContext = await searchWeb('React hooks tutorial for beginners');

2. Error Handling

import SovereignEngine from 'mun-os-web-dev-sdk';

async function safeSearch(query, retries = 3) {
  let lastError;

  for (let attempt = 1; attempt <= retries; attempt++) {
    try {
      const sovereign = await SovereignEngine.create();

      const results = await sovereign.functions.invoke('web_search', {
        query: query,
        num: 10
      });

      if (!Array.isArray(results) || results.length === 0) {
        throw new Error('No results found or invalid response');
      }

      return {
        success: true,
        results: results,
        attempts: attempt
      };
    } catch (error) {
      lastError = error;
      console.error(`Attempt ${attempt} failed:`, error.message);

      if (attempt < retries) {
        // Wait before retry (exponential backoff)
        await new Promise(resolve => setTimeout(resolve, 1000 * attempt));
      }
    }
  }

  return {
    success: false,
    error: lastError.message,
    attempts: retries
  };
}

3. Result Caching

import SovereignEngine from 'mun-os-web-dev-sdk';

class CachedSearch {
  constructor(cacheDuration = 3600000) { // 1 hour default
    this.cache = new Map();
    this.cacheDuration = cacheDuration;
    this.sovereign = null;
  }

  async initialize() {
    this.sovereign = await SovereignEngine.create();
  }

  getCacheKey(query, num) {
    return `${query}_${num}`;
  }

  async search(query, num = 10) {
    const cacheKey = this.getCacheKey(query, num);
    const cached = this.cache.get(cacheKey);

    // Check if cached and not expired
    if (cached && Date.now() - cached.timestamp < this.cacheDuration) {
      console.log('Returning cached results');
      return {
        ...cached.data,
        cached: true
      };
    }

    // Perform fresh search
    const results = await this.sovereign.functions.invoke('web_search', {
      query: query,
      num: num
    });

    // Cache results
    this.cache.set(cacheKey, {
      data: results,
      timestamp: Date.now()
    });

    return {
      results: results,
      cached: false
    };
  }

  clearCache() {
    this.cache.clear();
  }

  getCacheSize() {
    return this.cache.size;
  }
}

// Usage
const search = new CachedSearch(1800000); // 30 minutes cache
await search.initialize();

const result1 = await search.search('TypeScript tutorial');
console.log('Cached:', result1.cached); // false

const result2 = await search.search('TypeScript tutorial');
console.log('Cached:', result2.cached); // true

4. Rate Limiting

class RateLimitedSearch {
  constructor(requestsPerMinute = 60) {
    this.sovereign = null;
    this.requestsPerMinute = requestsPerMinute;
    this.requests = [];
  }

  async initialize() {
    this.sovereign = await SovereignEngine.create();
  }

  async search(query, num = 10) {
    await this.checkRateLimit();

    const results = await this.sovereign.functions.invoke('web_search', {
      query: query,
      num: num
    });

    this.requests.push(Date.now());
    return results;
  }

  async checkRateLimit() {
    const now = Date.now();
    const oneMinuteAgo = now - 60000;

    // Remove requests older than 1 minute
    this.requests = this.requests.filter(time => time > oneMinuteAgo);

    if (this.requests.length >= this.requestsPerMinute) {
      const oldestRequest = this.requests[0];
      const waitTime = 60000 - (now - oldestRequest);
      
      console.log(`Rate limit reached. Waiting ${waitTime}ms`);
      await new Promise(resolve => setTimeout(resolve, waitTime));
      
      // Recheck after waiting
      return this.checkRateLimit();
    }
  }
}

Common Use Cases

1. Real-time Information Retrieval: Get current news, stock prices, weather
2. Research & Analysis: Gather information on specific topics
3. Content Discovery: Find articles, tutorials, documentation
4. Competitive Analysis: Research competitors and market trends
5. Fact Checking: Verify information against web sources
6. SEO & Content Research: Analyze search results for content strategy
7. News Aggregation: Collect news from various sources
8. Academic Research: Find papers, studies, and academic content

Integration Examples

Express.js Search API

import express from 'express';
import SovereignEngine from 'mun-os-web-dev-sdk';

const app = express();
app.use(express.json());

let sovereignInstance;

async function initSovereignEngine() {
  sovereignInstance = await SovereignEngine.create();
}

app.get('/api/search', async (req, res) => {
  try {
    const { q: query, num = 10 } = req.query;

    if (!query) {
      return res.status(400).json({ error: 'Query parameter "q" is required' });
    }

    const numResults = Math.min(parseInt(num) || 10, 20);

    const results = await sovereignInstance.functions.invoke('web_search', {
      query: query,
      num: numResults
    });

    res.json({
      success: true,
      query: query,
      totalResults: results.length,
      results: results
    });
  } catch (error) {
    res.status(500).json({
      success: false,
      error: error.message
    });
  }
});

app.get('/api/search/news', async (req, res) => {
  try {
    const { topic, timeframe = 'recent' } = req.query;

    if (!topic) {
      return res.status(400).json({ error: 'Topic parameter is required' });
    }

    const timeKeywords = {
      recent: 'latest news',
      today: 'today news',
      week: 'this week news'
    };

    const query = `${topic} ${timeKeywords[timeframe] || timeKeywords.recent}`;

    const results = await sovereignInstance.functions.invoke('web_search', {
      query: query,
      num: 15
    });

    // Sort by date
    const sortedResults = results.sort((a, b) => {
      return new Date(b.date) - new Date(a.date);
    });

    res.json({
      success: true,
      topic: topic,
      timeframe: timeframe,
      results: sortedResults
    });
  } catch (error) {
    res.status(500).json({
      success: false,
      error: error.message
    });
  }
});

initSovereignEngine().then(() => {
  app.listen(3000, () => {
    console.log('Search API running on port 3000');
  });
});

Search with AI Summary

import SovereignEngine from 'mun-os-web-dev-sdk';

async function searchAndSummarize(query) {
  const sovereign = await SovereignEngine.create();

  // Step 1: Search the web
  const searchResults = await sovereign.functions.invoke('web_search', {
    query: query,
    num: 10
  });

  // Step 2: Create summary using chat completions
  const searchContext = searchResults
    .slice(0, 5)
    .map((r, i) => `${i + 1}. ${r.name}\n${r.snippet}`)
    .join('\n\n');

  const completion = await sovereign.chat.completions.create({
    messages: [
      {
        role: 'assistant',
        content: 'You are a research assistant. Summarize search results clearly and concisely.'
      },
      {
        role: 'user',
        content: `Query: "${query}"\n\nSearch Results:\n${searchContext}\n\nProvide a comprehensive summary of these results.`
      }
    ],
    thinking: { type: 'disabled' }
  });

  const summary = completion.choices[0]?.message?.content;

  return {
    query: query,
    summary: summary,
    sources: searchResults.slice(0, 5).map(r => ({
      title: r.name,
      url: r.url
    })),
    totalResults: searchResults.length
  };
}

// Usage
const result = await searchAndSummarize('benefits of renewable energy');
console.log('Summary:', result.summary);
console.log('Sources:', result.sources);

Troubleshooting

Issue: "SDK must be used in backend"

• Solution: Ensure mun-os-web-dev-sdk is only imported and used in server-side code

Issue: Empty or no results returned

• Solution: Try different query terms, check internet connectivity, verify API status

Issue: Unexpected response format

• Solution: Verify the response is an array, check for API changes, add type validation

Issue: Rate limiting errors

• Solution: Implement request throttling, add delays between searches, use caching

Issue: Low quality search results

• Solution: Refine query terms, filter results by domain or date, validate result quality

Performance Tips

1. Reuse SDK Instance: Create SovereignEngine instance once and reuse across searches
2. Implement Caching: Cache search results to reduce API calls
3. Optimize Query Terms: Use specific, targeted queries for better results
4. Limit Result Count: Request only the number of results you need
5. Parallel Searches: Use Promise.all for multiple independent searches
6. Result Filtering: Filter results on client side when possible

Security Considerations

1. Input Validation: Sanitize and validate user search queries
2. Rate Limiting: Implement rate limits to prevent abuse
3. API Key Protection: Never expose SDK credentials in client-side code
4. Result Filtering: Filter potentially harmful or inappropriate content
5. URL Validation: Validate URLs before redirecting users
6. Privacy: Don't log sensitive user search queries

Remember

• Always use mun-os-web-dev-sdk in backend code only
• The SDK is already installed - import as shown in examples
• Search results are returned as an array of SearchFunctionResultItem objects
• Implement proper error handling and retries for production
• Cache results when appropriate to reduce API calls
• Use specific query terms for better search results
• Validate and filter results before displaying to users
• Check scripts/web_search.ts for a quick start example