web-search

# Search — General-Purpose Web & Platform Search ## Setup Read your credentials from ~/.gooseworks/credentials.json: ```bash export GOOSEWORKS_API_KEY=$(python3 -c "import json;print(json.load(open('$HOME/.gooseworks/credentials.json'))['api_key'])") export GOOSEWORKS_API_BASE=$(python3 -c "import json;print(json.load(open('$HOME/.gooseworks/credentials.json')).get('api_base','https://api.gooseworks.ai'))") ``` If ~/.gooseworks/credentials.json does not exist, tell the user to run: `npx gooseworks login` All endpoints use Bearer auth: `-H "Authorization: Bearer $GOOSEWORKS_API_KEY"` Search the web, platforms, and proprietary datasets. Pick the best API for the task — or combine several for comprehensive results. ## 1. Tavily — Comprehensive Web Search & Research Best for: General web search, deep research reports, site mapping, and crawling. **Search the web:** ```bash curl -s -X POST $GOOSEWORKS_API_BASE/v1/proxy/orthogonal/run \ -H "Authorization: Bearer $GOOSEWORKS_API_KEY" \ -H "Content-Type: application/json" \ -d '{"api":"tavily","path":"/search"}' "query": "latest developments in AI agents", "search_depth": "advanced", "include_answer": true, "max_results": 10 }' ``` **Deep research** (async — returns a report with citations): ```bash # Step 1: Start research task curl -s -X POST $GOOSEWORKS_API_BASE/v1/proxy/orthogonal/run \ -H "Authorization: Bearer $GOOSEWORKS_API_KEY" \ -H "Content-Type: application/json" \ -d '{"api":"tavily","path":"/research","body":{"input":"Compare AI agent frameworks for production use","model":"pro"}}' # Step 2: Poll for results using request_id curl -s -X POST $GOOSEWORKS_API_BASE/v1/proxy/orthogonal/run \ -H "Authorization: Bearer $GOOSEWORKS_API_KEY" \ -H "Content-Type: application/json" \ -d '{"api":"tavily","path":"/research/{request_id}"}' ``` **Map a website** (discover all URLs): ```bash curl -s -X POST $GOOSEWORKS_API_BASE/v1/proxy/orthogonal/run \ -H "Authorization: Bearer $GOOSEWORKS_API_KEY" \ -H "Content-Type: application/json" \ -d '{"api":"tavily","path":"/map","body":{"url":"https://docs.example.com","limit":200}}' ``` **Crawl a website** (extract content from multiple pages): ```bash curl -s -X POST $GOOSEWORKS_API_BASE/v1/proxy/orthogonal/run \ -H "Authorization: Bearer $GOOSEWORKS_API_KEY" \ -H "Content-Type: application/json" \ -d '{"api":"tavily","path":"/crawl"}' "url": "https://docs.example.com", "max_depth": 3, "limit": 50 }' ``` Key parameters: `search_depth` (basic/fast/advanced/ultra-fast), `topic` (general/news), `time_range` (day/week/month/year), `include_domains`/`exclude_domains`, `include_answer`, `include_raw_content`, `country`. ## 2. Exa — Neural & Semantic Search Best for: Finding similar content, semantic/embeddings-based search, category-filtered search (people, companies), and deep research. **Neural web search:** ```bash curl -s -X POST $GOOSEWORKS_API_BASE/v1/proxy/orthogonal/run \ -H "Authorization: Bearer $GOOSEWORKS_API_KEY" \ -H "Content-Type: application/json" \ -d '{"api":"exa","path":"/search"}' "query": "startups building AI coding assistants", "numResults": 10, "type": "auto", "contents": {"text": true} }' ``` **Find similar pages:** ```bash curl -s -X POST $GOOSEWORKS_API_BASE/v1/proxy/orthogonal/run \ -H "Authorization: Bearer $GOOSEWORKS_API_KEY" \ -H "Content-Type: application/json" \ -d '{"api":"exa","path":"/findSimilar"}' "url": "https://example.com/article", "numResults": 10, "contents": {"text": true} }' ``` **Get a sourced AI answer:** ```bash curl -s -X POST $GOOSEWORKS_API_BASE/v1/proxy/orthogonal/run \ -H "Authorization: Bearer $GOOSEWORKS_API_KEY" \ -H "Content-Type: application/json" \ -d '{"api":"exa","path":"/answer","body":{"query":"What are the best practices for prompt engineering?"}}' ``` **Get page contents:** ```bash curl -s -X POST $GOOSEWORKS_API_BASE/v1/proxy/orthogonal/run \ -H "Authorization: Bearer $GOOSEWORKS_API_KEY" \ -H "Content-Type: application/json" \ -d '{"api":"exa","path":"/contents","body":{"urls":["https://example.com"],"text":true,"summary":true}}' ``` **Deep research** (async): ```bash # Step 1: Create task curl -s -X POST $GOOSEWORKS_API_BASE/v1/proxy/orthogonal/run \ -H "Authorization: Bearer $GOOSEWORKS_API_KEY" \ -H "Content-Type: application/json" \ -d '{"api":"exa","path":"/research/v1","body":{"instructions":"Research the current state of AI coding assistants","model":"exa-research-pro"}}' # Step 2: Poll with researchId curl -s -X POST $GOOSEWORKS_API_BASE/v1/proxy/orthogonal/run \ -H "Authorization: Bearer $GOOSEWORKS_API_KEY" \ -H "Content-Type: application/json" \ -d '{"api":"exa","path":"/research/v1/{researchId}"}' ``` Key parameters: `type` (auto/neural/fast/deep), `category` (people/company), `includeDomains`/`excludeDomains`, `startPublishedDate`/`endPublishedDate`, `includeText`/`excludeText`. ## 3. Andi — Fast Web Search Best for: Quick, high-quality web search with intelligent ranking and instant answers. Low latency. ```bash curl -s -X POST $GOOSEWORKS_API_BASE/v1/proxy/orthogonal/run \ -H "Authorization: Bearer $GOOSEWORKS_API_KEY" \ -H "Content-Type: application/json" \ -d '{"api":"andi","path":"/v1/search","query":{"q":"how%20does%20RAG%20work","depth":"deep","limit":"20"}}' ``` Key parameters: `q` (query), `limit` (1-100), `depth` (fast/deep), `intent` (NewsSearchIntent, VideoSearchIntent, ImageSearchIntent), `dateRange` (day/week/month/year), `includeDomains`/`excludeDomains`, `country`, `language`. ## 4. Linkup — Question-Based Search with Sourced Answers Best for: Natural language questions, sourced answers with citations, structured output. **Search with sourced answer:** ```bash curl -s -X POST $GOOSEWORKS_API_BASE/v1/proxy/orthogonal/run \ -H "Authorization: Bearer $GOOSEWORKS_API_KEY" \ -H "Content-Type: application/json" \ -d '{"api":"linkup","path":"/search"}' "q": "What are the latest AI agent frameworks?", "depth": "deep", "outputType": "sourcedAnswer" }' ``` **Structured output search:** ```bash curl -s -X POST $GOOSEWORKS_API_BASE/v1/proxy/orthogonal/run \ -H "Authorization: Bearer $GOOSEWORKS_API_KEY" \ -H "Content-Type: application/json" \ -d '{"api":"linkup","path":"/search"}' "q": "Top 5 AI startups in 2025", "depth": "deep", "outputType": "structured", "structuredOutputSchema": { "type": "object", "properties": { "startups": { "type": "array", "items": { "type": "object", "properties": { "name": {"type": "string"}, "description": {"type": "string"}, "funding": {"type": "string"} } } } } } }' ``` Key parameters: `q` (natural language question), `depth` (standard/deep), `outputType` (searchResults/sourcedAnswer/structured), `fromDate`/`toDate`, `includeDomains`/`excludeDomains`, `maxResults`. ## 5. Valyu — Web, Proprietary Datasets & News Search Best for: Searching across web, academic/proprietary datasets, and news. AI-generated answers. Deep research tasks. **Web search:** ```bash curl -s -X POST $GOOSEWORKS_API_BASE/v1/proxy/orthogonal/run \ -H "Authorization: Bearer $GOOSEWORKS_API_KEY" \ -H "Content-Type: application/json" \ -d '{"api":"valyu","path":"/v1/search","body":{"query":"AI agent frameworks comparison","search_type":"web","max_num_results":10}}' ``` **Academic/proprietary search:** ```bash curl -s -X POST $GOOSEWORKS_API_BASE/v1/proxy/orthogonal/run \ -H "Authorization: Bearer $GOOSEWORKS_API_KEY" \ -H "Content-Type: application/json" \ -d '{"api":"valyu","path":"/v1/search","body":{"query":"transformer architecture improvements","search_type":"proprietary"}}' ``` **News search:** ```bash curl -s -X POST $GOOSEWORKS_API_BASE/v1/proxy/orthogonal/run \ -H "Authorization: Bearer $GOOSEWORKS_API_KEY" \ -H "Content-Type: application/json" \ -d '{"api":"valyu","path":"/v1/search","body":{"query":"OpenAI latest announcements","search_type":"news"}}' ``` **AI-generated answer:** ```bash curl -s -X POST $GOOSEWORKS_API_BASE/v1/proxy/orthogonal/run \ -H "Authorization: Bearer $GOOSEWORKS_API_KEY" \ -H "Content-Type: application/json" \ -d '{"api":"valyu","path":"/v1/answer","body":{"query":"What are best practices for building AI agents?"}}' ``` **Deep research** (async): ```bash # Step 1: Create task curl -s -X POST $GOOSEWORKS_API_BASE/v1/proxy/orthogonal/run \ -H "Authorization: Bearer $GOOSEWORKS_API_KEY" \ -H "Content-Type: application/json" \ -d '{"api":"valyu","path":"/v1/deepresearch/tasks","body":{"query":"Comprehensive analysis of vector databases market","mode":"standard"}}' # Step 2: Poll for status