websearch

name	websearch
description	Hard-coded Playwright recipes for web research: engine rules, domain routing, parallel shotgun pattern, 3-call budget, and citation discipline.

Web Search Skill

STOP — READ THIS FIRST

Engine Rules (absolute)

Engine	Status	Why
Brave Search	THE ONLY WORKING ENGINE	Full results, no bot blocking
Google	BLOCKED	Headless detection → CAPTCHA / 0 results
Bing	BLOCKED	Headless detection → 0 results
DuckDuckGo / DDG Lite	BLOCKED	403 Forbidden on headless

Never attempt Google, Google News RSS, Bing, or DuckDuckGo web search. They will always fail or violate Travis-2 policy. Every web-search task MUST use Brave Search (recipe A1). Wikipedia, Reddit, PubMed, arXiv, and other domain APIs are corroboration sources only; they are not replacements for Brave.

Brave-First Rule (absolute)

If the user asks to "search", "web search", "look up", "latest", "current", or asks any factual question that needs the live web:

1. The first external lookup MUST be Brave Search (recipe A1).
2. Do not use Google News RSS at all. Do not use Wikipedia, Reddit, or domain APIs before Brave.
3. If doing multiple sources in parallel, Brave must be one of them and must carry the broad query.
4. If the task only permits one lookup, use Brave Search.
5. If Brave returns usable results, prefer Brave result URLs for article reads.

First Response Rule (absolute)

Your first response after loading this skill MUST be a single message containing all of:

write_todos([...])
+ run_playwright_code(A1: Brave Search)        ← mandatory for every web search
+ optional run_playwright_code(A2/B?: corroboration sources only)

Never issue write_todos alone. Never issue a single search then wait. Call 1 sources are independent → MUST be batched.

---

When This Skill Applies

Any task requiring factual answers from external sources: news, market data, research papers, product info, community discussion, current events. If the answer cannot come from workspace files alone, this skill governs.

---

Hard Budget: 3 Model Calls Maximum

Call 1: write_todos + SHOTGUN BLAST (3-4 parallel sources)
Call 2: write_todos update + READS (article URLs from Call 1, all parallel)
Call 3: write_todos update + SYNTHESIZE (text only, NO tool calls)

Simple lookup path (2 calls):

Call 1: write_todos + SHOTGUN BLAST
Call 2: write_todos update + SYNTHESIZE  (skip article reads if Call 1 returned enough)

Hard Stops

• There is no Call 4. Synthesize with what you have. Use Caveats for gaps.
• Call 3 is text only. No tool calls. Deliver the answer.
• If Call 1 returned 0 usable results from all sources → Call 2 = ONE retry with different terms → Call 3 = synthesize.

Why this works

• Call 1 sources share no dependencies → all parallel → 1 call returns 3-4 sources of data.
• Call 2 reads use URLs known from Call 1 → 1 call returns 2-3 full articles.
• Call 3 is pure synthesis — no new I/O, just reasoning.

---

Step 1: Identify Domain → Pick Sources

Before writing code, classify the topic and pick from this table:

Domain	Indicators	Tier 1 (always)	Tier 2 (pick 1-2)
News / Geopolitics	wars, elections, policy, diplomacy, economy	Brave	Al Jazeera (B1), AP News (B2)
Medical / Health	diseases, drugs, symptoms, clinical trials	Brave	PubMed (B4), Wikipedia (A2)
Science / Space / Physics	research papers, NASA, astronomy	Brave	arXiv (B5), NASA (B6), Wikipedia (A2)
Technology	software, hardware, AI, startups, programming	Brave	Hacker News (B7), StackExchange (B8), Wikipedia (A2)
Finance / Markets	stocks, crypto, earnings, economic data	Brave	CoinGecko (B10), EIA (B12), Wikipedia (A2)
Entertainment	movies, TV, comics, music, games	Brave	Reddit (B9), Wikipedia (A2)
Food / Drink	recipes, wine, restaurants, nutrition	Brave	Open Food Facts (B11), Reddit (B9), Wikipedia (A2)
Sports	scores, players, leagues, events	Brave	Reddit (B9), Wikipedia (A2)
Academic / Research	papers, citations, university topics	Brave	arXiv (B5), PubMed (B4), Wikipedia (A2)
How-to / DIY	tutorials, troubleshooting, guides	Brave	Reddit (B9), StackExchange (B8), Wikipedia (A2)
Weather / Time-sensitive	current weather, forecasts, time-of-day data	Brave	Domain-specific API if known
General / Unknown	anything not above	Brave	Reddit (B9), Wikipedia (A2)

Rule: Always include Brave. Add 1-2 corroboration sources only when they are useful for the domain.

---

Recipe A — UNIVERSAL WEB SEARCH SOURCE (always fire)

A1. Brave Search — General Web Results

20 real web results, all topics, no bot blocking.

import { test } from '@playwright/test';

test('brave search', async ({ page }) => {
  // Replace TOPIC with URL-encoded search terms (spaces as +)
  await page.goto('https://search.brave.com/search?q=TOPIC', {
    waitUntil: 'domcontentloaded', timeout: 20000
  });
  await page.waitForSelector('[data-type="web"]', { timeout: 10000 }).catch(() => {});
  const results = await page.$$eval('[data-type="web"]', els =>
    els.slice(0, 10).map(el => {
      const titleEl = el.querySelector('a h2, a .title, h2 a, .snippet-title');
      const linkEl = el.querySelector('a[href^="http"]');
      const descEl = el.querySelector('.generic-snippet .content, .snippet-description, [class*="generic-snippet"] .content, p.snippet, p');
      return {
        title: titleEl?.textContent?.trim() || '',
        url: linkEl?.getAttribute('href') || '',
        description: descEl?.textContent?.trim()?.slice(0, 200) || ''
      };
    }).filter(r => r.title && r.url)
  );
  console.log(JSON.stringify({ source: 'brave', count: results.length, results }, null, 2));
});

A2. Wikipedia Search API — Encyclopedic / Factual

Up to 5 article matches with snippets. Free, no key, no rate limit.

import { test } from '@playwright/test';

test('wikipedia search', async ({ request }) => {
  const resp = await request.get(
    'https://en.wikipedia.org/w/api.php?action=query&list=search&srsearch=TOPIC&format=json&srlimit=5',
    { timeout: 10000 }
  );
  if (resp.ok()) {
    const data = await resp.json();
    const results = (data.query?.search || []).map((r: any) => ({
      title: r.title,
      snippet: r.snippet?.replace(/<[^>]+>/g, '').trim(),
      url: `https://en.wikipedia.org/wiki/${encodeURIComponent(r.title.replace(/ /g, '_'))}`
    }));
    console.log(JSON.stringify({ source: 'wikipedia', count: results.length, results }, null, 2));
  }
});

---

Recipe B — DOMAIN-SPECIFIC SOURCES (pick 1-2)

B1. Al Jazeera Search — International News (navigable URLs)

import { test } from '@playwright/test';

test('al jazeera search', async ({ page }) => {
  // Replace TOPIC with URL-encoded search terms (spaces as %20)
  await page.goto('https://www.aljazeera.com/search/TOPIC', {
    waitUntil: 'domcontentloaded', timeout: 20000
  });
  await page.waitForSelector('a.u-clickable-card__link', { timeout: 10000 }).catch(() => {});
  const articles = await page.$$eval('a.u-clickable-card__link', els =>
    els.slice(0, 8).map(el => ({
      title: el.textContent?.trim() || '',
      url: el.getAttribute('href') || ''
    })).filter(a => a.title && a.title.length > 5)
  );
  console.log(JSON.stringify({ source: 'aljazeera', count: articles.length, articles }, null, 2));
});

B2. AP News Search — US/International Hard News (navigable URLs)

import { test } from '@playwright/test';

test('ap news search', async ({ page }) => {
  await page.goto('https://apnews.com/search?q=TOPIC', {
    waitUntil: 'domcontentloaded', timeout: 20000
  });
  await page.waitForSelector('.SearchResultsModule-results', { timeout: 10000 }).catch(() => {});
  const articles = await page.$$eval('.SearchResultsModule-results .PagePromo-title a.Link', els =>
    els.slice(0, 8).map(el => {
      const href = el.getAttribute('href') || '';
      return {
        title: el.textContent?.trim() || '',
        url: href.startsWith('http') ? href : 'https://apnews.com' + href
      };
    }).filter(a => a.title && a.title.length > 5)
  );
  console.log(JSON.stringify({ source: 'apnews', count: articles.length, articles }, null, 2));
});

B4. PubMed — Medical / Clinical Research

import { test } from '@playwright/test';

test('pubmed search', async ({ request }) => {
  const searchResp = await request.get(
    'https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=pubmed&term=TOPIC&retmode=json&retmax=5',
    { timeout: 15000 }
  );
  if (!searchResp.ok()) { console.log('PubMed search failed'); return; }
  const searchData = await searchResp.json();
  const ids: string[] = searchData.esearchresult?.idlist || [];
  if (ids.length === 0) { console.log(JSON.stringify({ source: 'pubmed', count: 0, results: [] })); return; }
  const sumResp = await request.get(
    `https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esummary.fcgi?db=pubmed&id=${ids.join(',')}&retmode=json`,
    { timeout: 15000 }
  );
  if (!sumResp.ok()) { console.log('PubMed summary failed'); return; }
  const sumData = await sumResp.json();
  const results = ids.map(id => {
    const r = sumData.result?.[id];
    return {
      pmid: id,
      title: r?.title || '',
      source: r?.source || '',
      pubDate: r?.pubdate || '',
      url: `https://pubmed.ncbi.nlm.nih.gov/${id}/`
    };
  }).filter(r => r.title);
  console.log(JSON.stringify({ source: 'pubmed', count: results.length, results }, null, 2));
});

B5. arXiv — Science / Physics / CS Papers

import { test } from '@playwright/test';

test('arxiv search', async ({ request }) => {
  const resp = await request.get(
    'http://export.arxiv.org/api/query?search_query=all:TOPIC&start=0&max_results=5',
    { timeout: 15000 }
  );
  if (!resp.ok()) { console.log('arXiv failed'); return; }
  const xml = await resp.text();
  const entries: { title: string; summary: string; url: string; published: string }[] = [];
  const re = /<entry>([\s\S]*?)<\/entry>/g;
  let m;
  while ((m = re.exec(xml)) && entries.length < 5) {
    const entry = m[1];
    const title = entry.match(/<title>([\s\S]*?)<\/title>/)?.[1]?.trim() || '';
    const summary = entry.match(/<summary>([\s\S]*?)<\/summary>/)?.[1]?.trim()?.slice(0, 300) || '';
    const id = entry.match(/<id>([\s\S]*?)<\/id>/)?.[1]?.trim() || '';
    const published = entry.match(/<published>([\s\S]*?)<\/published>/)?.[1]?.trim() || '';
    if (title) entries.push({ title, summary, url: id, published });
  }
  console.log(JSON.stringify({ source: 'arxiv', count: entries.length, entries }, null, 2));
});

B6. NASA Images — Space / Astronomy

import { test } from '@playwright/test';

test('nasa images search', async ({ request }) => {
  const resp = await request.get(
    'https://images-api.nasa.gov/search?q=TOPIC&media_type=image',
    { timeout: 15000 }
  );
  if (!resp.ok()) { console.log('NASA API failed'); return; }
  const data = await resp.json();
  const items = (data.collection?.items || []).slice(0, 5).map((item: any) => ({
    title: item.data?.[0]?.title || '',
    description: item.data?.[0]?.description?.slice(0, 200) || '',
    date: item.data?.[0]?.date_created || '',
    center: item.data?.[0]?.center || '',
    url: item.links?.[0]?.href || ''
  })).filter((r: any) => r.title);
  console.log(JSON.stringify({ source: 'nasa', count: items.length, items }, null, 2));
});

B7. Hacker News (Algolia) — Tech / Startups / Programming

import { test } from '@playwright/test';

test('hacker news search', async ({ request }) => {
  const resp = await request.get(
    'https://hn.algolia.com/api/v1/search?query=TOPIC&tags=story&hitsPerPage=8',
    { timeout: 10000 }
  );
  if (!resp.ok()) { console.log('HN failed'); return; }
  const data = await resp.json();
  const results = (data.hits || []).map((h: any) => ({
    title: h.title || '',
    url: h.url || `https://news.ycombinator.com/item?id=${h.objectID}`,
    points: h.points,
    comments: h.num_comments,
    date: h.created_at
  })).filter((r: any) => r.title);
  console.log(JSON.stringify({ source: 'hackernews', count: results.length, results }, null, 2));
});

B8. StackExchange — Technical Q&A

Sites: stackoverflow, askubuntu, superuser, electronics, cooking, scifi, bicycles, aviation.

import { test } from '@playwright/test';

test('stackexchange search', async ({ request }) => {
  const resp = await request.get(
    'https://api.stackexchange.com/2.3/search?order=desc&sort=relevance&intitle=TOPIC&site=SITE&pagesize=5',
    { timeout: 10000 }
  );
  if (!resp.ok()) { console.log('StackExchange failed'); return; }
  const data = await resp.json();
  const results = (data.items || []).map((item: any) => ({
    title: item.title || '',
    url: item.link || '',
    score: item.score,
    answered: item.is_answered,
    answerCount: item.answer_count,
    tags: item.tags?.slice(0, 3)
  })).filter((r: any) => r.title);
  console.log(JSON.stringify({ source: 'stackexchange', count: results.length, results }, null, 2));
});

B9. Reddit Search — Community / Entertainment / DIY

import { test } from '@playwright/test';

test('reddit search', async ({ request }) => {
  const resp = await request.get(
    'https://www.reddit.com/search.json?q=TOPIC&sort=relevance&limit=8&type=link',
    {
      headers: { 'User-Agent': 'Mozilla/5.0 (compatible; research-bot/1.0)' },
      timeout: 15000
    }
  );
  if (!resp.ok()) { console.log('Reddit failed'); return; }
  const data = await resp.json();
  const results = (data.data?.children || []).map((c: any) => ({
    title: c.data?.title || '',
    url: c.data?.url || '',
    subreddit: c.data?.subreddit_name_prefixed || '',
    score: c.data?.score,
    comments: c.data?.num_comments,
    selftext: c.data?.selftext?.slice(0, 200) || ''
  })).filter((r: any) => r.title);
  console.log(JSON.stringify({ source: 'reddit', count: results.length, results }, null, 2));
});

B10. CoinGecko — Crypto / Digital Assets

import { test } from '@playwright/test';

test('coingecko search', async ({ request }) => {
  const resp = await request.get(
    'https://api.coingecko.com/api/v3/search?query=TOPIC',
    { timeout: 10000 }
  );
  if (!resp.ok()) { console.log('CoinGecko failed'); return; }
  const data = await resp.json();
  const coins = (data.coins || []).slice(0, 5).map((c: any) => ({
    name: c.name, symbol: c.symbol, marketCapRank: c.market_cap_rank
  }));
  const exchanges = (data.exchanges || []).slice(0, 3).map((e: any) => ({ name: e.name }));
  console.log(JSON.stringify({ source: 'coingecko', coins, exchanges }, null, 2));
});

B11. Open Food Facts — Food / Nutrition

import { test } from '@playwright/test';

test('open food facts search', async ({ request }) => {
  const resp = await request.get(
    'https://world.openfoodfacts.org/cgi/search.pl?search_terms=TOPIC&json=1&page_size=5&fields=product_name,brands,categories,nutrition_grades,ingredients_text',
    { timeout: 15000 }
  );
  if (!resp.ok()) { console.log('Open Food Facts failed'); return; }
  const data = await resp.json();
  const results = (data.products || []).map((p: any) => ({
    name: p.product_name || '',
    brands: p.brands || '',
    categories: p.categories?.split(',').slice(0, 3).join(', ') || '',
    nutritionGrade: p.nutrition_grades || '',
    ingredients: p.ingredients_text?.slice(0, 150) || ''
  })).filter((r: any) => r.name);
  console.log(JSON.stringify({ source: 'open_food_facts', count: results.length, results }, null, 2));
});

B12. EIA — Energy / Oil Markets

import { test } from '@playwright/test';

test('eia petroleum data', async ({ request }) => {
  const resp = await request.get('https://www.eia.gov/petroleum/supply/weekly/', { timeout: 15000 });
  if (!resp.ok()) { console.log('EIA failed'); return; }
  const html = await resp.text();
  const title = html.match(/<title>([\s\S]*?)<\/title>/)?.[1]?.trim();
  const releaseDate = html.match(/Release Date:\s*([\w\s,]+)/)?.[1]?.trim();
  const tables = html.match(/<table[\s\S]*?<\/table>/g);
  console.log(JSON.stringify({ source: 'eia', title, releaseDate, tablesFound: tables?.length || 0 }, null, 2));
  if (tables?.[0]) console.log('TABLE:\n' + tables[0].replace(/<[^>]+>/g, ' ').replace(/\s+/g, ' ').slice(0, 2000));
});

---

Recipe C — READ FULL ARTICLES (Call 2)

Use URLs from Brave (A1), Al Jazeera (B1), or AP News (B2). NEVER use Google News RSS URLs.

C1. Page Render

import { test } from '@playwright/test';

test('read article', async ({ page }) => {
  await page.goto('ARTICLE_URL', { waitUntil: 'load', timeout: 25000 });
  await page.waitForTimeout(2000);
  const body = await page.$$eval(
    'article p, .article-body p, .story-body p, [data-testid="paragraph"], .wysiwyg p, .article__body p, .post-content p, .entry-content p, [class*="article"] p, main article p, main p',
    els => els.map(el => el.textContent?.trim()).filter(t => t && (t?.length ?? 0) > 20).join('\n\n')
  );
  const title = await page.title();
  const date = await page.$eval(
    'meta[property="article:published_time"], meta[name="date"], time[datetime]',
    el => el.getAttribute('content') || el.getAttribute('datetime') || ''
  ).catch(() => '');
  console.log(`TITLE: ${title}\nDATE: ${date}\nBODY_LENGTH: ${body.length}\n\nBODY:\n${body.slice(0, 5000)}`);
});

C2. JSON-LD Metadata Extract (faster, paywall-resistant)

import { test } from '@playwright/test';

test('extract metadata', async ({ request }) => {
  const resp = await request.get('ARTICLE_URL', {
    headers: { 'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36' },
    timeout: 15000
  });
  const html = await resp.text();
  const jsonLdMatch = html.match(/<script type="application\/ld\+json">([\s\S]*?)<\/script>/);
  if (jsonLdMatch) {
    try {
      const data = JSON.parse(jsonLdMatch[1]);
      console.log(JSON.stringify({
        headline: data.headline, description: data.description,
        datePublished: data.datePublished,
        author: data.author?.name || data.author?.[0]?.name,
        publisher: data.publisher?.name,
        articleBody: data.articleBody?.slice(0, 3000)
      }, null, 2));
    } catch { console.log('JSON-LD parse failed'); }
  }
  const ogTitle = html.match(/property="og:title" content="([^"]+)"/)?.[1];
  const ogDesc = html.match(/property="og:description" content="([^"]+)"/)?.[1];
  if (ogTitle) console.log(`OG Title: ${ogTitle}`);
  if (ogDesc) console.log(`OG Desc: ${ogDesc}`);
});

Read Rules

1. URLs from Brave / Al Jazeera / AP News only.
2. Max 3 articles per call. Batch all reads in one response.
3. C1 for full text; C2 alongside C1 for metadata on JS-heavy / paywalled sites.
4. Empty body (paywalled / blocked) → skip it, do not retry. Move to synthesis.
5. No expect() on count or body length — empty body is a valid outcome (paywall, blocked).

---

Execution Workflow

Call 1 — Shotgun Blast (always parallel)

write_todos([
  { content: "Shotgun: <domain> search for <topic>", status: "in_progress" },
  { content: "Read: top 2-3 results in detail",        status: "pending" },  // omit if simple lookup
  { content: "Synthesize: answer with citations",      status: "pending" }
])
+ run_playwright_code(A1: Brave Search)        ← always
+ optional run_playwright_code(A2/B?: 1-2 corroboration sources)

Decision Gate After Call 1

Inspect each result's data_ok and content size:

Situation	Next call
≥2 sources data_ok=true AND content sufficient to answer	Skip Call 2 reads → go straight to synthesize on Call 2
≥1 source has navigable article URLs AND deeper content needed	Call 2 = parallel article reads
All sources data_ok=false or 0 results	Call 2 = ONE retry with different terms

Call 2 — Reads (always parallel) OR Retry (one source)

write_todos(update: #1 completed, #2 in_progress)
+ run_playwright_code(C1: read article 1)   ← all reads batched
+ run_playwright_code(C1: read article 2)
+ run_playwright_code(C2: metadata for article 3, optional)

Call 3 — Synthesize (NO tool calls)

write_todos(update all completed)
Deliver final answer with citations, caveats, confidence level.

---

Budget Enforcement Table

Call #	Dispatch	Do	Never
1	Parallel	write_todos + Brave + optional corroboration sources (one response)	One source per response; starting without Brave; using Google News RSS
2	Parallel reads OR ONE retry	write_todos + 2-3 reads (one response) OR ONE retry search	Search again if Call 1 had data; read one article per call
3	No tools	write_todos update + text answer	Any tool call

---

Anti-Patterns (immediate course-correction triggers)

• About to scrape Google / Bing / DuckDuckGo → STOP, use Brave instead.
• About to use Google News RSS → STOP, use Brave instead.
• About to use expect() on result count → REMOVE that assertion (0 results is valid).
• About to issue write_todos alone → BATCH with the first action call.
• About to retry a failed source → SWITCH source instead.
• About to read articles one-per-call → BATCH 2-3 in one response.
• About to call tools on Call 3 → STOP, synthesize from what you have.
• More than 3 todos for a search task → TRIM the list.
• Adding https://domain.com prefix to already-absolute URL → DO NOT.

---

Source Credibility (for synthesis)

1. Official / institutional: WHO, NIH, NASA, OPEC, IEA, EIA, government agencies
2. Peer-reviewed: PubMed, arXiv, academic journals
3. Wire services: Reuters, AP, AFP
4. Domain authorities: Bloomberg (finance), ComicVine (comics), IMDB (films)
5. Major news outlets: Al Jazeera, BBC, AP News
6. Community: Reddit, HN — context only, not sole source for facts

---

Citation Format

Dark matter accounts for ~27% of the universe [1], with ongoing detection efforts [2].

---
Sources:
[1] Wikipedia, "Dark matter" - https://en.wikipedia.org/wiki/Dark_matter
[2] arXiv, "Direct dark matter detection..." (2024-11-03) - https://arxiv.org/abs/...

• Every factual claim cited.
• URLs come from tool output — never fabricated.
• Do not cite Google News RSS. Use Brave result URLs or corroborating source URLs.

---

Confidence Levels

Level	When	Phrasing
High	2+ authoritative sources agree, recent	State as fact
Medium	1 authoritative source or minor gaps	"According to [source]..."
Low	Community / blog only, stale, conflicting	"Reports suggest..."

Recency flags:

• Flag data >30 days old.
• Flag >7 days for markets, active conflicts, policy.
• For weather / time-sensitive data: include retrieval timestamp explicitly.

---

Response Template

## Summary
[1-3 sentences with inline citations]

## Key Findings
- Finding 1 [n]
- Finding 2 [n]

## Caveats
- [Recency / data gaps / single-source — one line each]

## Confidence: [High / Medium / Low]

---
Sources:
[1] ...
[2] ...

---

Interaction with Other Skills

• Core: inherits Pre-Flight Gates, Parallel Tool Calling, Retry Rules, Todo Discipline.
• Playwright: recipes use run_playwright_code. Always check data_ok, warnings before claiming success.