web-research

name	web-research
description	Research a question on the open web — search, fetch several pages, extract their readable text, and write a cited markdown brief. Use when a task needs current information, facts, or a survey of sources that aren't already in the workspace.
metadata	{"homepage":"https://github.com/bradflaugher/LFG"}

web-research

Turn a question into a short, cited brief backed by real pages. Two helper scripts in scripts/ do the slow parts (searching and de-cluttering HTML) so you can spend tool calls on judgement, not plumbing. Both use only the Python libraries preinstalled in the sandbox (requests, beautifulsoup4, lxml).

Workflow

1. Search for candidate sources:

   python3 /skills/*/web-research/scripts/search.py "your query" --max 8
   

   It prints TITLE\tURL lines from DuckDuckGo's HTML endpoint (no API key).

2. Read the 3–5 most promising results. fetch.py strips nav/scripts and prints the main text, so you get signal instead of markup:

   python3 /skills/*/web-research/scripts/fetch.py "https://example.com/article"
   

   Pass --max-chars 6000 to cap very long pages.

3. Synthesize. Write the answer to /workspace/brief.md: lead with a 2–3 sentence answer, then supporting detail, then a ## Sources list of the URLs you actually used. Cite inline like [1], [2].

Rules

• Prefer primary/official sources over aggregators. Cross-check any surprising claim against a second source before stating it as fact.
• Quote sparingly and attribute. Never invent a URL — only cite pages you fetched.
• If a page is paywalled or empty, say so and move on; don't fabricate content.
• If search returns nothing useful, broaden the query and try again before giving up.