search

name	search
description	Search the internet for information, discover platforms, and collect structured data. Primary search: Playwright browser → DuckDuckGo/Google HTML scraping. Fallback: httpx zero-auth → DuckDuckGo HTML. Last resort: vscode-websearchforcopilot_webSearch. 3-tier URL fetching for results: fetch_webpage → httpx → Playwright stealth mode. Three modes: standard search (single topic), deep research (multi-dimension exhaustive), and data collection (structured items with direct URLs from specific platforms). Auto-reviews search quality and expands queries if content is insufficient. Always use this skill when the user wants to find information online, research a topic, or collect structured data from platforms — "tìm kiếm về X", "search Google", "tìm hiểu", "danh sách việc làm", "so sánh các nền tảng", or when the pipeline needs web content. Do NOT use for reading local files or fetching explicit user-provided URLs → use gather.
argument-hint	[search query or topic]
version	1.2
compatibility	{"requires":["Python >= 3.10","httpx, beautifulsoup4 (URL fallback + HTTP search)","playwright (primary search + bot-protected URL fallback)"],"tools":["run_in_terminal","fetch_webpage (fetching search result URLs)","open_browser_page, read_page, click_element (VS Code mini browser — Tier 0)","vscode-websearchforcopilot_webSearch (last-resort fallback, often unavailable)"]}

Tìm Kiếm — Internet Search & Discovery Skill

References: references/tool-availability-probe.md | references/playwright-search-fallback.md | references/web-search-enrichment.md | references/deep-research.md | references/playwright-stealth.md | references/data-collection.md | references/dom-exploration.md | references/adaptive-flow.md

Governance: Read and follow .github/RULE.md — it overrides all instructions below.

This skill searches the internet, discovers platforms, and collects structured data. It returns clean Markdown text with source attribution.

Quality loop (RULE-2): After each search execution, self-review runs automatically. If results are insufficient, pivots through strategies (max 3 pivots per RULE-2):

1. Different query formulation (rephrase, add year, switch language)
2. Different source domain (switch to alternative platform)
3. Broaden/narrow scope, switch to deep research mode

Core principle: Never rely solely on training knowledge for time-sensitive or region-specific information. Always search at runtime to discover what currently exists, then combine runtime findings with training knowledge for best coverage.

All responses to the user are in Vietnamese.

---

Step 0: State Read-Back (RULE-13)

Call save_state.py read-context search as FIRST action before any processing. Check relevant_artifacts[] for upstream outputs to incorporate.

---

Step 1: Choose Search Mode

standard:
  when: Single topic, quick question, or synthesize needs web context
  flow: Steps 2 → 5

deep_research:
  when: >
    Multiple dimensions, temporal range, exhaustive coverage, comparison/classification,
    or synthesize passes research_depth: deep
  flow: Deep Research Protocol (below)

data_collection:
  when: >
    Structured list of items with specific fields (jobs, products, companies, reviews).
    synthesize passes mode: data_collection.
    Must return individual item detail URLs — listing/search page URLs are never acceptable.
  flow: Source Intelligence Protocol → Data Collection Protocol (below)

---

Step 2: Construct Queries

1. Analyze the user request to identify key search dimensions
2. Generate 1–3 targeted queries per dimension
3. Use English queries for technical topics (better index coverage)
4. Include year/date qualifiers for time-sensitive topics
5. Report: "🔍 Tìm kiếm: {query_count} hướng tìm kiếm"

---

Step 2.5: Search Engine Selection (Priority Cascade)

The search skill uses a 4-tier cascade — try each tier in order, move to next on failure. Playwright browser and httpx are the primary engines. vscode-websearchforcopilot_webSearch is last resort only (often requires Tavily auth that users don't configure).

SEARCH_CASCADE:
  tier_0_browser:
    tool: open_browser_page + read_page + click_element (VS Code mini browser)
    engine: Google / DuckDuckGo in real browser
    when: Always try first — most reliable, real browser context
    how: |
      1. open_browser_page("https://www.google.com/search?q={url_encoded_query}")
      2. read_page() to extract search results (titles, URLs, snippets)
      3. If Google blocks: try DuckDuckGo ("https://duckduckgo.com/?q={query}")
      4. click_element on result links to navigate and read_page for content
    on_success: Use results, skip lower tiers
    on_failure: Proceed to Tier 1

  tier_1_playwright:
    tool: python3 .github/skills/search/scripts/playwright_search.py "<query>" --limit 8
    engine: DuckDuckGo HTML via headless Playwright
    when: Tier 0 fails or insufficient results
    how: Returns JSON {query, results: [{url, title, snippet}]}
    on_success: Use results
    on_failure: Proceed to Tier 2

  tier_2_http:
    tool: python3 .github/skills/search/scripts/http_search.py "<query>" --limit 8
    engine: DuckDuckGo HTML via httpx (no browser needed)
    when: Tier 1 fails or Playwright unavailable
    how: Returns same JSON shape as Tier 1
    on_success: Use results
    on_failure: Proceed to Tier 3

  tier_3_websearch:
    tool: vscode-websearchforcopilot_webSearch
    engine: Tavily API (requires auth — often unavailable)
    when: ALL above tiers failed
    priority: LOWEST — only as absolute last resort
    how: Standard tool call with query
    on_auth_error: |
      Silently skip. Do NOT show Tavily/auth/config text to user.
      Set session flag primary_unavailable.
      Log to diagnostics. Continue pipeline with empty results.
    on_success: Use results

  all_tiers_exhausted:
    action: |
      Log diagnostic. Skip search step.
      Emit: "Không tìm thấy kết quả tìm kiếm cho yêu cầu này."
      Continue pipeline with empty search results — never block on search failure.

Session caching: Once a tier succeeds, prefer that tier for subsequent queries in the same session. If a tier fails, mark it unavailable for the session — don't retry.

---

Step 3: Execute Search & Fetch

For each query:

1. Run search using the cascade from Step 2.5 (tier 0 → 1 → 2 → 3)
2. Select 2–3 most relevant result URLs (prefer authoritative sources)
3. Fetch each URL using 3-tier content fallback:
   • fetch_webpage (default) → if content ≥ 50 chars: done
   • httpx + BeautifulSoup → if Tier 1 fails
   • Playwright stealth → if bot-detection signals (403, Cloudflare, empty JS content)
   • VS Code browser → open_browser_page + read_page as ultimate fallback
4. After each fetch: read first 200–500 chars. Reject error pages, login walls, empty stubs.
5. Tag content with source URL and dimension

---

Step 4: Quality Review & Auto-Expansion

Check gathered content:

• Volume: ≥5,000 chars total (standard) / ≥15,000 (deep research)
• Specificity: contains numbers, named entities, dates — not just generic descriptions
• Coverage: all requested dimensions have ≥500 chars of relevant content
• Diversity: ≥3 unique source domains

If any check fails: generate 2–3 targeted supplementary queries, fetch, re-check. Max 2 supplementary rounds. If still insufficient, proceed with an honest coverage report.

---

Step 5: Combine & Return

Structure each source:

## Nguồn: {source_name}
> {url} | {char_count} ký tự

{content}

---

Final summary: "📋 Tìm kiếm hoàn tất: {N} nguồn / {total_chars} ký tự / {quality_assessment}"

---

Source Intelligence Protocol

Purpose: Discover which platforms are currently active and accessible for a target domain and region. Run before data collection whenever sources are not universally known.

When to run (all three must be true):

1. Mode is data_collection
2. Item type relies on region-specific or domain-specific platforms (job boards, review sites, local marketplaces, regional directories) — sources that change over time
3. No explicit source list provided by user or orchestrator

When to skip: User provides explicit URLs, or targets universally-known stable platforms.

Phase 1: Discover

Search for currently active platforms for the domain + region + current year. Use English queries. Run ≥2 queries for diversity. Parse into candidate list: {name, url, type}. Exclude news/blog sites. Tag aggregators so collection extracts original-source URLs. Target: ≥3 candidates. Report: "🔍 Tìm thấy {N} nền tảng: {list}"

Phase 2: Test and rank

Test each candidate with 3-tier fetch. Assess: accessible? Data present? Playwright needed?

• Primary: accessible directly + data visible
• Fallback: accessible via Playwright only
• Skip: login wall, paywall, or no relevant data
• Keyword validation: try 2-3 keyword variants per search term — pick the variant with most results
• Detect JS-rendered sites early → escalate to Playwright immediately instead of wasting fetch_webpage

If no viable sources: report failure with concrete next-step options.

Phase 3: Present plan and proceed

Present as information, not a question. Auto-proceed to data collection immediately. In autonomous mode: skip override window. If user sends an override before collection starts: apply it.

---

Data Collection Protocol

Rule: Every collected item MUST have a direct_url to its own detail page. Search/listing page URLs are never acceptable as output URLs.

Per-source collection loop

For each source (primary first, fallback if needed):

1. Search — site:{source} queries to find item URLs. If < 3 items: explore DOM structure (nav links, search forms, URL patterns) to find internal search paths, then retry. See ../gather/references/data-collection.md for details.

2. Extract — Fetch each item detail URL. Extract required fields. Missing → "Không rõ". Validate URLs: must have unique ID/slug. Reject ?q=, ?page=, /search? patterns.

3. Verify quality — ≥3 items, >60% required fields present, all URLs are item pages.

4. Retry on failure (max 2 per source) — different query, different URL pattern, Playwright escalation, detail URL extractor. Different strategy each retry.

5. Mark and move on — if still insufficient, mark source failed with friendly reason. Never stop the pipeline for one source failure.

After all sources

Combine items from succeeded sources. Report: "✅ Thu thập: {N} nguồn / {total} kết quả" If zero sources succeeded: report failure clearly — do NOT fabricate.

Adaptive fallback: When multiple sources fail, call advisory agent for alternatives.

---

Deep Research Protocol

Full protocol: ../gather/references/deep-research.md.

1. Decompose — identify distinct information dimensions → 1–2 queries each
2. Round 1 — broad search across all dimensions, 2–3 sources per dimension
3. Gap analysis — which dimensions are thin? Missing specifics? Temporal gaps?
4. Round 2+ — targeted searches for gaps (max 3 rounds, max 15 URL fetches)
5. Consolidate — structured output with dimension headers + honest coverage assessment

---

Step 6: Artifact Registration (RULE-13)

Call save_state.py register-artifact --step search --path <file> --type search_result --summary "<text>" for every file created in tmp/ or output/.

---

What This Skill Does NOT Do

• Does NOT read local files → gather
• Does NOT fetch user-provided explicit URLs → gather
• Does NOT synthesize or translate content → compose
• Does NOT generate output files → gen-* skills