cmm-web-search

name	cmm-web-search
description	Use this skill as the general web search and page-fetch entry for CMM workflows whenever the user asks to search keywords, find related pages or articles, verify facts, collect background sources, compare competitors, find cases, fetch source pages, or build normalized web source cards. This skill includes a unified CLI, runtime-safe runner, built-in AnySearch API backend, Tavily API backend, multi-engine URL generation, public-page fetching, GitHub README fetching, and normalized source-card output with quality scoring. Do not use for subscription feeds, RSS/media-source collection, daily news push channels, AI HOT queries, publishing, or drafting final content.
version	0.7.0

CMM Web Search

cmm-web-search is a general active-search and page-fetch skill. It is not limited to content creation. Any CMM workflow that needs active web search or source-page inspection can use it as the search entrance.

Its job is to understand the user's search intent, choose an available search backend, filter results, fetch public source pages when needed, and return normalized source cards or readable page content that downstream workflows can reuse.

Files

cmm-web-search/
├── SKILL.md
├── README.md
├── config/
│   └── engines.json
├── references/
│   ├── advanced-search.md
│   └── international-search.md
└── scripts/
    ├── cmm_search.sh
    ├── search.py
    └── tavily_search.py

scripts/search.py is the main entrypoint. It integrates:

• Built-in AnySearch API backend for real search without relying on a separate AnySearch skill directory.
• Tavily search through scripts/tavily_search.py.
• Tavily search for high-quality paid/enhanced search when TAVILY_API_KEY is available.
• Multi-engine search URL generation using config/engines.json.
• Public HTTP(S) page fetching with localhost/private-network safeguards.
• GitHub repository README fetching without a GitHub API key.
• Source-card normalization, snippet cleaning, evidence-level tagging, and source-quality scoring for downstream workflows.
• Structured backend status so downstream skills can distinguish real result cards, inspectable search URLs, degraded empty results, and failed searches.

Read README.md when the user needs practical command examples, configuration checks, or key setup instructions.

Boundary

Use this skill for active web search:

• Keyword search.
• Related page or related article search.
• Fact checking.
• Background source discovery.
• Competitor, product, company, or case search.
• Source discovery for research, writing, product work, market analysis, or other workflows.
• Building normalized source cards from search results.
• Fetching readable text from public source pages after search.
• Fetching GitHub repository README files for tool or project research.

Do not use this skill for subscription or push channels:

• Daily news subscriptions.
• AI HOT queries.
• RSS or fixed media-source collection.
• Daily source pool updates.
• Topic pool generation from subscribed feeds.

Defer subscription and push-channel work to the relevant feed, collector, or topic-selection skills.

Do not use this skill for final production:

• It does not draft final articles.
• It does not generate images or videos.
• It does not publish content.
• It does not manage daily feed ingestion.

Search Intent

Classify the request before choosing a backend.

Intent	When to choose	Expected result
keyword_search	User gives a topic, phrase, entity, or question and wants broad discovery	Diverse relevant sources
related_pages	User has a URL, title, article, or draft and wants related references	Comparable pages and adjacent angles
fact_check	User needs to verify a claim, date, number, quote, release, rule, or current fact	Primary or reputable sources, with uncertainty noted
case_search	User wants examples, competitors, companies, products, creator cases, or implementation references	Case-oriented sources
background_search	User needs context before a downstream task	Sources grouped by sub-question
source_refresh	User has old notes or stale assumptions and wants newer sources	Recent sources with date-sensitive caveats

Backend Selection

Choose one backend path based on availability and the search need.

1. runtime-native (preferred when available): capability detection, not platform detection. If the current agent runtime exposes any native web-search tool and/or page-fetch tool — whatever it is called on that platform (Claude Code / Cowork, Codex, OpenCode, openclaw, or others) — use those tools first. They are free, need no API key, return real result content, and allow direct page inspection before citation. The agent runs the searches itself and writes the results into the standard source-card contract below, setting backend: runtime-native and backend_status.status: ok. Page-inspected results may be tagged evidence_level: primary or reputable_secondary as appropriate.
2. anysearch via scripts/cmm_search.sh: use as the first scriptable real search backend. It calls https://api.anysearch.com/mcp directly from this skill, supports anonymous access with lower limits, and can use ANYSEARCH_API_KEY from the environment or ~/.openclaw/.env when present. Use this backend for script-only, cron, or headless workflows where runtime-native is unavailable.
3. tavily via scripts/cmm_search.sh: use when AnySearch is unavailable, when the user asks for Tavily, or when a paid/enhanced search backend is needed. Requires a valid TAVILY_API_KEY; may cost credits and can fail when the key is invalid, expired, or out of quota.
4. engine-url: stable no-key fallback. Generates inspectable search URLs only; never cite these cards without inspecting the pages.

Script auto modes (only relevant when scripts are used instead of runtime-native):

• auto --mode stable: anysearch -> tavily -> engine-url.
• auto --mode live: anysearch -> tavily -> engine-url.

The previous unstable legacy public-search endpoint was removed after repeated 404 failures.

Backend status rules:

• ok: real normalized result cards are available.
• inspect_required: URL-only fallback such as engine-url; useful for manual inspection, not citation.
• degraded: backend was reachable but returned no parseable or insufficient result cards.
• failed: backend failed before usable output.

Downstream CMM stages must not cite inspect_required, degraded, or failed cards as evidence. They may only use them as leads, retry hints, or manual inspection tasks.

Active web search and subscription/source-pool collection are separate concerns. This skill handles active, user-initiated search only; daily source-pool collection is a separate, optional channel.

Commands

Determine {baseDir} as this skill directory.

Auto backend:

python3 {baseDir}/scripts/search.py --query "AI agent workflow" --backend auto --max-results 5 --format source-card

AnySearch backend:

python3 {baseDir}/scripts/search.py --query "AI agent workflow" --backend anysearch --max-results 5 --format source-card

Preferred runner:

{baseDir}/scripts/cmm_search.sh --query "AI agent workflow" --backend auto --mode stable --max-results 5 --format source-card

Live result mode for content research:

{baseDir}/scripts/cmm_search.sh --query "AI agent workflow" --backend auto --mode live --max-results 5 --format source-card

Tavily backend:

python3 {baseDir}/scripts/search.py --query "AI agent workflow" --backend tavily --include-answer --format md

Multi-engine URL generation:

python3 {baseDir}/scripts/search.py --query "AI agent workflow" --backend engine-url --region global --max-results 5
python3 {baseDir}/scripts/search.py --query "AI agent workflow" --backend engine-url --engine "Google"

Fetch a public page:

python3 {baseDir}/scripts/search.py --fetch-url "https://example.com/article" --max-chars 30000 --format md

Fetch a GitHub repository README:

python3 {baseDir}/scripts/search.py --fetch-github-readme "https://github.com/Aas-ee/open-webSearch" --format md

List configured engines:

python3 {baseDir}/scripts/search.py --list-engines

Check local configuration without network calls:

python3 {baseDir}/scripts/search.py --check-config

If --check-config reports idna_encoding_available: false, HTTPS search backends will fail in that Python runtime. On macOS, retry with {baseDir}/scripts/cmm_search.sh ... or /usr/bin/python3 {baseDir}/scripts/search.py ... before judging AnySearch or Tavily as unavailable.

Requirements

The Tavily backend requires TAVILY_API_KEY in either:

• environment variable TAVILY_API_KEY
• ~/.openclaw/.env

engine-url does not require API keys.

The AnySearch backend does not require an API key. It uses anonymous access with lower rate limits by default. For higher limits, set ANYSEARCH_API_KEY in either:

• environment variable ANYSEARCH_API_KEY
• ~/.openclaw/.env

runtime-native requires no key or script; it is available whenever the current agent runtime provides any native web search/fetch tools, regardless of platform. When those tools are absent (plain scripts, cron, headless runtimes), fall back to the script backends above.

Workflow

1. Restate the search target in one sentence.
2. Classify the search intent.
3. Identify constraints:
   • language
   • freshness window
   • geography or market
   • source type preference
   • sources to avoid
   • downstream workflow, if provided
4. Choose a backend: runtime-native search first, scripts as fallback.
5. Run one focused search first. With runtime-native, inspect the most important result pages with the fetch tool before tagging evidence levels. With script-only workflows, use --fetch-url or --fetch-github-readme to inspect important source pages after search.
6. For broad topics, run follow-up searches by sub-question instead of spraying unrelated queries.
7. Deduplicate results by URL, source, and substantially similar title.
8. Classify useful results by source type and likely use.
9. Produce normalized source cards.
10. Summarize what was found, what is reliable, and what still needs inspection.

Query Planning

Build queries from intent rather than blindly searching the literal words.

For broad topic discovery, use:

• core topic query
• case/example query
• opposing-view or risk query
• recent update query when freshness matters

For fact checking, prefer:

• official documentation
• primary company pages
• official announcements
• laws, standards, or regulator pages when relevant
• reputable reporting when primary sources are unavailable

For related page search, use:

• the page title or thesis
• unique named entities
• core concept plus analysis, case study, explainer, review, or Chinese equivalents

For competitor or case search, use:

• entity plus competitor, alternative, case study, pricing, launch, or Chinese equivalents
• category plus named market or platform
• problem statement plus solution category

Source Card Contract

Return source cards in this structure when the user needs reusable search output.

query: ""
search_intent: keyword_search
backend: ""
generated_at: "YYYY-MM-DD"
status: ok
backend_status:
  status: ok
  result_count: 0
  degraded_reason: ""
  citation_ready_count: 0
  weak_lead_count: 0
results:
  - title: ""
    url: ""
    source_name: ""
    source_type: article
    language: ""
    published_or_updated_at: ""
    snippet: ""
    key_takeaway: ""
    relevance_score: 0
    source_quality_score: 0
    evidence_level: ""
    credibility_note: ""
    suggested_use: background
    caveats: ""

Use evidence_level conservatively:

• primary: official, documentation, or paper-like source.
• reputable_secondary: reputable media or secondary reporting.
• vendor_claim: vendor or product-company claim.
• social_signal: social discussion or community lead.
• unverified: inspect before use.

Allowed source_type values:

• official
• documentation
• news
• article
• paper
• social
• video
• company
• case
• database
• unknown

Allowed suggested_use values:

• background
• evidence
• quote
• case
• counterpoint
• data
• comparison
• lead
• further_reading

Output Format

For ordinary user-facing results, use this structure:

• ## 搜索结论: 2-4 sentences on what the search found and which sources look most useful.
• ## 可用来源: numbered links with use, key point, and credibility note.
• ## 可继续追的问题: follow-up questions or gaps.
• ## 来源卡: YAML or JSON source-card block when the user needs reusable structured output.

If the user only asks for a quick lookup, return a concise answer with links instead of the full source-card block.

Quality Rules

• Prefer fewer, stronger sources over long undifferentiated link lists.
• Separate "found in search results" from "verified fact".
• For current, legal, medical, financial, technical, or policy-sensitive claims, use primary or reputable sources and name uncertainty clearly.
• Do not invent publication dates, authors, metrics, quotes, or source types.
• Do not treat search snippets as full evidence when the underlying page needs inspection.
• Keep active search separate from subscription/feed outputs even if both later enter the same workflow.
• engine-url results are search-result URLs, not evidence. Inspect the result pages before citing them.
• Treat vendor_claim, social_signal, and unverified sources as weak evidence unless inspected and corroborated.