Menü
统一采集框架 - 支持 RSS/Web/API，207+ 采集源，AI 评分/分类/摘要
Mit Codex oder Claude installieren Kopieren Sie diesen Prompt, fügen Sie ihn in Codex, Claude oder einen anderen Assistant ein und lassen Sie die Skill-Seite prüfen und installieren.
Basierend auf der SOC-Berufsklassifikation
Inspect and manage guarded Codex App-native or launchd heartbeats for Codex main control threads. Use when the user wants Codex to periodically continue a specific thread only when it is idle, avoid interrupting active work, keep App-visible interactions, inspect thread status, or enable/disable project autopilot heartbeats.
Coordinate Lark/Feishu CLI tasks from Codex. Use when a request involves Feishu/Lark docs, wiki, Drive files, Sheets, Base, Calendar, Minutes, IM, Mail, Tasks, Contacts, Slides, whiteboards, approvals, attendance, or when a Lark URL/token must be read, edited, summarized, routed, or turned into a real work artifact.
Use when the user asks Codex to research, find learning materials, process "素材：" links, "请你读" / "精读" a material, build a material radar, or use SenSight-like broad information retrieval for career learning and Agent infra tracking.
Organize tasting menus, menu photos, and visit notes into the right gourmet note. Use when the user shares a restaurant menu, wants a tasting write-up, or asks to merge a new visit into Notes/Gourmet.md without overwriting earlier visits.
Track and synthesize current AI hotspots into a bilingual HTML daily report. Use when the user says "AI热点", asks for AI日报/AI日报 HTML/AI hot topics, or wants the best recent AI products, papers, viewpoints, and cognition-shifting items from AI/news aggregators.
Extract a Markdown heading tree with line numbers before editing or integrating notes. Use when you need to inspect a long Markdown file quickly and choose the best insertion point instead of editing blind.
| name | common-fetcher |
| version | 1.0.0 |
| description | 统一采集框架 - 支持 RSS/Web/API，207+ 采集源，AI 评分/分类/摘要 |
| metadata | {"openclaw":{"emoji":"🕸️","category":"data","requires":{"bins":["node","npm"],"env":[],"config":[]},"primaryEnv":"","install":[{"id":"npm-install","kind":"node","package":"common-fetcher","bins":["common-fetcher"],"label":"Install common-fetcher (npm)"}]}} |
统一采集框架，为 AI Agent 提供强大的信息采集能力。
# 抓取煤炭行业数据
common-fetcher --industry coal --output daily.md
# 抓取房地产行业数据
common-fetcher --industry realestate --output daily.md
# 抓取 AI 技术数据
common-fetcher --industry ai --output daily.md
# 自定义采集源
common-fetcher --config custom-sources.json --output daily.md
import { CommonFetcher } from 'common-fetcher';
const fetcher = new CommonFetcher({
industry: 'coal',
maxArticles: 50,
timeout: 15000,
});
const result = await fetcher.fetch();
console.log(`成功抓取 ${result.totalArticles} 篇文章`);
在 openclaw.json 中配置：
{
"skills": {
"common-fetcher": {
"enabled": true,
"industry": "coal",
"schedule": "0 8 * * *"
}
}
}
┌─────────────────────────────────────────┐
│ Common-Fetcher │
├─────────────────────────────────────────┤
│ Source Layer (采集源层) │
│ ├─ RSS 源 │
│ ├─ 网页源 │
│ └─ API 源 │
├─────────────────────────────────────────┤
│ Fetcher Layer (抓取层) │
│ ├─ RSS Fetcher (并发 + 超时) │
│ ├─ Web Scraper (cheerio) │
│ └─ Cache Manager │
├─────────────────────────────────────────┤
│ Processor Layer (处理层) │
│ ├─ 去重 (标题/URL 哈希) │
│ ├─ 时间过滤 │
│ ├─ AI 评分/分类 │
│ └─ AI 摘要 │
├─────────────────────────────────────────┤
│ Output Layer (输出层) │
│ ├─ Markdown 报告 │
│ ├─ JSON 数据 │
│ └─ 多渠道推送 │
└─────────────────────────────────────────┘
| 解析器 | 文章数/次 | 耗时 | 成功率 |
|---|---|---|---|
| 观点地产网 | 30 篇 | 605ms | 100% |
| 煤炭资源网 | 30 篇 | 455ms | 100% |
| 房天下 | 17 篇 | 579ms | 100% |
| MIT Tech Review | 9 篇 | 393ms | 100% |
| 总计 | 86 篇/次 | ~2s | 100% |
在 config/ 目录下管理采集源：
coal-sources.json - 煤炭行业采集源realestate-sources.json - 房地产行业采集源ai-sources.json - AI 技术采集源自定义解析器参考 src/parsers/ 目录：
export function parseGuandian(html: string, baseUrl: string): Article[] {
// 解析逻辑
}
欢迎提交 Issue 和 PR！
MIT License
Common-Fetcher - 为 AI Agent 提供强大的信息采集能力 🕸️