Menú
De forma predeterminada está seleccionado el prompt que primero revisa el origen. Puedes cambiar a un comando directo o descargar una copia local.
Lee SKILL.md y los archivos complementarios que muestra SkillsMP antes de decidir si quieres instalarlo.
Instalar con Codex o Claude Copia este prompt, pégalo en Codex, Claude u otro asistente, y deja que revise la página de la skill y la instale por ti.
Un comando directo omite el prompt de revisión. Revisa el origen antes de ejecutarlo.
npx skills add https://github.com/Leoyishou/personal-ai-company --skill webfetch-plusEl comando permanece en una sola línea. Desplázate horizontalmente para revisarlo antes de copiarlo.
¿Prefieres una copia local? Descarga los archivos que SkillsMP tiene disponibles ahora.
Basado en la clasificación ocupacional SOC
Mostrando SKILL.md
| name | webfetch-plus |
| shortname | 🌐 webfetch+ |
| description | 智能网页内容抓取工具，根据URL自动选择最佳抓取策略 |
| version | 1.0.0 |
| author | Claude |
| tags | ["fetch","scraper","twitter","social-media","firecrawl","api"] |
WebFetchPlus 是一个强大的智能网页抓取工具，能够根据URL类型自动选择最佳的抓取策略：
# 自动识别并抓取
skill webfetch-plus "https://x.com/elonmusk/status/123456789"
# 抓取抖音视频
skill webfetch-plus "https://v.douyin.com/xxxxx"
# 搜索Twitter
skill webfetch-plus --search "AI news"
# 批量抓取
skill webfetch-plus --batch urls.txt
# 指定策略
skill webfetch-plus "https://example.com" --strategy firecrawl
# JSON输出
skill webfetch-plus "https://x.com/OpenAI" --json
from webfetch_plus import WebFetchPlus
# 初始化
fetcher = WebFetchPlus()
# 抓取推文
result = fetcher.fetch("https://x.com/elonmusk/status/123456789")
print(result["text"])
print(result["author"]["username"])
# 搜索Twitter
results = fetcher.search_twitter("GPT-4", max_results=20)
for tweet in results["results"]:
print(f"@{tweet['author']['username']}: {tweet['text']}")
# 批量抓取
urls = ["url1", "url2", "url3"]
results = fetcher.batch_fetch(urls)
# 获取用户信息
user = fetcher.fetch("https://x.com/OpenAI")
print(f"{user['name']} - {user['metrics']['followers_count']} followers")
{
"id": "123456789",
"text": "推文内容",
"created_at": "2024-01-28T10:00:00Z",
"author": {
"name": "Elon Musk",
"username": "elonmusk",
"verified": true
},
"metrics": {
"like_count": 10000,
"retweet_count": 5000,
"reply_count": 1000
},
"media": [...],
"_metadata": {
"url": "原始URL",
"strategy"
{
"title": "视频标题",
"author": "作者名",
"video_url": "视频地址",
"cover_url": "封面地址",
"stats": {
"likes": 10000,
"comments": 500,
"shares": 200
},
"_metadata": {...}
}
配置文件位置：~/.claude/skills/webfetch-plus/config.json
{
"twitter_api_key": "your_key",
"firecrawl_api_key": "your_key",
"proxy": null,
"timeout": 30,
"max_retries": 3,
"save_path": "~/webfetch_downloads",
"debug": false,
"strategies": {
"twitter": {
"enabled": true,
"api_base": "https://api.twitterapi.io"
},
"firecrawl": {
"enabled": false,
"api_base": "https://api.firecrawl.dev"
支持通过环境变量配置：
export TWITTER_API_KEY="your_twitter_api_key"
export FIRECRAWL_API_KEY="your_firecrawl_api_key"
export WEBFETCH_PROXY="http://proxy:port"
# 添加自定义抓取策略
from webfetch_plus import WebFetchPlus, FetchStrategy
fetcher = WebFetchPlus()
fetcher.add_strategy(
FetchStrategy.CUSTOM,
patterns=[r'custom\.site\.com'],
handler=my_custom_handler
)
# 添加请求中间件
def auth_middleware(request):
request.headers["Authorization"] = "Bearer token"
return request
fetcher.add_middleware(auth_middleware)
# 启用缓存
fetcher.enable_cache(ttl=3600) # 1小时缓存
# 清除缓存
fetcher.clear_cache()
所有错误都会在返回的JSON中包含error字段：
{
"error": "错误描述",
"_metadata": {
"url": "请求的URL",
"strategy": "使用的策略",
"timestamp": "时间戳",
"success": false
}
}
获取抓取统计：
skill webfetch-plus --stats
输出：
{
"total_fetches": 100,
"success": 95,
"failed": 5,
"success_rate": "95.0%"
}