Cedriccmh/claude-code-skill-scrapling

最近记录的来源活动: 2026年6月18日 02:39

SkillsMP 收录数据更新: 2026年11月9日 14:22

已收集 skills: 1

GitHub 星标: 470
GitHub Forks: 56

用 AI 助手安装

复制这段 Prompt，发给你正在使用的 AI 助手。

Follow https://skillsmp.com/skill-install/prompt.md to install Agent Skills from https://github.com/Cedriccmh/claude-code-skill-scrapling.

仓库 README

仓库 README 来源：README.md

查看 README 源文件

展开完整 README

# Claude Code Skill: Scrapling [English](#english) | [中文](#中文) --- ## English A [Claude Code](https://docs.anthropic.com/en/docs/claude-code) skill for web scraping and data extraction using [scrapling](https://github.com/D4Vinci/Scrapling). Automatically selects the best Fetcher based on target website characteristics, uses Scrapling CLI quick paths for simple extraction, and generates Python scripts when needed. ### Features - **Fetcher Decision Tree** — Auto-select between Fetcher, StealthyFetcher, DynamicFetcher, FetcherSession, or Selector - **Cloudflare Bypass** — Built-in support for Cloudflare/WAF protected sites via StealthyFetcher (Camoufox) - **Session Login** — HTTP form-based login with cookie persistence - **Site Pattern Library** — Reusable patterns for common site types (Discourse, SPA, static blogs, APIs) - **Cookie Vault** — Local storage for login cookies with per-site templates - **Troubleshooting Guide** — Solutions for common errors indexed by error message - **Safety Guardrails** — `references/security.md` covers authorization, Prompt injection, `--ai-targeted`, local cookie vaults, and redaction - **Upstream Alignment** — `references/upstream-map.md` keeps this lightweight skill aligned with `D4Vinci/Scrapling` and `scrapling-official` ### Installation #### 1. Install scrapling Pick whichever package manager your project uses: ```bash # pip pip install "scrapling[fetchers]" scrapling install # Install browser dependencies # uv (project) uv add "scrapling[fetchers]" uv run scrapling install # uv (global / standalone, no pyproject.toml) uv pip install "scrapling[fetchers]" scrapling install ``` #### 2. Install this skill Copy the skill directory to your Claude Code skills folder: ```bash # Copy to user-level skills (available in all projects) cp -r . ~/.claude/skills/scrapling # Or copy to a specific project cp -r . /path/to/project/.claude/skills/scrapling ``` ### Structure ``` . ├── SKILL.md # Skill definition (entry point) ├── references/ │ ├── api-quick-ref.md # Fetcher/Selector API cheat sheet │ ├── cookie-vault.md # Cookie storage template │ ├── maintenance.md # Installation & upgrade guide │ ├── security.md # Authorization, prompt-injection, cookie/token guardrails │ ├── site-patterns.md # Site-specific scraping patterns │ ├── troubleshooting.md # Error solutions │ └── upstream-map.md # Official Scrapling docs/skill sync map └── templates/ ├── basic_fetch.py # Static page scraping ├── stealth_cloudflare.py # Cloudflare bypass ├── session_login.py # Login + multi-page scraping └── parse_only.py # HTML parsing without network ``` ### Usage Once installed, Claude Code will automatically activate this skill when you ask it to: - Scrape or crawl a website - Extract data from a URL - Bypass Cloudflare protection - Parse HTML content - Login and scrape protected pages For simple text/Markdown extraction, prefer Scrapling CLI with `--ai-targeted` before writing Python: ```bash scrapling extract get "https://example.com/article" article.md --ai-targeted scrapling extract fetch "https://example.com/app" app.md --ai-targeted --network-idle scrapling extract stealthy-fetch "https://protected.example.com" page.md --ai-targeted --solve-cloudflare ``` #### Examples ``` > Scrape the title and content from https://example.com/blog > Extract all product prices from this page: https://shop.example.com > This site has Cloudflare, scrape it anyway: https://protected.example.com > I have this HTML, extract all links from it ``` ### Cookie Vault The `references/cookie-vault.md` file is a **template**. For actual use: 1. Copy it to `cookie-vault.local.md` (or keep it in your local skill installation) 2. Fill in real cookie values from your browser's DevTools 3. **Never commit real cookie values to version control** ### Local overlays - Put real cookies/tokens only in `references/cookie-vault.local.md`. - Put private company site patterns, internal API details, CSRF notes, and non-public domains only in `references/site-patterns.local.md`. - Both local overlay files are ignored by `.gitignore`. - Keep public `references/site-patterns.md` generic and scrubbed of secrets. --- ## 中文基于 [scrapling](https://github.com/D4Vinci/Scrapling) 的 [Claude Code](https://docs.anthropic.com/en/docs/claude-code) 网页抓取技能。根据目标网站特征自动选择最佳 Fetcher；简单抽取优先用 Scrapling CLI quick path，复杂流程再生成并执行 Python 脚本。 ### 功能特性 - **Fetcher 决策树** — 自动选择 Fetcher、StealthyFetcher、DynamicFetcher、FetcherSession 或 Selector - **Cloudflare 绕过** — 通过 StealthyFetcher (Camoufox) 内置支持 Cloudflare/WAF 防护站点 - **Session 登录** — 基于 HTTP 表单的登录，自动保持 cookie 会话 - **站点模式库** — 常见站点类型的可复用抓取模式（Discourse 论坛、SPA、静态博客、API） - **Cookie 保险库** — 按站点模板存储登录 cookie - **故障排查指南** — 按错误信息索引的常见问题解决方案 - **安全边界** — `references/security.md` 说明授权、Prompt injection、`--ai-targeted`、本地 cookie vault 与 redaction - **上游同步** — `references/upstream-map.md` 说明如何与 `D4Vinci/Scrapling` 和 `scrapling-official` 保持轻量对齐 ### 安装 #### 1. 安装 scrapling 按项目所用的包管理器选一种即可： ```bash # pip pip install "scrapling[fetchers]" scrapling install # 安装浏览器依赖 # uv（项目内） uv add "scrapling[fetchers]" uv run scrapling install # uv（全局 / 独立，无 pyproject.toml） uv pip install "scrapling[fetchers]" scrapling install ``` #### 2. 安装此技能将技能目录复制到 Claude Code 的 skills 文件夹： ```bash # 复制到用户级 skills（所有项目可用） cp -r . ~/.claude/skills/scrapling # 或复制到特定项目 cp -r . /path/to/project/.claude/skills/scrapling ``` ### 目录结构 ``` . ├── SKILL.md # 技能定义（入口文件） ├── references/ │ ├── api-quick-ref.md # Fetcher/Selector API 速查表 │ ├── cookie-vault.md # Cookie 存储模板 │ ├── maintenance.md # 安装与升级指南 │ ├── security.md # 授权、prompt-injection、cookie/token 安全边界 │ ├── site-patterns.md # 站点专用抓取模式 │ ├── troubleshooting.md # 错误解决方案 │ └── upstream-map.md # 官方 Scrapling 文档 / skill 同步地图 └── templates/ ├── basic_fetch.py # 静态页面抓取 ├── stealth_cloudflare.py # Cloudflare 绕过 ├── session_login.py # 登录 + 多页抓取 └── parse_only.py # 纯 HTML 解析（无需网络） ``` ### 使用方式安装后，当你向 Claude Code 提出以下需求时，技能会自动激活： - 抓取或爬取网站 - 从 URL 提取数据 - 绕过 Cloudflare 防护 - 解析 HTML 内容 - 登录后抓取受保护页面简单文本 / Markdown 抽取优先使用 Scrapling CLI，并默认加 `--ai-targeted`： ```bash scrapling extract get "https://example.com/article" article.md --ai-targeted scrapling extract fetch "https://example.com/app" app.md --ai-targeted --network-idle scrapling extract stealthy-fetch "https://protected.example.com" page.md --ai-targeted --solve-cloudflare ``` #### 示例 ``` > 抓取 https://example.com/blog 的标题和正文 > 提取这个页面的所有商品价格：https://shop.example.com > 这个站点有 Cloudflare，帮我绕过抓取：https://protected.example.com > 我有这段 HTML，提取里面所有链接 ``` ### Cookie 保险库 `references/cookie-vault.md` 是一个**模板文件**。实际使用时： 1. 复制为 `cookie-vault.local.md`（或在本地 skill 安装目录中保存） 2. 从浏览器 DevTools 填入真实 cookie 值 3. **切勿将真实 cookie 值提交到版本控制** ### 本地 overlay - 真实 cookie / token 只放 `references/cookie-vault.local.md`。 - 公司内站点模式、内部 API、CSRF 细节、非公开域名只放 `references/site-patterns.local.md`。 - 这两个 local overlay 文件都已被 `.gitignore` 忽略。 - 公共 `references/site-patterns.md` 只保留去敏感化后的通用经验。 ## License MIT

正在加载仓库 README…

这个仓库中的 skills

软件开发工程师

1 个职业分类 · 已分类 100%

skill

职业分类

描述

更新

scrapling

软件开发工程师

使用 scrapling 进行网页抓取和数据提取。根据目标网站特征自动选择最佳 Fetcher，生成并执行 Python 脚本完成任务。Use when: (1) 抓取/爬取网页内容或数据（scrape, crawl, fetch page, extract data） (2) 需要绕过 Cloudflare/WAF 等反爬保护 (3) 登录后抓取受保护页面 (4) 解析已有 HTML 提取结构化数据 (5) 用户提供 URL 并要求获取页面内容或特定元素 (6) 批量采集多个页面

2026-06-18

已展示 1 / 1 个已收集 Skill。