메뉴
Extracts, cleans, and structures content from web pages by URL or HTML input
Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.
SOC 직업 분류 기준
Analyzes project structure, module dependencies, imports, and entry points to generate architecture diagrams in Mermaid format
Analyzes ETL and data pipeline code for optimization opportunities across Python (Pandas, PySpark), Rust (polars, datafusion), SQL, and general pipeline descriptions
Validates environment variable configurations and config files (YAML, TOML, JSON, .env) for missing required variables, type mismatches, deprecated keys, naming convention violations, secret exposure risks, and invalid value ranges
Analyzes code for performance bottlenecks including N+1 queries, O(n^2) or worse algorithms, unnecessary allocations, sync I/O in async contexts, excessive cloning, missing caching opportunities, and large payload transfers. Supports Rust, Python, TypeScript, and Go.
Analyzes, improves, and restructures LLM prompts for clarity, efficiency, and reliability
Analyzes source code for common security vulnerabilities including SQL injection, XSS, command injection, hardcoded secrets, insecure deserialization, path traversal, and SSRF
| name | web-scraper |
| description | Extracts, cleans, and structures content from web pages by URL or HTML input |
| version | 1.0.0 |
| author | go-on-team |
| tags | ["web","scraping","html","data-extraction","url"] |
| min_go_on_version | 1.0.0 |
Extracts meaningful content from web pages — article text, tables, metadata, links, and structured data — given a URL or raw HTML. Handles pagination, JavaScript-rendered content hints, and common anti-scraping patterns via fallback extraction strategies.
| Parameter | Type | Description |
|---|---|---|
url | string | Target URL to scrape (mutually exclusive with html) |
html | string | Raw HTML to process (mutually exclusive with url) |
extract | string | Content to extract: article, metadata, links, tables, all (default: article) |
max_length | integer | Optional: maximum characters in output (default: 10000) |
include_raw_html | boolean | Optional: include cleaned HTML alongside text (default: false) |
{
"url": "https://example.com/blog/rust-async-await-guide",
"extract": "article",
"max_length": 5000
}
Example output (abbreviated):
{
"url": "https://example.com/blog/rust-async-await-guide",
"title": "A Practical Guide to Rust Async/Await",
"author": "Jane Developer",
"publish_date": "2026-06-15",
"content": "Async/await in Rust provides zero-cost abstractions for concurrent programming...",
"word_count": 1247,
"links": [
{"text": "Tokio tutorial", "href": "https://tokio.rs/tutorial"},
{"text": "Async book", "href": "https://rust-lang.github.io/async-book/"}
]
}