Menu
Extracts, cleans, and structures content from web pages by URL or HTML input
Instalar com Codex ou Claude Copie este prompt, cole no Codex, Claude ou outro assistente e deixe que ele revise a página da skill e instale para você.
Baseado na classificação ocupacional SOC
Analyzes project structure, module dependencies, imports, and entry points to generate architecture diagrams in Mermaid format
Analyzes ETL and data pipeline code for optimization opportunities across Python (Pandas, PySpark), Rust (polars, datafusion), SQL, and general pipeline descriptions
Validates environment variable configurations and config files (YAML, TOML, JSON, .env) for missing required variables, type mismatches, deprecated keys, naming convention violations, secret exposure risks, and invalid value ranges
Analyzes code for performance bottlenecks including N+1 queries, O(n^2) or worse algorithms, unnecessary allocations, sync I/O in async contexts, excessive cloning, missing caching opportunities, and large payload transfers. Supports Rust, Python, TypeScript, and Go.
Analyzes, improves, and restructures LLM prompts for clarity, efficiency, and reliability
Analyzes source code for common security vulnerabilities including SQL injection, XSS, command injection, hardcoded secrets, insecure deserialization, path traversal, and SSRF
| name | web-scraper |
| description | Extracts, cleans, and structures content from web pages by URL or HTML input |
| version | 1.0.0 |
| author | go-on-team |
| tags | ["web","scraping","html","data-extraction","url"] |
| min_go_on_version | 1.0.0 |
Extracts meaningful content from web pages — article text, tables, metadata, links, and structured data — given a URL or raw HTML. Handles pagination, JavaScript-rendered content hints, and common anti-scraping patterns via fallback extraction strategies.
| Parameter | Type | Description |
|---|---|---|
url | string | Target URL to scrape (mutually exclusive with html) |
html | string | Raw HTML to process (mutually exclusive with url) |
extract | string | Content to extract: article, metadata, links, tables, all (default: article) |
max_length | integer | Optional: maximum characters in output (default: 10000) |
include_raw_html | boolean | Optional: include cleaned HTML alongside text (default: false) |
{
"url": "https://example.com/blog/rust-async-await-guide",
"extract": "article",
"max_length": 5000
}
Example output (abbreviated):
{
"url": "https://example.com/blog/rust-async-await-guide",
"title": "A Practical Guide to Rust Async/Await",
"author": "Jane Developer",
"publish_date": "2026-06-15",
"content": "Async/await in Rust provides zero-cost abstractions for concurrent programming...",
"word_count": 1247,
"links": [
{"text": "Tokio tutorial", "href": "https://tokio.rs/tutorial"},
{"text": "Async book", "href": "https://rust-lang.github.io/async-book/"}
]
}