القائمة
Extracts, cleans, and structures content from web pages by URL or HTML input
التثبيت باستخدام Codex أو Claude انسخ هذا Prompt والصقه في Codex أو Claude أو مساعد آخر ليراجع صفحة Skill ويثبّتها لك.
استنادا إلى تصنيف SOC المهني
Analyzes project structure, module dependencies, imports, and entry points to generate architecture diagrams in Mermaid format
Analyzes ETL and data pipeline code for optimization opportunities across Python (Pandas, PySpark), Rust (polars, datafusion), SQL, and general pipeline descriptions
Validates environment variable configurations and config files (YAML, TOML, JSON, .env) for missing required variables, type mismatches, deprecated keys, naming convention violations, secret exposure risks, and invalid value ranges
Analyzes code for performance bottlenecks including N+1 queries, O(n^2) or worse algorithms, unnecessary allocations, sync I/O in async contexts, excessive cloning, missing caching opportunities, and large payload transfers. Supports Rust, Python, TypeScript, and Go.
Analyzes, improves, and restructures LLM prompts for clarity, efficiency, and reliability
Analyzes source code for common security vulnerabilities including SQL injection, XSS, command injection, hardcoded secrets, insecure deserialization, path traversal, and SSRF
| name | web-scraper |
| description | Extracts, cleans, and structures content from web pages by URL or HTML input |
| version | 1.0.0 |
| author | go-on-team |
| tags | ["web","scraping","html","data-extraction","url"] |
| min_go_on_version | 1.0.0 |
Extracts meaningful content from web pages — article text, tables, metadata, links, and structured data — given a URL or raw HTML. Handles pagination, JavaScript-rendered content hints, and common anti-scraping patterns via fallback extraction strategies.
| Parameter | Type | Description |
|---|---|---|
url | string | Target URL to scrape (mutually exclusive with html) |
html | string | Raw HTML to process (mutually exclusive with url) |
extract | string | Content to extract: article, metadata, links, tables, all (default: article) |
max_length | integer | Optional: maximum characters in output (default: 10000) |
include_raw_html | boolean | Optional: include cleaned HTML alongside text (default: false) |
{
"url": "https://example.com/blog/rust-async-await-guide",
"extract": "article",
"max_length": 5000
}
Example output (abbreviated):
{
"url": "https://example.com/blog/rust-async-await-guide",
"title": "A Practical Guide to Rust Async/Await",
"author": "Jane Developer",
"publish_date": "2026-06-15",
"content": "Async/await in Rust provides zero-cost abstractions for concurrent programming...",
"word_count": 1247,
"links": [
{"text": "Tokio tutorial", "href": "https://tokio.rs/tutorial"},
{"text": "Async book", "href": "https://rust-lang.github.io/async-book/"}
]
}