Menu
从网页提取和处理数据，使用CSS选择器、XPath智能解析，支持限速和错误处理。
Instalar com Codex ou Claude Copie este prompt, cole no Codex, Claude ou outro assistente e deixe que ele revise a página da skill e instale para você.
Baseado na classificação ocupacional SOC
自动分析B站视频内容，下载视频并拆解成帧图片，使用AI分析并生成详细的专题文档或实操教程。
设计RESTful API并生成OpenAPI/Swagger规范文档，遵循行业最佳实践。包括端点命名、请求/响应模式和错误处理模式。
为GitHub Actions、GitLab CI、Azure DevOps和Jenkins生成CI/CD流水线，包含构建、测试、部署阶段、缓存和密钥管理。
全面的代码审查技能，分析代码质量、识别问题、安全漏洞，并提供带严重性评级的改进建议。
设计和优化数据库模式，支持PostgreSQL、MySQL、SQLite和MongoDB。包括ER建模、规范化、索引优化和迁移脚本生成。
创建和管理多容器应用的Docker Compose配置，包含生产级设置、健康检查和网络配置。
| name | web-scraper |
| description | 从网页提取和处理数据，使用CSS选择器、XPath智能解析，支持限速和错误处理。 |
| metadata | {"short-description":"从网页提取数据"} |
Extract and process data from web pages with intelligent parsing capabilities.
/scrape commandYou are a web scraping expert that extracts data efficiently and ethically.
import puppeteer from 'puppeteer';
interface Product {
name: string;
price: number;
rating: number;
url: string;
}
async function scrapeProducts(url: string): Promise<Product[]> {
const browser = await puppeteer.launch({ headless: 'new' });
const page = await browser.newPage();
// Set user agent to avoid detection
await page.setUserAgent('Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36');
await page.goto(url, { waitUntil: 'networkidle2' });
// Wait for products to load
await page.waitForSelector('.product-card');
const products = await page.evaluate(() => {
const items = document.querySelectorAll('.product-card');
return Array.from(items).map(item => ({
name: item.querySelector('.product-name')?.textContent?.trim() ?? '',
price: parseFloat(item.querySelector('.price')?.textContent?.replace('$', '') ?? '0'),
rating: parseFloat(item.querySelector('.rating')?.getAttribute('data-rating') ?? '0'),
url: item.querySelector('a')?.href ?? '',
}));
});
await browser.close();
return products;
}
import axios from 'axios';
import * as cheerio from 'cheerio';
async function parseArticle(url: string) {
const { data } = await axios.get(url, {
headers: { 'User-Agent': 'Mozilla/5.0' }
});
const $ = cheerio.load(data);
return {
title: $('h1.article-title').text().trim(),
author: $('span.author-name').text().trim(),
date: $('time').attr('datetime'),
content: $('article.content p').map((_, el) => $(el).text()).get().join('\n\n'),
tags: $('a.tag').map((_, el) => $(el).text()).get(),
};
}
class RateLimiter {
private queue: (() => Promise<void>)[] = [];
private processing = false;
constructor(private delayMs: number = 1000) {}
async add<T>(fn: () => Promise<T>): Promise<T> {
return new Promise((resolve, reject) => {
this.queue.push(async () => {
try {
resolve(await fn());
} catch (e) {
reject(e);
}
});
this.process();
});
}
private async process() {
if (this.processing) return;
this.processing = true;
while (this.queue.length > 0) {
const fn = this.queue.shift()!;
await fn();
await new Promise(r => setTimeout(r, this.delayMs));
}
this.processing = false;
}
}
// Usage
const limiter = new RateLimiter(2000); // 2 seconds between requests
const results = await Promise.all(
urls.map(url => limiter.add(() => scrapeProducts(url)))
);
web-scraping, data-extraction, parsing, automation, html