一键导入
data-engineering
Transforms, validates, loads data in ETL pipelines. Use when building scrapers, validating NDJSON feeds, or importing data into CMS/DB targets.
用 Codex 或 Claude 帮你安装 复制这段 Prompt,粘贴到 Codex、Claude 或其他助手里,让它检查 Skill 页面并帮你完成安装。
菜单
Transforms, validates, loads data in ETL pipelines. Use when building scrapers, validating NDJSON feeds, or importing data into CMS/DB targets.
用 Codex 或 Claude 帮你安装 复制这段 Prompt,粘贴到 Codex、Claude 或其他助手里,让它检查 Skill 页面并帮你完成安装。
基于 SOC 职业分类
Scaffolds production-ready monorepo projects using the backbone CLI: configures workspace packages, wires build tooling, sets up CI pipelines, and initializes framework/backend/CMS integrations. Use when creating, bootstrapping, or initializing a new application, project, starter template, or monorepo from scratch. Trigger terms: scaffold, bootstrap, init, new repo, new app, boilerplate, starter, greenfield.
Resolves task dependencies, generates machine-actionable delegation specs, structures phased subtask plans for multi-agent work. Use when writing delegation specs, resolving task dependencies, building phased subtask plans for multi-agent work, assigning work to sub-agents, or partitioning a feature into parallelizable phases.
Reviews mature LESSONS-LEARNED.md entries, rewrites them as permanent rules in skill/instruction files, archives graduated lessons. Use when graduating lessons into skills, promoting validated lessons, updating skills from past learnings, archiving mature lessons, codifying repeated patterns, or cleaning up a crowded LESSONS-LEARNED.md.
Logs sessions, tracks activity, records delegation decisions, and stores review/dispute outcomes as NDJSON audit trails. Use when logging session activity, tracking work, recording decisions, building audit trails, capturing delegation history, or running pre-response verification checklists. Trigger terms: log, track activity, audit trail, session record, delegation log, NDJSON
Enforces React-specific patterns: functional components with hooks, TypeScript prop interfaces, CSS Modules co-location, React Testing Library behavioral tests. Use when creating React components, writing custom hooks, structuring component folders, applying RTL test patterns, or wiring TypeScript prop types. Trigger terms: React, .tsx, component, hook, RTL, jsx, useState, useEffect, prop interface
Implements technical SEO: meta tags, JSON-LD structured data, sitemaps, crawlability fixes. Use when adding schema markup, JSON-LD, robots.txt updates, canonical URLs, Open Graph tags, or improving crawlability.
| name | data-engineering |
| description | Transforms, validates, loads data in ETL pipelines. Use when building scrapers, validating NDJSON feeds, or importing data into CMS/DB targets. |
Generic pipeline patterns. For project-specific sources, full schema references see REFERENCE.md.
Launch a headless browser cluster (Puppeteer Cluster / Playwright) with retryLimit: 3, retryDelay: 5000, timeout: 30000, args: ['--no-sandbox', '--disable-setuid-sandbox'].
One record per line. Schema:
| Field | Type | Notes |
|---|---|---|
name | Required | Preserve original encoding |
lat/lng | Required | GPS coordinates |
address | Required | Full text |
source | Required | e.g. google-maps |
sourceId | Required | Source-unique ID |
category | Required | Domain category |
rating, reviewCount, phone, website, openingHours, photos, priceLevel | Optional | — |
--dry-run to collect sample (50–200 records).
validate-ndjson.js example).
ndjson-filter to isolate failing records; inspect source HTML.createOrReplace disabled; check counts, duplicates.
node ./scripts/scrape-to-ndjson.js --out=data.ndjson --pages=100
node ./scripts/validate-ndjson.js data.ndjson
node ./scripts/dry-import.js data.ndjson --target=staging
node ./scripts/import.js data.ndjson --target=production
const fs = require('fs'), rl = require('readline'), { z } = require('zod');
const schema = z.object({ name: z.string(), source: z.string(), sourceId: z.string() });
const iface = rl.createInterface({ input: fs.createReadStream(process.argv[2]) });
let line = 0, errors = 0;
for await (const l of iface) { line++; try { schema.parse(JSON.parse(l)); } catch(e) { console.error(`Line ${line}:`, e.message); errors++; } }
if (errors) { console.error(`${errors} errors`); process.exit(2); }
console.log('OK');
Full scraper, extended validator: see REFERENCE.md.