| name | raincloud-build |
| description | Run the full Raincloud pipeline (fetch → extract → parse → transform → write → validate → convert) for one or more dataset slugs. Use when the user asks to build a dataset, rebuild a slug, or process a batch. |
| argument-hint | <slug>... | --all [--loose] [--clean-workdir] |
| disable-model-invocation | true |
| allowed-tools | Bash(python -m scripts.pipeline.build *) |
Run the Raincloud build orchestrator with the user's args:
python -m scripts.pipeline.build $ARGUMENTS
Selection (at least one required):
<slug>... — positional dataset slugs (any number)--all — every dataset in sources.json
Modifiers:
--loose — downgrade expect.rows mismatches from errors to warnings. Use on the first build of a new slug before you know the exact row count.--clean-workdir — wipe _workdir/<slug>/ after each successful build. Essential for large batch runs (Public BI decompressed CSVs can hit ~100 GB).
Before running:
- Confirm with the user before triggering anything non-trivial. JSONBench 100M ≈ 6 h, Wikipedia Structured Contents → 34 GB parquet, OSM Germany ~45 min per kind. Small (<100 MB) parquets are fine without asking. (See AGENTS.md "Rebuilding is expensive".)
- For large builds, set
RAINCLOUD_DUCKDB_MEMORY_LIMIT and RAINCLOUD_DUCKDB_TEMP_DIRECTORY — see /raincloud-large-build for the full pattern.
- For Kaggle/HF datasets, ensure
uv sync --extra kaggle --inexact (or --extra huggingface --inexact) was run. The --inexact flag is important: without it, syncing one extra removes the others.
After a successful build, suggest running /raincloud-docs to regenerate derived docs.
Context: SKILLS.md, AGENTS.md.
Note (0.1.4)
Per-column profiles are a separate opt-in stage:
python -m scripts.pipeline.profile <slug> after a build. Not part of the
default build pipeline. See raincloud-profile skill.
