메뉴
Push locally-built Raincloud artifacts to a mirror (`scripts.pipeline.publish`). Use when the maintainer wants to upload one or more slugs' parquet/vortex bytes to a configured mirror after a successful build, gated on the snapshot's recorded sha256.
Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.
SOC 직업 분류 기준
Walk through adding a new dataset to sources.json and producing its first build. Use when the user wants to onboard a new upstream source, add a slug to the manifest, or extend the catalog.
Walk through writing a new transform handler under scripts/pipeline/handlers/. Use when the default tighten_types/identity paths can't produce the right shape — row-level JSON parsing, streaming to avoid OOM, multi-output splitting, or VARIANT-from-the-start.
Load a Raincloud dataset via the lightweight Python loader API (`raincloud.load(slug)`). Use when the user wants to read a prepared parquet/vortex artifact, inspect a slug's metadata, or pull a dataset for downstream analysis from cache → mirror → local build.
Use when the user asks to compute or refresh per-column statistics for a raincloud dataset — produces `outputs/v1/<slug>/profile.json` for the TUI's detail pane and `list_datasets --inspect`.
Run the full Raincloud pipeline (fetch → extract → parse → transform → write → validate → convert) for one or more dataset slugs. Use when the user asks to build a dataset, rebuild a slug, or process a batch.
Run only stage 7 — emit a sibling .vortex next to each opted-in parquet. Use when the user asks to (re)convert parquet files to Vortex format without rebuilding from raw bytes.
| name | raincloud-publish |
| description | Push locally-built Raincloud artifacts to a mirror (`scripts.pipeline.publish`). Use when the maintainer wants to upload one or more slugs' parquet/vortex bytes to a configured mirror after a successful build, gated on the snapshot's recorded sha256. |
| argument-hint | <slug>... | --all --mirror <s3://… | https://… | file://…> [--dry-run] |
| disable-model-invocation | true |
| allowed-tools | Bash(python -m scripts.pipeline.publish *) |
Sync locally-built artifacts to a mirror so downstream raincloud.load() users can fetch them. The CLI verifies each artifact's sha256 against docs/v1/snapshot.json before upload — mismatches block the push (publish is the one place an integrity gate is correct: corrupted bytes should never reach a shared mirror).
python -m scripts.pipeline.publish $ARGUMENTS
Selection (one required):
<slug>... — positional dataset slugs (any number).--all — every slug whose parquet OR vortex artifact exists locally under outputs/v{n}/<slug>/.Required:
--mirror <url> — destination root. Examples:
s3://my-bucket/raincloud (needs pip install raincloud[s3])https://artifacts.example.com/raincloud (needs raincloud[http])file:///mnt/shared/raincloud-mirror (built-in)RAINCLOUD_MIRROR=<url> env var works as an alternative to the flag.Modifiers:
--dry-run — print the upload plan (paths + keys) without writing. Always preview large publishes this way first./raincloud-build <slug> or python -m scripts.pipeline.build <slug>). Publish does not build; it only pushes what's already in outputs/v{n}/<slug>/./raincloud-docs or python -m scripts.pipeline.docs). The publish gate compares local bytes against parquet_sha256 / vortex_sha256 in docs/v1/snapshot.json — a stale snapshot causes false PublishMismatch failures.For each slug × format pair (parquet, vortex) where a local artifact exists:
v1/<slug>/<fmt>/<slug>.<ext>sha256(local) == snapshot[<fmt>_sha256] (raises PublishMismatch on disagree; an unpinned slug — sha256 is null — uploads without verification, so prefer to regen the snapshot first).| Error | Meaning | Fix |
|---|---|---|
PublishMismatch: ... | Local bytes diverge from the snapshot's recorded sha. | Re-run /raincloud-docs to refresh the snapshot, OR confirm the local artifact is correct and commit the new snapshot. |
FileNotFoundError: outputs/v1/<slug>/... | Slug isn't built locally. | Run /raincloud-build <slug> first. |
ImportError: Install s3fs ... | s3:// mirror without [s3] extra. | pip install 'raincloud[s3]'. |
Downstream users can now raincloud.load(<slug>) and the loader will pull from the configured mirror (cache → mirror → build).
Context: AGENTS.md "The loader package", scripts/pipeline/publish.py.