메뉴
Load a Raincloud dataset via the lightweight Python loader API (`raincloud.load(slug)`). Use when the user wants to read a prepared parquet/vortex artifact, inspect a slug's metadata, or pull a dataset for downstream analysis from cache → mirror → local build.
Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.
SOC 직업 분류 기준
Walk through adding a new dataset to sources.json and producing its first build. Use when the user wants to onboard a new upstream source, add a slug to the manifest, or extend the catalog.
Walk through writing a new transform handler under scripts/pipeline/handlers/. Use when the default tighten_types/identity paths can't produce the right shape — row-level JSON parsing, streaming to avoid OOM, multi-output splitting, or VARIANT-from-the-start.
Push locally-built Raincloud artifacts to a mirror (`scripts.pipeline.publish`). Use when the maintainer wants to upload one or more slugs' parquet/vortex bytes to a configured mirror after a successful build, gated on the snapshot's recorded sha256.
Use when the user asks to compute or refresh per-column statistics for a raincloud dataset — produces `outputs/v1/<slug>/profile.json` for the TUI's detail pane and `list_datasets --inspect`.
Run the full Raincloud pipeline (fetch → extract → parse → transform → write → validate → convert) for one or more dataset slugs. Use when the user asks to build a dataset, rebuild a slug, or process a batch.
Run only stage 7 — emit a sibling .vortex next to each opted-in parquet. Use when the user asks to (re)convert parquet files to Vortex format without rebuilding from raw bytes.
| name | raincloud-load |
| description | Load a Raincloud dataset via the lightweight Python loader API (`raincloud.load(slug)`). Use when the user wants to read a prepared parquet/vortex artifact, inspect a slug's metadata, or pull a dataset for downstream analysis from cache → mirror → local build. |
| argument-hint | <slug> [--format vortex|parquet] [--materialize to_arrow|scan|to_pandas] |
| disable-model-invocation | true |
| allowed-tools | Bash(python -c *), Bash(python -m raincloud *), Bash(python examples/use_loader.py *) |
The Raincloud loader is a separate, lightweight Python package (raincloud) for reading already-prepared artifacts. Resolution order is local cache → mirror → local build; nothing is fetched until you call an accessor.
python -c "
import raincloud
ds = raincloud.load('$ARGUMENTS') # default format='vortex' with parquet fallback
print('rows :', ds.num_rows)
print('cols :', ds.column_names[:5])
print('source :', ds.info.get('source_url'))
print('path :', ds.path()) # triggers cache/mirror/build resolution
"
| Accessor | Returns | Notes |
|---|---|---|
ds.path() | pathlib.Path to the on-disk artifact | First call resolves; subsequent calls are cache hits. |
ds.to_arrow() | pyarrow.Table | Materializes the whole table; expensive on multi-GB slugs. |
ds.to_vortex() | vortex.VortexFile (lazy) | Vortex-native handle. |
ds.scan() | duckdb.DuckDBPyRelation | Requires raincloud[duckdb]. Always reads the parquet sibling — if the slug was loaded as vortex, you'll see a [raincloud] stderr note before the parquet is resolved. |
ds.to_pandas() | pandas.DataFrame | Requires raincloud[pandas]. |
ds.schema | pyarrow.Schema | Footer-only read for parquet; opens the file for vortex. |
| Env var | Effect |
|---|---|
RAINCLOUD_CACHE | Local artifact cache dir (default ~/.cache/raincloud). |
RAINCLOUD_MIRROR | Mirror URL (s3://…, https://…, file://…). Tried before the local build. |
RAINCLOUD_OFFLINE=1 | Block mirror + build; raise OfflineMiss on cache miss. |
RAINCLOUD_SNAPSHOT | Override docs/v1/snapshot.json (catalog). |
RAINCLOUD_MANIFEST | Override sources.json. |
When a slug has a sha256 pinned in the snapshot and the mirror or local build produces bytes that disagree, the loader prints a [raincloud] WARN: <slug> from <mirror|build> sha256 drifted ... to stderr and adopts the new bytes anyway. Upstream content drifts; that's not a panic case. Catch + escalate via raincloud.ChecksumMismatch only if you want a hard gate (e.g. _cache.adopt(..., strict=True) — used by python -m scripts.pipeline.publish).
raincloud.RaincloudError)UnknownSlug, FormatUnavailable, ArtifactNotFound, OfflineMiss, BuildToolingMissing, MissingDependency, ChecksumMismatch.
python examples/use_loader.py --slug $ARGUMENTS # metadata only
python examples/use_loader.py --slug $ARGUMENTS --materialize # full path
examples/use_loader.py walks the full API end-to-end against the packaged catalog with no network.
pip install raincloud # lightweight loader
pip install 'raincloud[duckdb]' # adds .scan()
pip install 'raincloud[pandas]' # adds .to_pandas()
pip install 'raincloud[s3]' # s3:// mirror transport
pip install 'raincloud[http]' # https:// mirror transport
pip install 'raincloud[build]' # adds local-build fallback (heavyweight)
Context: AGENTS.md "The loader package", examples/use_loader.py.