name	pipelines
description	Source of truth for choosing and combining computational-execution skills (Snakemake, marimo, RunPod). Load when planning the orchestration shape of an analysis after methodology is decided.

Pipelines

Decision aid for execution shape. Load only after methodology is decided (see skills/INDEX.md and science-plan-analysis). Picking an execution substrate before the analysis question is specified usually produces ceremony without rigor.

For analysis-readiness planning, start at ../INDEX.md or run science-plan-analysis.

When to use which

Skill	Load when	Avoid when
`snakemake.md`	Multi-step pipeline with file dependencies; intermediates worth caching; reproducible re-runs matter	One-off exploration; no DAG of dependencies
`marimo.md`	Interactive exploration; parameter sweeps; presentation with widgets; pre-pipeline prototyping	Production batch; long jobs; CI
`runpod.md`	Short-lived rented GPU; uv-based project; workload too large/slow for workstation	Long-lived managed cluster; CPU-only work

These three are not mutually exclusive: marimo for prototyping -> snakemake for the pipeline -> runpod for the GPU rule. The hub records the decision order; the leaves cover the mechanics.

Cross-cutting principles

Tool-agnostic plans first. science-plan-pipeline produces tool-agnostic task lists. Only commit to a specific orchestration substrate after the task list stabilizes.
Side effects belong outside the workflow tree. Snakemake's protected() does not save you from cleanup-before-rerun (see snakemake.md "protected() does NOT prevent rerun-cleanup"). Apply the marker-file pattern to any rule whose outputs live outside out_dir.
Reproducibility = environment + seeds + inputs. Pin tool versions, lock random seeds, hash inputs (datapackage.json). Without all three the pipeline is decorative.

Companion Skills

../data/SKILL.md — input-data conventions; pipelines should read from data/raw/ and write to data/processed/ or results/.
../research/research-package-spec.md — terminal rule should produce a research package.
../statistics/SKILL.md — statistical decisions that should be made before pipeline construction.

Pipelines

For analysis-readiness planning, start at ../INDEX.md or run science-plan-analysis.

When to use which

Skill

Load when

Avoid when

snakemake.md

Multi-step pipeline with file dependencies; intermediates worth caching; reproducible re-runs matter

One-off exploration; no DAG of dependencies

marimo.md

Interactive exploration; parameter sweeps; presentation with widgets; pre-pipeline prototyping

Production batch; long jobs; CI

runpod.md

Short-lived rented GPU; uv-based project; workload too large/slow for workstation

Long-lived managed cluster; CPU-only work

These three are not mutually exclusive: marimo for prototyping -> snakemake for the pipeline -> runpod for the GPU rule. The hub records the decision order; the leaves cover the mechanics.

Cross-cutting principles

Tool-agnostic plans first. science-plan-pipeline produces tool-agnostic task lists. Only commit to a specific orchestration substrate after the task list stabilizes.

Side effects belong outside the workflow tree. Snakemake's protected() does not save you from cleanup-before-rerun (see snakemake.md "protected() does NOT prevent rerun-cleanup"). Apply the marker-file pattern to any rule whose outputs live outside out_dir.

Reproducibility = environment + seeds + inputs. Pin tool versions, lock random seeds, hash inputs (datapackage.json). Without all three the pipeline is decorative.

Companion Skills

../data/SKILL.md — input-data conventions; pipelines should read from data/raw/ and write to data/processed/ or results/.

../research/research-package-spec.md — terminal rule should produce a research package.

../statistics/SKILL.md — statistical decisions that should be made before pipeline construction.