メニュー
Use when working in the epoch8 datapipe repo with the examples/* folders, when asked to run or debug any datapipe example pipeline on your own data, or when unsure which example or setup skill applies.
Codex または Claude でインストール この Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。
SOC 職業分類に基づく
Use when asked to create, scaffold, or build a NEW datapipe pipeline or project (any size — plain ETL/ingest, ML training with metrics, LLM-processing steps, annotation loops, viewers/UI), usually in an empty directory outside the datapipe repo. Also when a user describes a recurring data-processing task ("files keep arriving and I need to...") and no pipeline exists yet.
Use when working in examples/ocr, or running / debugging the OCR→FiftyOne datapipe example (multi-LLM structured OCR on passport/id-doc images).
Use when setting up or running examples/detection_tags — a self-contained datapipe detection demo built around tags (per-scenario metrics), split into two parts around a checkpoint (train a baseline, then add a tagged TRAIN batch and retrain, watching the tag metric rise), with a FiftyOne view and injected ground truth (no Label Studio). Also for "add a tagged batch, retrain, watch the tag metric rise" and for rehearsing that retraining demo from a saved checkpoint.
Use when setting up or running examples/e2e_template (image_detection or image_keypoints) — datapipe YOLO/Label Studio pipelines — or debugging their install, stages, training, or FiftyOne output.
Use when working in examples/embedder_fiftyone, or running / debugging the embedder→FiftyOne datapipe example (DINOv2/DINOv3 image embeddings, UMAP, similarity search).
Use when working in examples/sam_cvat, or when setting up / running / debugging the datapipe SAM3→CVAT pre-annotation example, on the cats-n-dogs demo data or on your own images.
| name | datapipe-examples |
| description | Use when working in the epoch8 datapipe repo with the examples/* folders, when asked to run or debug any datapipe example pipeline on your own data, or when unsure which example or setup skill applies. |
Every example here is meant to run on YOUR data; each bundled demo dataset is only a smoke-test. Pick the example, then its setup skill, then set that skill's "Run on YOUR data" knobs first.
Each examples/* pipeline has a dedicated setup skill — pick by what it does:
| Example | What it does | Skill |
|---|---|---|
embedder_fiftyone | DINOv2/DINOv3 image embeddings → FiftyOne UMAP + similarity | setup-embedder-fiftyone |
e2e_template/image_detection | YOLO detection + Label Studio human-in-the-loop → train → FiftyOne | setup-e2e-template |
e2e_template/image_keypoints | YOLO-pose keypoints + Label Studio → train → FiftyOne | setup-e2e-template |
sam_cvat | SAM3 text-prompt boxes+masks → CVAT pre-annotations | setup-sam-cvat |
detection_tags | YOLO detection + tags (per-scenario metrics), no Label Studio / FiftyOne, GT injected | setup-detection-tags |
ocr | Multi-LLM structured OCR (OpenAI/Gemini/Qwen) on passport/id-doc images → FiftyOne | setup-ocr |
Clarify what's not obvious before acting — don't spin up services or pull data you don't need. Ask only the unresolved:
docker compose; embedder/sam/ocr need external Postgres)DB_URL? Never point it at an existing DB or drop in a localhost default without confirming.uv env, or create a fresh one? · Which GPU? (SAM3 >8 GB; DINOv2/YOLO ~8 GB; OCR needs none — LLM APIs) · Annotation backend up? (LS for e2e / external CVAT for sam)OCR_ENGINES → openai/gemini/qwen) · Surface stage logs or run quiet? · Per-tag scenario metrics (retrain new case, old vs new)? → the detection_tags example / setup-detection-tagsRead the setup, then propose a short plan and get a go-ahead before touching anything. Prepare .env and pause for the user to verify it before running. Run each stage with its logs shown and, after each, say what you did and what changed — don't run the pipeline silently. Trust the status table (*_training_status/brain_status), not the exit code. If a stage fails and the cause isn't clear from the normal logs, re-run it with datapipe --debug … run (or --debug-sql for SQL errors) sent to a file and grepped, rather than dumping the verbose output inline.
When you swap in your own subject, align your class name across every place it appears — text prompt / label config, the label field, and any class filter. A mismatch runs fine but yields 0 useful results. The per-example skill lists the exact knobs.
DB_URL. embedder + sam_cvat + ocr need an EXTERNAL Postgres (they don't start one);
e2e_template / detection_tags bundle Postgres in docker compose. Empty DB is fine; tables auto-create via
datapipe db create-all. .env.example default ...postgres:postgres@localhost:5432/postgres.
External quick start: docker run -d -e POSTGRES_PASSWORD=postgres -e POSTGRES_USER=postgres -e POSTGRES_DB=postgres -p 5432:5432 postgres:16
(k8s pod / no docker → conda postgres, initdb as non-root with --locale=C.)nvidia-smi): DINOv2/YOLO fit ~8 GB; SAM3 wants >8 GB (OOM'd on an 8 GB Pascal in our tests).
OCR needs no GPU (calls LLM APIs; needs API keys for enabled engines).docker compose; sam_cvat
needs an external CVAT you provide (URL + creds + a project whose labels match its config).uv + Python ≥3.10,<3.13. Each example has a pyproject.toml → uv sync (cu124 torch pinned,
CUDA OOTB; no manual venv/pip). Read-only/small $HOME: export UV_CACHE_DIR=/tmp/uvcache HF_HOME=/tmp/hf.Needed for dinov3_* (embedder, off by default) and sam3 (sam_cvat). Accept the model license on
its HF page, then export HF_TOKEN=... with gated-repo read access. A plain token → 403, often
masked as LocalEntryNotFoundError: check your connection. Public models (dinov2, YOLO) and the OCR
demo HF dataset (ud-synthetic/printed-usa-passports) need no token.
cp .env.example .env # set DB_URL and the example's data + backend vars
uv sync
datapipe db create-all # auto-creates all tables
datapipe run # or: datapipe step --labels stage=<s> run
Datapipe caches by content: re-running does little work unless inputs or tracked config change. Then follow the per-example skill — set its Run on YOUR data knobs first.