transform-workflow-to-pi

name

transform-workflow-to-pi

description

Take any Claude Code Workflow (a `.claude/workflows/*.js` script that uses agent()/parallel()/ pipeline()/phase()) and run the IDENTICAL pipeline efficiently on a fleet of pi agents (pi.dev / earendil-works/pi) driven by non-Claude coding-plan models — with Claude Code as the single console and monitor. Use when someone wants to run a proven Workflow at lower cost / at scale, "run my workflow on pi", "run this on a non-Claude model", "pi-runner", "offload the workflow to more efficient agents", or to stand up the pi-runner harness in a new repo. Ships copy-paste templates (extract.mjs, run.mjs, provider extension, .env) so any project can adopt it.

Transform a Claude Code Workflow → pi agents

One-line model: the Claude Code Workflow .js is the single source of truth; pi-runner extracts the exact realized prompts + DAG from that same file and replays them, one efficient pi process per node, while Claude Code owns the graph and monitors run-status.json. No port, no codegen, no hand-sync, no drift.

Claude Code (you) ── 1 driver per instance ─► run.mjs (owns the DAG)
                                               │ extract.mjs runs workflow.js under recording stubs
                                               │ → exact prompts + parallel lanes
                                               ▼  one `pi` per node (non-Claude coding-plan model)
                              <repo>/* artifacts + out/<id>/run-status.json  (you poll)

When this applies

• You have a Workflow you've already proven on Claude (it runs via the Workflow tool) and want to run it efficiently / at scale.
• The workflow is pipeline-shaped: a fixed set of waves over one input, coordinating through the filesystem. (Data-driven fan-out needs one extra step — see reference/architecture.md "Dynamic workflows".)
• You want Claude Code to stay the operator: it runs everything, the user runs nothing.

If there is no workflow yet, write and prove one with the Workflow tool first. This skill transforms an existing workflow; it does not author the pipeline logic.

The transform — seven steps

1. Confirm the source of truth. There is exactly one .claude/workflows/<name>.js, it begins with export const meta = {…} (a pure literal), and its body uses only the Workflow hooks (agent/parallel/pipeline/phase/log/args/budget) — no import/export besides meta, and top-level return/await are fine. You edit and prove the workflow on Claude; pi inherits it. Never edit pi's copy of a prompt — there is no copy.

2. Set the credential ONCE in pi's own global config (per machine, not per repo). The model + key live in pi's native ~/.pi/agent/models.json, which pi resolves for EVERY project — so no product ever needs its own key, .env credential, or provider extension.

   cp templates/models.json.example ~/.pi/agent/models.json   # edit: apiKey + baseUrl + model ids
   chmod 600 ~/.pi/agent/models.json
   pi --list-models cp                                          # verify: lists your models
   

   The provider name MUST stay cp (that's what the driver passes as --provider). Any OpenAI-compatible endpoint works (api: "openai-completions"). Skip this if it's already set up on the machine. See reference/provider-and-headless.md.

3. Drop in the harness — verbatim. Copy templates/pi-runner/ into the repo (alongside .claude/): extract.mjs, run.mjs, .env.example, .gitignore (and providers/coding-plan.ts only if a provider needs a custom API impl / OAuth — models.json covers the OpenAI-compatible case). You edit none of the engine files. run.mjs / extract.mjs are generic and stay byte-identical across every repo (and this template) — a future fix is a one-file copy, never a manual merge. The copy also brings + (the cross-project monitor, step 7). — so it appears in the global console before its first run; every run thereafter self-registers too.

The laws (do not violate)

• Single source of truth = the workflow .js. Improve a wave by editing its prompt/skill in the workflow and re-proving on Claude; pi runs the new prompts automatically. Zero hand-sync.
• The engine files never diverge. run.mjs / extract.mjs / watch.mjs / status.mjs stay byte-identical across every repo and this template; 100% of per-repo specifics live in the wiring .env, and the credential lives once in pi's global ~/.pi/agent/models.json. A fix is a one-file copy. If you find yourself editing an engine file for one repo, you're introducing the drift this whole pattern exists to prevent — push it into .env (wiring) or models.json (credential) instead.
• Extraction, not codegen. extract.mjs runs the workflow under recording stubs and captures the exact prompts + grouping. New/removed/reordered waves propagate for free.
• Driver owns the graph; pi owns the node. Plain code decides stage order + parallel lanes + halt-on-failure; the model never decides control flow. Nodes coordinate via the filesystem.
• The workflow orchestrates; the SKILL carries the craft — never duplicate craft into a node body. When a node loads a skill (SKILL TO LOAD AND FOLLOW: …), split content by OWNER. The workflow .js holds ORCHESTRATION ONLY: the node sequence + parallel lanes (the DAG), each node's I/O contract (contract({artifacts,owns,readScope}) + the return schema), and a THIN wiring body — who the node is · which input artifacts it reads · which output it writes · the load-and-follow pointer. The CRAFT — how to do the work: the method, the bar, the build path, the domain detail — lives ONLY in the skill, its single canonical home. A node body that RESTATES the skill's craft is two ground truths: every craft edit then needs two edits and they drift (the dual-maintenance trap). So put all craft in the skill, keep the body a pointer, and improve a wave by editing its SKILL / improve the chain by editing the workflow. Extraction is unchanged — the realized prompt is still the (thin) body, and the model reads the loaded skill at runtime; only WHERE the craft text lives collapses to one home. (Inline-prompt nodes with no backing skill are exempt — there the body IS the only home; this law governs the skill-backed pattern.)
• Verified against the declared contract, not the self-report. Each node ends with one fenced json block; the driver s every . But the self-reported list is honest only when the model is — so a node may declare, in its prompt, the files it is to leave on disk () and the only paths it may write (). The driver verifies the set independent of the self-report: a clean exit that did not produce a required artifact is , not . See below and .

Companion Mode (dev-time) — the orchestrator IS the verification node

A workflow ships with both an automated in-pipeline VERIFICATION surface (the verify nodes) AND the human-judged criteria fixture (step 13). Production runs the verify nodes for stable, unattended output. But during development/debugging — when you're babysitting the run — they're slow, and you (orchestrator + human) judge better. Companion Mode makes the orchestrator the standing verifier:

• One static toggle, two clean DAGs. Branch the workflow on a mode INPUT arg (const COMPANION = (args.mode === 'companion')) and wrap every verify node if (!COMPANION). Because mode is a static input (resolved BEFORE any node runs), extract.mjs/run.mjs realize a FIXED DAG per mode — NOT the result-dependent branching the extractor can't see. production (default) = full pipeline; companion = producing nodes only.
• This only works because verify nodes create nothing (the law above). Drop them and every key artifact still exists, because PRODUCERS made them. If a verify node is also a producer you could not drop it — split it FIRST, then add the toggle.
• Run in the background; judge every stage as it lands. Poll run-status.json; the moment a node goes ok, compare its artifact to (a) the GOLD sample and (b) its criteria-fixture entry — the same dual reference the hermes-skill-system node-validation loop uses (criteria stay a JUDGING reference, NEVER injected). You are the verifier for EVERY surface the skipped nodes would have covered.
• On a heavy mistake, stop — don't pour effort onto a bad upstream artifact. Fix at the canonical owner (Hermes), rerun the SUFFIX fixed by the first changed node (--from/--only), reuse unchanged upstream. Borderline → surface to the human (the eye), don't guess. Promote a fresh artifact over the gold when it's better (this is also how the gold + criteria get sharpened each run). A dev-time POSTURE, not a code path beyond the one mode branch. Pairs with the criteria fixture (step 13) and hermes-skill-system's node-validation loop.

Designing a node's I/O — the standards + the I/O map

Designing a workflow IS setting each node's input/output standards — this is the most meaningful place to fix them, and where an edit must be reconciled against the rest. For every node:

• One node, one task, so its I/O is a clean boundary; split a two-job node. A producer creates each key artifact; a verifier never does (the verify-node law above).
• Format the output for its CONSUMER, not by default. Strict typed JSON ONLY at a machine boundary (a parser / schema / the driver). PROSE/Markdown for an LLM-reasoning hand-off (a middle product the next model THINKS over): reason-in-prose, structure LAST — strict JSON on a reasoning hand-off taxes reasoning ~5–15% and ~35% tokens, worst on non-Claude models, and inverts CoT when a decision field precedes its rationale. Push the schema boundary as LATE as possible; keep a small fenced-JSON tail only for the fields a parser reads. Merge/denormalize for a single downstream reasoner; split only for parallel agents. (Full prompt-craft + citations: agentic-prompt-design §5.)
• Declare two things at node-creation time: the contract({ artifacts, owns, readScope }) (what it WRITES
  • its read surface) AND the artifact's CONSUMERS in the I/O map.
• Split MECHANICAL from INTELLIGENT, then push the mechanical into a DRIVER HOOK — don't leave it to the model. List the node's steps; for each ask: "is this output a fixed function of already-frozen on-disk inputs, with no judgment?" YES → a deterministic driver hook, run in the driver, not the model — so it becomes a TESTED CODE PATH, not a per-run gamble. Two verbs, one family: a PRE-hook (DRIVER-SEED, the proven one) STAGES inputs before the model (copy a skeleton/tree → FILL-don't-COMPOSE); a POST-hook (DRIVER-PROJECT/DRIVER-MERGE, its derive-don't-recompute sibling) DERIVES/validates outputs after the model (project a frozen spec → its runtime data file, merge fragments, schema-check). NO (design reasoning / open-ended coding / prose authoring / diagnose-and-fix) → the model. Declare the deterministic part as DATA in the registry (next to the routing facts), so the engine stays uniform + genre-agnostic and adding a type needs ZERO node-prompt and ZERO engine edit. Aim the model's residual to be WRITE-DOMINANT + health-checkable (the write-first gate + a build/validate signal cover it). Why: it removes the non-Claude-model explore-forever / mis-project thrash surface, makes mechanical output un-hallucinatable, and cuts tokens — a structural invariant belongs in the harness, not in prose the model can ignore. Hook spec + the DRIVER-SEED/contract markers: reference/artifact-contract.md.
• Design for parallelism from the I/O up — the map is where independent lanes become visible. As you set each node's I/O, look for nodes whose inputs are ALL already-frozen upstream artifacts and whose set is DISJOINT from a sibling's — those are independent that need not wait on each other, so run them as a for wall-clock. The : two concurrent lanes must share NO writable file (the collapses to one extractor stage and runs them via — a real race). If a "dependency" is only that lane B reads an artifact lane A produces, check whether B can read the SAME upstream source instead (re-pointing the read dissolves the artificial edge). When two lanes would BOTH touch one shared file, don't serialize the whole lane — split that file: give each lane its own per-node fragment and add a tiny SERIAL JOIN node after the parallel stage to merge them deterministically (e.g. fragments → a merged ; an asset lane's own manifest → reconciled into the build lane's index). The shared-write becomes one serial node, not a serial pipeline. (The script has no fs at eval time, so the join is a NODE, never raw fs in the workflow.)

The node I/O map — <repo>/.agents/skill-system-io-map.md, the THIRD standing artifact beside the skill-system map (composition) and the criteria fixture (quality). It is the producer→consumer ledger keyed by ARTIFACT: for each on-disk artifact, which node PRODUCES it, which nodes CONSUME it, and HOW (strict parse vs LLM read — because that is a format change's blast radius). It is the interface contract of a filesystem-coordinated pipeline, and the lookup that makes the reconcile-consumers law efficient: before changing a node's output, read the artifact's consumer row and reconcile every one; after, verify no consumer reads the stale shape. Derive it once from the node read-lines; update it on the SAME trigger as the map (any change to a node's read/write set). When an agent edits a node or its skill, it CHECKS IN here, fixes the downstream consumers the change affects, and reconciles with the upstream artifacts it reads — so a format change can never leave a dangling consumer (the class of bug where W1 moved to gdd.md and its readers still opened gdd.json).

Files in this skill

• reference/cli.md — the COMPLETE, exact CLI: every run.mjs flag, the --from/--until/--only node-range model + worked examples, the .env knobs, the monitors, and the verify-the-model duty. Read this so node ranges + syntax are never guessed.
• reference/architecture.md — why the workflow runs unchanged: the four invariants, the observability tiers, and the one dynamic-workflow caveat. Read this to understand the pattern.
• reference/artifact-contract.md — the Output Contract: the fourth contract layer Claude Code leaves to the orchestrator (DRIVER-ARTIFACTS/DRIVER-OWNS markers + the contract() helper + driver enforcement). Read this to make a node deliver the right artifact to the right place.
• reference/escalation.md — the escalation gate (advisor inversion): the empirical classifier, the non-blind consult preamble, the .env wiring + cross-family target, DRIVER-NO-ESCALATE, and the Hermes tie-in. Read this to arm PI_RUNNER_ESCALATE.
• reference/orchestration.md — Claude-Code-as-console: dry-run → background live → poll run-status.json, fleet, --until, debug vs production. Read this to operate a run.
• reference/worktree-isolation.md — the opt-in --worktree physical isolation for parallel fleets: what it does, the prompt-rewrite, node_modules symlink, status-stays-in-main, and the conflict-free merge-back recipe. Read this before running a fleet with --worktree.
• reference/read-scope-sandbox.md — the opt-in --sandbox OS read-scope (macOS Seatbelt): the DRIVER-READ-SCOPE marker, buildSandboxProfile's full-runtime-surface grants (cwd + extension dir + linked-package realpath targets), worktree-compatibility, and the two behavioral watchdogs. Read this before running with --sandbox.
• reference/provider-and-headless.md — the native ~/.pi/agent/models.json credential setup and the headless pi invariants/watchdog. Read this for setup + when a node hangs.

Reference implementation

The original, battle-tested instance lives in the animation-test repo at pi-runner/ (the lesson-build workflow → 14 nodes / 10 stages with parallel voice/asset/compose lanes). Its run.mjs / extract.mjs are byte-identical to these templates — the animation-test instance was converged onto this generic engine, with its wiring (PI_RUNNER_CWD=remotion-svg-primitives, PI_RUNNER_WORKFLOW=.claude/workflows/lesson-build.js) living in its gitignored wiring-only .env, and the credential in the machine-global ~/.pi/agent/models.json. When in doubt about a detail, that repo is the worked example; to re-sync after a template fix, cp the generic files (run.mjs/extract.mjs/watch.mjs/status.mjs).