weft-sdp

# Sequential Diffusion Programming You build one stage against a real input, run it, and read the result before growing the next stage. You judge a result by what the program is meant to do: completion alone says nothing. If you catch yourself calling a run good because it completed, stop and write: "Wait. Read the result." Then inspect `weft events <run-id> --node <id>` and `weft logs <run-id>`. The terms below: - A [start] is where a run begins: a node named by `--from`, a group by `--group`, a trigger by `--fire`, or outputs supplied by `--emit`. - A [cut] is where a run stops: `--target` includes that endpoint, `--before` excludes it and everything beyond. - A [backup] is a value you supply on a [start]. The runtime waits for the run's real input first and uses the [backup] only after the real source closes without a value or no source participates. A real value wins, including null; a running source is never overwritten; errors and invalid real values stay errors, a [backup] does not hide them. - An [example] is a set of running parameters saved by `--save name` in `examples/name.json`, whose field names are camelCase (`keepFor`, `holdSecs`), like every JSON file weft writes. A [frozen example] is an [example] that `freeze` has given accepted output evidence, stored as `expected`. - [head] is the version this folder is on, with the run it came from when there is one. Each working copy has its own: this folder, another folder of the same project, the copy programs write into. A folder's first command starts it at the head of the project's most recently changed working copy. `weft branch` moves it, and so does every command that records a version. ## Carve the work you want to exercise `weft run` builds the current code and starts one run. You pass `--detach`, then read `weft runs --json`, `weft events <run-id> --node <id>` and `weft logs <run-id>`: failures and waiting states as well as outputs. You keep the user's graph intact while trying a stage: ```bash weft run --from classify='{"text":"the invoice is wrong"}' --target reply --save invoice --detach weft run --group triage='{"text":"the invoice is wrong"}' --detach weft run --from triage='{"text":"the invoice is wrong"}' --before publish --detach ``` - `--from node='{"port":value}'` starts that node and its downstream work; the downstream work brings its other input producers with it. A bare `--from node` supplies no [backup]. Repeat it for several [start]s. Walking upstream stops at every named from/emit and at triggers: those are the boundaries. - `--target node` and `--before node` set the [cut]; repeat either for several endpoints. A group or loop name includes the whole container with `--target`, or excludes it with `--before`. - `--group group='{"port":value}'` runs the whole group or loop alone; `--from group='{"port":value}'` runs it and continues downstream. You choose one form: `--group` cannot combine with from, emit, target or before. An included file uses its group alias, and a node inside one is named through that alias (`--from triage.up`, `--target triage.up`), which is also how `weft events` prints it. A group's entry and exit rows print under the group's own name (`triage`, with `boundary=in|out`), and `--node triage` shows them; there is no `triage__in` to name. - Whatever cut you choose, an input left with nothing is checked before the run starts: its wire's source is outside the run and you handed no value. If a node would skip without it (a required input, or the last member of a `@require_one_of` set that could get a value; the node's own, or one inside the group it enters, followed through every group, loop and included file), the run is refused, naming the input, its source, and what it feeds. Hand it a value at your start, or start further up so the source runs. A start is where the walk upstream stops, so neither `--group` nor `--from` runs what feeds the start's own ports: only what feeds the nodes AFTER it. On a `--seed` run a saved result can feed it instead. When the source is a trigger the run does not fire, the refusal says so: `reply.chatId gets nothing in this run: it only comes from the trigger ask, which this run does not fire, and reply cannot run without it. Fire it (--fire ask='<event>'), or hand the value at a start (...)`. `--emit` standing in for the trigger works too. - `--check`, with the same `--from`, `--group`, `--target`, `--before`, `--emit` and `--fire` (never with `--seed`, `--save` or `--detach`), only says whether that run could start (`the run can start`, or the refusal and exit `1`), building and running nothing. - `--feed start` runs what feeds a start: for each of its inputs you did not hand, the node that feeds it (through every group or include door on the way, however deep), and nothing above that node. A loop feeding it runs whole. The way to run one group whose connections come from outside: `--from 'hear.note={}' --feed hear.note --target hear.note`. Naming the feeders as starts yourself works too: when one start lies upstream of another, the run keeps both. - A start whose gate (`_should_flow`, its own or a surrounding group's) only comes from triggers the run does not fire, or from outside the run, is refused: it would skip and still report completed. `--feed` never opens a gate: it runs what feeds a start's inputs, and the gate is not one of them. Hand the gate a value at its own door, the start for its own gate (`--from 'start={"_should_flow": true}'`) and the group for a surrounding group's (`--from 'group={"_should_flow": true}'`, or `--group` if the run was a `--group` run), or `--fire` the trigger. - Without `--target` a run reaches everything downstream of its starts, so `--from db` runs every branch `db` feeds. `--target` keeps only what the target needs, from the starts down: it is a shape, not "run until you reach the target". - `--emit node='{"port":value}'` supplies that node's outputs without running its body. Repeat for several supplied nodes. An output with no consumer remains valid output evidence. For a generator port, you supply all its items in one array: `--emit batches='{"items":["first","second"]}'`. The runtime emits each item and then closes the port; `[]` closes it without an item. A list-typed ordinary port receives its array as one value, so you read the actual port type before choosing. If a branch needs a file its real source cannot give you here (a WhatsApp voice note needs a paired phone), you hand one from disk. Any value given to `--emit`, `--from`, `--group` or `--fire`, or saved in an example, may be a marker written exactly as in source, with the same meaning: `"@asset(\"samples/hello.ogg\", Audio)"` is uploaded into the project's storage and becomes the file value the port expects (its type names one kind: `Image`, `Video`, `Audio` or `Blob`), and `"@file(\"prompts/x.md\")"` becomes the file's text. A path is relative to the project root. Inside the JSON the inner quotes are escaped: ```bash weft run --emit receive='{"messageType":"audio","file":"@asset(\"samples/hello.ogg\", Audio)","seconds":3}' --target transcribe ``` A missing file is refused before the run starts, and so is a file on a port whose type does not take it, the same check as any other handed value. A saved example keeps the marker, so it reads the file afresh on every run. A [cut] inside an ordinary group stays at the requested node, and the group's `_should_flow` gate still applies: false stops the branch; true dispatches only the selected work. Loops are indivisible: you use the public loop name for a [start], `--group`, `--target`, or `--before`; a node inside the loop cannot be an endpoint. Trigger setup and infra preparation obey the same group cuts and loop restriction. ## Choose how a trigger participates You read the trigger's implementation and its ports before forming either payload. ```bash weft bake weft run --fire incoming='{"event":"the trigger wake payload"}' --detach weft run --emit incoming='{"message":"the emitted output value"}' --detach ``` If the program carries infrastructure, bring it up first with `weft infra start`. A run refuses while a node's container is not up, naming the node, and so does a bake whose trigger setup reads one; neither starts it. This is also how you test an infra node: live tests do not cover infra nodes for now, so you prove one inside a real program. Start its infra, then `--target` the node that reads it, with `--from` or `--emit` supplying the values upstream of it, so each run exercises the container and nothing else. A fire names exactly one trigger; its payload wakes that trigger, which runs and decides what to emit. An emit supplies the trigger's declared outputs directly. You choose one per trigger. Trigger ports receive their prepared settings through `weft bake`; a trigger cannot be a `--from` [start]. A run that reads nothing from a running [infra node] but its baked outputs uses the values the node saved instead of running it (baked outputs have nothing to do with `weft bake`). After you change what the node emits on one, name the node in the run (`--target <node>` or `--from <node>` always runs it) to test the new code. To update the value every other run gets, run `weft build` and then `weft infra rebake <node>`, which restarts nothing. A fire is durable or fast, and kept as long, as the trigger's `durable` and `keepRunsFor` say, and `--durable`, `--fast` and `--keep-for` override them for that run. A fire you start by hand is always recorded, even when the trigger has `recorded: false`. `bake` runs preparation and saves the trigger settings without listening. Changed code or configuration needs a matching new bake. Closed group gates can leave a trigger unprepared; you inspect the bake instead of forcing it. `weft activate` prepares and arms real listeners. `weft bake <project-id>` uses that project's registered build. If you deliberately use `--referenced`, you pass it to both bake and run. ## Reuse completed work while developing `weft run --seed --detach` takes eligible results from [head]'s run. If [head] is a version without a run, it finds a finished run on that version or its nearest ancestor. `--seed=<run-id>` takes them from any run of the project instead, one a trigger started or one that failed included, which is how a run that failed or broke off is picked up again after a fix (the `=` is required, since a bare word after `--seed` is an example name). `--root` starts a new tree with nothing to reuse, so it is refused beside a seed. If you also want that run's code back, `weft branch <run-id>` first: it writes that run's files into the folder, after recording the files you had as a version, and prints the `weft branch <id>` that comes back to them. - `--seed --seed-before classify` reuses earlier compatible work and runs `classify` and what follows. - `--seed --seed-until classify` also permits reusing `classify`. These flags limit reuse within the requested run; the [cut] limits what runs. A node whose saved output names a stored file that has been edited since, or is gone, cannot be reused by a node that runs again: the run is refused before it starts, naming the node, the port and the file, and you either hand the file in with `--emit` or run the node again with `--seed-before` it. Changed implementations, inputs, dependencies, failed work, and live handles can prevent reuse. A loop is reused whole. An entirely reused run is valid and does no new node work. This applies inside a saved or carved run too: `--from` chooses its boundary, not a forced rerun. Identical used [backup]s permit reuse; changing a used [backup] reruns its consumers. You omit `--seed` to run the selected work again. If you catch yourself reporting a seeded run without reading its warnings and inherited-node markers, stop and write: "Wait. What actually ran?" Then read them; never assume a requested endpoint was reusable. Builds include the whole catalog by default. The unchanged standard library uses a shared, finished worker image, so no project compilation is needed when that image is installed or published. Node edits, added nodes, or custom build settings need their own image, and building it compiles only the nodes no earlier build on this machine compiled. Adding an unchanged catalog node or editing graph configuration needs no new image, so you use ordinary `weft run` while iterating. `--referenced` opts into a separate, reduced build compiling only the graph's node types, which can rebuild when those types change; implementation edits still rebuild. ## Save parameters, then freeze accepted results `weft run name` runs the current program with the [example]'s parameters, frozen or not. You add `--seed` only when deliberately reusing results and omit it when reviewing how the current program answers the saved use case. ```bash weft run invoice --detach weft events <run-id> --full weft freeze invoice <run-id> --expect reply # After editing the program: weft run invoice --detach weft diff example:invoice <new-run-id> # After inspecting and accepting the new result: weft freeze invoice <new-run-id> --expect reply ``` `freeze` preserves that completed run's starting parameters and accepted output history together. A seeded whole run preserves its original starting inputs; a carved run preserves its [cut] and the inputs entering it. Replaying the [frozen example] recomputes the selected work on current code; interior reused results do not become hidden fixed inputs. `expected` holds the output evidence `diff` uses, including finite streams and closures. A closure a failure left reads `"closed": true` with `"failure": {"node": "query", "error": "..."}`, the node that broke and its message. You repeat `--expect node` to focus review on particular node outputs; focus changes comparison, not what runs, and a focused output that disappeared remains visible as a difference. Stored media compares by its content hash. `diff` presents changed values for your inspection. Differences are not a verdict and do not produce a failing exit status: you inspect the run's status separately, explain meaningful changes, and replace the [frozen example] only after accepting the new result. Running and diffing leave the accepted file untouched. `weft examples` lists [example]s and [frozen example]s; it does not certify them. ## Recover an example after changing the graph Removed input ports are ignored with a warning. Missing [start]s or [cut] endpoints are errors: you move the [cut] explicitly before running. ```bash weft run invoice --clear from --from newClassifier='{"text":"the invoice is wrong"}' --target reply --detach weft run invoice --clear group --from triage='{"text":"the invoice is wrong"}' --before publish --detach ``` Explicit `--from`, `--target`, `--before`, `--group`, and `--fire` replace the corresponding saved settings. Repeated new from flags build the replacement starting map. Emit flags replace the named supplied ports and keep the other saved emit entries. `--clear <field>` clears a saved field before those edits: `from`, `emit`, `target`, `before`, `group`, `feed`, `fire`, `instance`, `keeping` (what `--durable` or `--fast` saved), `keep_for` or `hold_secs`; repeat the flag for several fields. Duplicate newly supplied ports are errors. Values are never guessed for renamed nodes. You use `--save another-name` to keep revised parameters separately, or freeze an accepted new run to replace the [example]'s parameters and output evidence together. ## Read outside interactions A [frozen example] records how each wait of the run ended, with the question the node showed, and the incoming caller messages, for inspection; the runtime does not replay them on its own. When a run waits on a person, you compare the current question with the recorded one, end the wait the way the recorded entry says within the user's authority, and keep inspecting that same run. Each entry holds exactly one of these: - `payload`: the value the person answered with. Answer the current token with it. - `"skipped": true`: the person skipped the question. Skip it the same way, `POST /signal/{token}/skip`. - `gaveUp`: nobody answered. The run could not pause, held the wait in its worker until nothing had moved for its hold time, and gave the wait up; the text is the error the waiting node failed with. Leave the current one unanswered, so the new run gives up the same way when its own hold runs out. The entries cover every wait that ended, including timers and nodes that poll a job, not only questions to a person. You answer only the waits that ask a person: timers and polls run on their own, and answering one by hand would cut a real wait short. For live connections, you send the recorded messages through a new connection and review the new responses. `weft wake <run-id> <node>` resolves a pure time wait. A wait requiring a value must receive that value instead. Logs and inherited markers identify which earlier run supplied reused history; historical costs are not new charges. ## Keep a reviewable trail `weft checkpoint [label]` records a version without running anything, and the label is a name `weft branch` and `weft prune` accept in place of its id. `weft tree` shows every working copy with its head, then the versions and their runs; `weft branch <version|label|run-id>` puts this folder on a point in that tree. A switch loses nothing: the files the folder held become a version first, and the command prints the `weft branch <id>` that comes back to them. Another folder of the project has its own [head], but a program writing code with `ctx.code()` writes into the project's most recently changed folder unless it names a working copy, so its files can land in yours. `weft prune <version>` deletes history: the version, everything under it, and their runs, so you ask the user before you run it. It refuses when any working copy's [head], an activated version, the version the project's build runs, a frozen example's origin, a version that still supplies trigger settings, or a run that has not ended (a paused one counts) is inside that history. The first refusal names each thing in the way and what to do (`weft branch <version> --copy <copy>` or `weft forget-copy <copy>` for a working copy, `weft deactivate`, build another version, `weft freeze` from another run, `weft stop` the run). Once those are clear, a second one can still come: `these versions still supply trigger settings or running runs; stop the runs and replace or wipe those trigger settings before pruning`, which names no version. No flag skips either (`--yes` only skips the question). The trigger bakes made from the pruned versions are deleted with them. After an edit, you run the relevant saved use cases, inspect their diffs, and report what actually ran, what changed, why a result is acceptable or still wrong, and the run ids.