weft-api

# Building an API Everything here is in the `api` package of the catalog. You read each node's `metadata.json` before wiring it; when a name below drifts from it, the metadata wins. A [route] is one `Route` or `Socket` node: one trigger, and one call is one fresh run carrying the request on its ports. An [answer] is any node that writes back to that caller: the runtime hands the caller's connection to every node in the run, so a node of your own is one as readily as a catalog node. The `api` package ships three: `Reply` sends a body, `Stream` sends a bus as it flows, `Close` ends the exchange. A [gate] is an auth access node wired into a [route]'s `auth`; the package ships one per way of proving who is calling, and the listing names them. A [stored-file value] is a few hundred bytes saying where a file sits in storage, the only way bytes travel a wire. ## The URL is known before anything runs A [route] answers at `<install>/connect/local/<project id>/<path>`, and every part of that address is fixed before the route is ever activated: the install, and the project id, which is the `id` under `[package]` in the project's `weft.toml` (the same on every install, so two projects can serve the same path). On this machine the install is `http://127.0.0.1:14111`, so `hello = Route { path: "hello" }` answers at `http://127.0.0.1:14111/connect/local/<project id>/hello`, and a socket at the same address with `ws://`. On a cloud install it is the target's `url` in `weft.toml` (`https://weft-role-dispatcher-123456789.us-central1.run.app/connect/local/<project id>/hello`, and `wss://`). You write those URLs into the frontend-builder's [the brief] the moment the routes are shaped, while the graph is still being built; `weft activate` only turns them on. Each project also has an address of its own where the program answers its routes at the root with nothing of weft's in between, which is the fastest way in: on this machine a free port it gets the first time it is activated (14200 or above) and keeps, moving to another free one if a program took it (`http://127.0.0.1:14200/hello`; `weft activate --port <n>` opens a chosen one and keeps it; if another program or project holds it, the command fails and nothing is activated), on a cloud install the project's own Cloud Run address. `weft activate` prints it and `weft status` shows it, or says why it is unavailable. While a route's trigger is not on (taken down, or still coming on), a call is answered `503` with `Retry-After: 5` and `This service is unavailable right now; try again in a moment.`, so tell the frontend to retry a `503` that carries `Retry-After`, and never one without it. When you brief a frontend, give it the `/connect/local/<project id>/` address: it has the same shape on every install, so the frontend's code runs unchanged on a cloud. Both local addresses, the project's own port and `127.0.0.1:14111`, answer only on this machine. When the install has a public address (a tunnel), `weft daemon status` prints it, and each route answers there at `<public address>/connect/local/<project id>/<path>`, reaching the same program. A frontend that runs anywhere else (a hosted site, a phone, a browser on another machine) uses the public one; a server on this same machine may use either. A program that is an API can have a domain of its own on a cloud install. If the user wants one and has agreed to its price, the deployer runs `weft domain add api.shop.com --for api --on prod` (with `--accept-cost` for the install's first domain), which serves the project's routes at the root of that domain, so `hello` answers at `https://api.shop.com/hello` as well as at its `/connect/local/<project id>/` address. A domain costs money, because it needs a load balancer, so design for the free address and offer a domain only when the user wants a name people see; read weft-deploying before you offer one. ## The shape The run ends when it has sent its [answer]. The whole request body comes out on one port, `body`, and the type you declare on it on the arrow is the shape you expect: ```weft hello = Route -> (body: { name: String }) { path: "hello", method: "POST" } answer = Reply { status: 201, body: hello.body.name } ``` The [route]'s other fixed output ports carry the request as the gateway saw it: the method, the path as called, the query, the headers, and who the [gate] let in. Its metadata lists them. A path capture has no fixed port: you declare an output of its name (below). What to declare on `body` follows the body shape the [route] is set to (`dataType` in the source, "Body shape" in the editor): - `json` (the default), an object body: declare a record type, or `JsonDict` to take any object unchecked. The record is checked before the run starts, and a body that does not fit it is answered `400` naming the field, with nothing downstream run. A key the record does not declare is refused too, unless the record ends with `..` (`{ email: String, .. }`), which drops it. A field written `?` may be left out by the caller; reading it then closes that one wire (`hello.body.nick` with `nick?: String`). Record fields keep any spelling, so a caller's snake_case key is just declared as it is (`{ user_id: String }`); if your own frontend sends the body, name its keys camelCase, like every field weft writes. - `text`: the whole body is a `String`, declared `-> (body: String)`. - `bytes`: the body is stored and its [stored-file value] lands on `body`, declared `File`, or `Image`, which is checked against the bytes. A request with no body (a `GET`) closes `body`. Declare the capture as an output, and read it as `card.id`: ```weft card = Route -> (id: String) { path: "cards/{id}", method: "GET" } ``` Every output you add to a `Route` must be a capture of its path, typed `String`; the compiler refuses anything else with `route-output-not-a-capture`. A capture may not share a name with one of `Route`'s own outputs (`body`, `method`, `path`, `query`, `headers`, `caller`, `fired`): `cards/{body}` is `route-capture-shadows-output`, asking you to rename the capture. The first thing the program sends commits the status line. A `Reply` on a branch with `status: 404` works. A `Reply` or a second `Stream` after a `Stream` is a compile error, because the head already went out: end a streamed answer with a `Close`. A program that never sends anything holds the caller while it runs. When that run ends, it is recorded as failed with `the run ended without answering: every path to its Reply (or Stream, or Close) was skipped, usually because a value it waited on never arrived`, and the caller gets `500`. If you catch yourself wiring a branch that reaches no [answer], stop and write: "Wait. Every branch answers." Then end that branch on a `Reply` or `Close`. Whenever a run ends before it answers, the caller gets the body `{"error": "Something went wrong", "runId": "<run id>"}`, never the failure's own message (that is on the run: `weft logs <run id>`; opening a run needs the owner's credential, so the id is safe to show), and the status tells it which kind of end it was: | Status | When | |---|---| | `500` | the program failed, got stuck, or ended without answering (the case above) | | `504` | a time limit ran out: the run's hour on a cloud install, the route's `maxSessionSecs`, a wait held for its whole `holdSecs`, or, on a cloud install, the worker stopping just before Cloud Run cuts its connection to weft (which it does after an hour), because every run on it was only waiting | | `503` | weft or a person stopped the run (`weft stop`, `ctx.stop_tagged`, a take-down with `--running cancel`), or the run carries on without this caller (it paused, or moved to another worker) | This `503` has no `Retry-After`: calling again would start a second run beside the one that may still be going. The `503` a route sends while its trigger is not on does carry `Retry-After: 5`, because no run started. If the answer had already started streaming, its status line is gone, so the error comes in the body: a JSON-lines answer ends with `{"error": "Something went wrong", "runId": "...", "status": 504}`. A socket closes with `1011` for a failure or a time limit, and `1013` when the run was stopped or carries on without this caller, with the reason `Something went wrong (run <id>)`. Every answer of a route's run, success or failure, carries the header `Weft-Run-Id`, so a site can log the run id beside a failure. Two routes of one account can share a call as long as one of them spells out what the other captures: `chat/general` beside `chat/{room}` is fine, and the literal takes that one call while the capture takes the rest. What `weft activate` refuses, naming both, is the pair where neither is the more specific (`chat/{room}` beside `chat/{name}`, or `a/{x}/c` beside `a/b/{y}`, which both answer to `a/b/c`). Give one of them a literal the other captures, or distinct methods. ## Request-response The graph decides the status. A Python node reading the `id` capture and `query` returns the body and the status together: ```weft user = Route -> (id: String) { path: "users/{id}", method: "GET" } lookup = ExecPython(uid: String, query: Dict[String, String]) -> (body: JsonDict, status: Number) { uid: user.id query: user.query code: "if uid == '42':\n return {'body': {'id': uid, 'verbose': query.get('verbose')}, 'status': 200}\nreturn {'body': {'error': 'no user ' + uid}, 'status': 404}" } found = Reply { body: lookup.body, status: lookup.status } ``` A `GET` route has no body: read the captures and the fixed ports. A `text` route: `say = Route -> (body: String) { path: "say", dataType: "text" }`, and the `Reply` body must be a `String`. If the answer has a different shape from what came in, the [answer] node says which with `answerAs`: `json`, `text` or `bytes`, and empty follows the route's `dataType`. A `GET` that answers a picture is the usual case: it takes no body in, so the route keeps its default, and the [answer] node (a `Reply`, say) sets `answerAs: "bytes"` with a [stored-file value] as the body. Sent that way over HTTP, the file goes out with its own content type and an inline `content-disposition` carrying its filename, unless `headers` sets them. A run that stops without a body ends through the closing [answer] node, which takes no data from upstream: its `_should_flow` gate is the whole wiring. So the shape for "do this, and if it did not work, stop here" is a `Switch` on the outcome, the failing case gating a close with its reason and status already written on it, the passing case gating the rest. ```weft sweep = Route -> (what: String) { path: "sweep/{what}", method: "DELETE" } clear = ExecPython(what: String) -> (removed: Number) { what: sweep.what code: "return {'removed': 3 if what == 'cards' else 0}" } outcome = Switch { value: clear.removed cases: [ { "kind": "gt", "value": 0, "port": "some" }, { "kind": "otherwise", "port": "none" } ] } report = Reply { body: clear.removed, _should_flow: outcome.some } nothing = Close { status: 404, reason: "nothing to sweep", _should_flow: outcome.none } ``` The gate takes any port of any type; the value is never read, only a `false` says no, so wire a Boolean into it only when its `false` should mean "do not close". A branch that skipped closes its ports, so the `Close` behind it skips too and the other branch's [answer] stands. The gateway answers on its own before any run starts: `405` for a served path called with the wrong method (naming the verbs it serves), `404` for an unknown path, `401` for a caller the [gate] refused. If you want a webhook receiver that says `200` at once and does the slow part after, wire the `Reply` early in the graph, then the rest, and set `outlivesCaller: true` so the caller hanging up does not cancel the run. A slow job (anything that takes minutes: a render, a build, a batch of model calls) never sits in one call the caller waits on. Answer early (a `202` with an id), run the work after the answer with `outlivesCaller: true`, and show progress: on the infra node's display when the work runs in a container (the `weft-node-authoring` skill has the display), or as a status the caller polls. On a cloud install one stretch of a run lasts an hour at most, so a job that can run longer is split with a pause (a timer), which gives it a fresh hour. If the caller polls for a status, you need no table: the run's own record has it. The answering route hands back its run's id (`GetRunId`), and a second route reads one node of that run with `GetNodeState { runId, node }`, which answers where it stands (its status, an outside job's last answer while it waits, or its error). A table in the project's Postgres is for a status the program itself decides (a step count, a percentage it works out): the answering route writes the row, the work updates it, and the second route reads it back. If a page asks a route for a status every few seconds, set `recorded: false` on the `Route` so each call does not leave a run behind. weft keeps no history of such a run: one that fails is listed in `weft runs` with the node that failed and why, one that reported a cost, or asked weft for something its worker does not already hold (a stored file, a new connection or infrastructure address, starting its infra, a stop by tag, a wait on a timer or a form), is listed with its costs and how it ended, and any other leaves nothing. An unrecorded run keeps no record to pause on, and it cannot be tagged. A run that cannot pause (an unrecorded one, or one whose caller is still on the line with `outlivesCaller` off) still waits on a timer or a form: the wait holds the worker and the call open, and the answer carries the run on. If nothing moves in the run for the route's `holdSecs` (60 by default, `0` to never wait), the waiting node fails saying it gave up, and the caller gets a 504 with the plain failure body and the run id (the reason stays on the run, for you). A route also has two settings for what a streamed answer leaves on the run's record: `journalEphemeral` (on keeps only counts and sizes, never what was said; off by default) and `journalWindowMs` (messages sent within one window share one record row; 100 to 60000, 1000 by default; shorter shows them in the inspector sooner, longer writes fewer rows for a fast stream). A route also has `durable`, off by default, and it cannot be combined with `recorded: false`. With it off, a run whose worker dies ends cancelled. With it on, another worker carries the run on from its finished nodes, but the caller's connection died with the old worker, so a node that answers the caller after that fails and the caller has to call again. For the details, go and read How a run is kept in the `weft-language` skill. ## Pictures in and out A value a node emits is at most 100 KB; above that the node fails, naming the port. A picture a caller sends inside a JSON body is declared as a field of the body's record, typed as its kind, and the [route] stores it on the way in: ```weft upload = Route -> (body: { photo: Image, caption: String }) { path: "cards", method: "POST" } ``` The body key `photo` holds a `data:image/png;base64,...` URL (the media type comes from it) or bare base64 (the media type comes from the bytes' own signature); `upload.body.photo` then carries the [stored-file value]. Bytes that are not what the field declares get a `400` naming the field, and nothing past the route runs. Multipart bodies are not read: the caller sends JSON with a data URL. A file too big for a JSON body (a video, a long recording) goes through an upload slot. The program mints one with `UploadSlot` and hands its `slot` string (`weft-upload:...`) to the page; the page uploads the file with it, then calls the route with that string in a field the route declares as a file (`{ "video": "weft-upload:..." }`), and the route receives the stored file. A slot takes one file, and an upload no route received within its `minutes` is deleted. For the page's side, go and read the `weft-frontend` skill. A file sent in the body lands at run scope, walled to that run and swept when it ends: a later request cannot read it even if it was kept. If you want to serve a picture from a later request, wire it through `KeepFile { scope: "project" }`, which copies it into the project's storage, then write the whole [stored-file value] it emits into a jsonb column, exactly as it arrived. If it should not live for ever, give `KeepFile` a `ttlDays` too: the copy then expires that many days after anybody last read it. Two columns, and only one of them is right: - the whole value in a jsonb column: correct, and every answer builds a fresh link out of it. - a `url` you pulled out of that value and put in a text column: dead by morning, because the link is minted per answer and expires in minutes. So never unwrap the value, never pull a field out of it, never strip its `url` before writing it. `Reply` and `Stream` recognise a stored file by its whole shape, and a value you edited comes back to the caller as a raw storage key instead of a picture. Sending that picture back needs nothing else. An [answer] node walks the whole body and turns every stored file it finds, however deep (in an object, in a list of rows), into `{ url, mimeType, filename, sizeBytes }`, so a list route serving rows with pictures is three nodes and no loop. The `url` is built on the address the caller used: a call to `127.0.0.1:14111` gets a loopback link, a call through the tunnel gets a tunnel link. A run no request started (`weft run --fire`) gets the install's internet address when it has one, loopback otherwise. A frontend puts the `url` straight in an `<img>`. `Cast` the value to `Image` or `File` only when you want the file as a typed value on a wire, to hand it to a node that takes a picture. If you want to serve a picture that is redrawn often (a live preview), the shape is one place that always names the latest copy, which every render overwrites and the serving route reads and answers as bytes. A render's own file is at run scope and swept when its run ends, so a link to it dies with that run: each render is kept in the project scope first. One way to build it, with a table: each render goes through `KeepFile { scope: "project", ttlDays: 1 }`, and its whole value goes over one row per preview (`INSERT INTO preview (view, picture) VALUES ($view, $picture) ON CONFLICT (view) DO UPDATE SET picture = EXCLUDED.picture`). No storage node rewrites a picture in place, so the row is what stays put: the old copies expire a day after their last read, and the one on screen is read on every call, which resets its clock. The route that serves it, with `db` the program's `PostgresDatabase`: ```weft preview = Route -> (view: String) { path: "session/preview/{view}", method: "GET" } latest = PostgresExecuteQuery(view: String) -> (row: { picture: JsonDict }) { account: db.access view: preview.view query: "SELECT picture FROM preview WHERE view = $view" } show = Reply { answerAs: "bytes", body: latest.row.picture } ``` An `<img>` can point straight at that route. A json answer works for a project-scope file too, through the `url` the [answer] builds. If you catch yourself putting base64 on a wire or reading a multipart body, stop and write: "Wait. Files are links." Then declare the port `Image` or `File`. ## Streaming