timon-workflows

# Workflows with flow Run `timon` as a command of its own. A host that allows only `timon` commands denies the whole line when it has `;`, `&&`, `|`, `$?`, a redirect or a heredoc around timon. Pass multi-line Lua as one single-quoted argument, which may span lines and cannot contain `'`: use `"` for Lua strings, and put code whose text needs an apostrophe in a file run with `timon eval -f FILE`. Read files with `fs.read` in Lua instead of piping them in, and skip `$?`: a failure prints `timon: kind: message` and exits nonzero. `flow` holds patterns over agent runs, commands and plain Lua. A pattern returns a record and never raises for an outcome: - `status` says whether the pattern ran to the end: `ok`, `failed`, `timeout`, `invalid`. - What it concluded is a separate field: `winner`, `accepted`, `kept`. A review that requests changes, a run that failed and an answer that broke its contract stay three different things. - `runs` holds every agent record the steps returned, and `usage` is `{ runs, usd, unknown_cost }`; Codex reports no cost, so its runs count as unknown, not as $0. A step returns an agent record (its `answer`, or its `text`, becomes the value), another pattern's record, or a plain value. Wrap a pattern in `scope(fn, { budget = { runs = N } })` to cap what it spends. ## Gates A gate checks a value. Prefer a deterministic gate when one exists; ask a model when only judgment will do. ```lua local tests = flow.check({ "true" }) -- a command: passes on exit 0 local short = flow.check(function(s) return #s <= 72, "at most 72 characters" end, "fix: a subject") local real = flow.check({ prompt = "Is this finding real?" }, { file = "a.rs", line = 3 }) -- yes_no return tests.passed, short.passed, real.status ``` A gate function returns `true` or `false` (and a reason), `{ passed, reason }`, or an agent record with a `yes_no` or `review` answer. `passed` is nil when the check did not run. ## Verify, then keep what holds ```lua local findings = { { title = "off by one", line = 3 }, { title = "style nit", line = 0 } } local r = flow.verify{ items = findings, checks = { function(f) return f.line > 0, "points at a line" end, { prompt = "Is this finding a real bug? Answer no for style." }, }, keep = "majority", -- or "all", "any" } return #r.kept, r.usage.runs ``` `majority` refutes: it keeps an item unless most of the gates reject it, so a tie keeps it, and a gate that failed to run counts as no rejection. Kept is not verified. `all` keeps an item only when every gate ran and passed; `any` keeps one when at least one gate ran and passed. ```lua local yes = function() return true end local no = function() return false, "not a bug" end local down = function() error("reviewer unavailable", 0) end local function kept(keep, checks) return #flow.verify{ items = { "finding" }, keep = keep, checks = checks }.kept end assert(kept("majority", { yes, no }) == 1) -- a tie keeps assert(kept("majority", { no, down }) == 1) -- one rejection is not most of two gates assert(kept("majority", { down, down }) == 1) -- no gate ran, so none rejected assert(kept("all", { yes, down }) == 0) -- every gate must run and pass assert(kept("any", { no, down }) == 0) -- no gate passed assert(kept("any", { no, yes }) == 1) -- one gate passed ``` `status = "ok"` says the pattern ran to the end, not that a gate accepted anything. Read each item's `passed` and `rejected` counts and its `verdicts` to tell an accepted item from an inconclusive one: ```lua local down = function() error("reviewer unavailable", 0) end local r = flow.verify{ items = { "critical finding" }, checks = { down, down } } local item = r.items[1] local inconclusive = {} for _, v in ipairs(item.verdicts) do if v.status ~= "ok" then inconclusive[#inconclusive + 1] = v.status .. ": " .. v.error.message end end assert(r.status == "ok" and #r.kept == 1 and item.passed == 0 and item.rejected == 0) assert(#inconclusive == 2 and inconclusive[1] == "failed: reviewer unavailable") return inconclusive ``` Use `keep = "all"` when every gate must accept an item; an item no gate could judge is then rejected, not kept. ## Vote ```lua local r = flow.vote{ n = 3, run = function(i) return agent.run{ prompt = "Is the release ready? Voter " .. i, output = "yes_no" } end, } return r.winner, r.agreed, r.tally ``` The vote stops as soon as one key reaches `quorum` (default: a majority of `n`). `key` decides what counts as the same answer; the default reads a `yes_no` answer. ## Best of n ```lua local r = flow.best_of{ n = 3, run = function(i) return ("candidate %d: "):format(i) .. ("x"):rep(i) end, score = function(text) return -#text end, -- deterministic: the shortest wins } return r.winner, r.value ``` `judge = function(values) return index end` picks instead of `score`; it may run an agent. ## Refine until a gate passes ```lua local r = flow.refine{ draft = function() return "feat: add the thing that does the stuff and more besides that" end, check = function(subject) return #subject <= 50, "subjects are at most 50 characters" end, revise = function(subject, verdict, round) return subject:sub(1, 50 - round) end, rounds = 3, } return r.accepted, r.value ``` `revise(value, verdict, round, last)` gets the failed verdict and the last step's agent record, so an agent can revise in its own session with `agent.continue(last, ...)`. ## Pipeline, retry and plan ```lua local p = flow.pipeline({ "src/a.rs", "src/b.rs" }, { { name = "size", run = function(path) return #path end }, { name = "double", run = function(n) return n * 2 end }, }) local attempts = 0 local r = flow.retry(function(attempt, last) attempts = attempt if attempt == 1 then error("flaky", 0) end return "done after " .. last.error.message end, { times = 2 }) local plan = flow.plan{ steps = { find = { run = function() return { "a", "b" } end }, fix = { needs = { "find" }, run = function(deps) return #deps.find end }, } } return p.values[2], r.value, plan.value.fix ``` - `pipeline` runs every item through the stages with no barrier; an item stops at its first stage that is not ok. - `retry` calls again on `failed`, `timeout` or `invalid`; `last.error.data` holds an invalid answer to repair from. - `plan` starts each step when the steps it `needs` succeed; the first failure stops new steps. ## As tasks A workflow is a task or a script. The review a team runs every day: ```lua fragment -- .timon/tasks.lua review = { description = "Review the branch, then keep the findings both verifiers think are real", budget = { runs = 8, usd = 2 }, run = function() local review = agent.run{ agent = "reviewer", prompt = "Review the diff against main", output = "review" } if review.status ~= "ok" then return review end local r = flow.verify{ items = review.answer.findings, keep = "all", checks = { { prompt = "Is this finding real? Read the code it names." }, function(f) return git.diff{ paths = { f.file } } ~= "", "the file changed on this branch" end, } } return { kept = r.kept, dropped = #r.rejected, usd = r.usage.usd + review.usage.usd } end, }, ``` ## From a sequence to a workflow Offer a workflow once you run the same sequence of agent work and checks on a second input. Bug reports, audited by hand: reproduce one with a failing test, check the test fails and shows that bug, report. The offer is a script in the project's `.timon/scripts/`; kept after the person agrees, the steps and the success criteria are fixed and the report is the input: ```lua fragment -- .timon/scripts/audit.lua -- Reproduce a bug report with a failing test, verify it, report: timon audit REPORT.md local report = fs.read(arg[1]) phase("reproduce") local repro = agent.run{ access = "write", worktree = true, output = "investigation", prompt = "Reproduce this bug with a failing test. Do not fix it.\n\n" .. report, } if repro.status ~= "ok" or not (repro.worktree and repro.worktree.kept) then return { reproduced = false, record = repro.id } -- failed, or wrote no test end local dir = repro.worktree.path local function a_test_fails() local t = proc.run({ "cargo", "test" }, { cwd = dir }) if t.code == 0 then return false, "every test passes" end if not t.out:find("test result: FAILED", 1, true) then return false, "cargo test exited " .. t.code .. " before a test failed" end return true, t.out:match("test %S+ %.%.%. FAILED") or "a test fails" end phase("verify") local r = flow.verify{ items = { repro.answer }, keep = "all", checks = { a_test_fails, { cwd = dir, prompt = "Does the new failing test here show this bug and no other?\n\n" .. report }, } } return { reproduced = #r.kept == 1, -- both gates ran and passed branch = repro.worktree.branch, -- the test, to review or build the fix on verdicts = r.items[1].verdicts, -- why, gate by gate usd = (repro.usage.usd or 0) + (r.usage.usd or 0), } ``` ```sh timon audit reports/88.md # the report it was built on: does it agree with the audit by hand? timon audit reports/91.md # the next report ``` A workflow worth keeping does a job the project has, on the project's inputs, and its result says what to do next. A script that calls each `flow` pattern on made-up values is a smoke test of the API: it belongs in tests, if anywhere, not in `.timon/`.