# Factory
A software factory: one Cloudflare Worker that receives GitHub webhooks and
routes them through a deterministic dispatcher to durable
[Flue](https://flueframework.com/) agents. Each capability (triage, review,
and more to come) is a folder of workflows, coordinators, and agents; adding a
capability means adding a folder and a routing rule.
Built by combining [withastro/astro-review](https://github.com/withastro/astro-review)
(absorbed nearly as-is — it already had this architecture) and
[withastro/triagebot-action](https://github.com/withastro/triagebot-action)
(ported from GitHub Actions to Workers).
## Scope
Factory is the triage and review automation **for the Astro repository's own
use** (`withastro/astro`): its routing rules, labels, skills, and the
release-security capability are tuned for how the Astro team works. It is
**not a general-purpose tool** for anyone to install — the package is private
by design, and the default configuration targets Astro's workflows.
If you want similar automation for your own project, **fork this repository**
and build your own workflows: point the router (`src/router.ts`) at your
events, adjust `.github/factory.yml` and the labels for your repository, and
replace the skills where your workflow differs. The architecture is modular
on purpose — capabilities are folders plus routing rules — so forking and
adapting beats installing.
## Architecture
```
GitHub webhooks ─→ Hono ingress (signature verification)
└→ router.ts (pure rule table: event → capability dispatch)
├→ ReviewCoordinator DO (one per PR) ─→ ReviewWorkflow ─→ PullRequestReviewer agent
├→ AdversaryCoordinator DO (one per PR) ─→ AdversaryWorkflow ─→ BlueTeam / PurpleTeam agents
├→ TriageCoordinator DO (one per issue) ─→ TriageWorkflow ─→ FixVerifier / RetriageJudge agents
├→ AuthorCoordinator DO (one per PR) ─→ AuthorWorkflow ─→ CodeAuthor agent (one conversation per PR)
├→ AdvisoryCoordinator DO (one per advisory) ─→ AdvisoryWorkflow ─→ AdvisoryTriager agent ─→ Discord
├→ ReleaseSecurityCoordinator DO (one per PR) ─→ ReleaseSecurityWorkflow ─→ ReleaseSecurityReviewer agent
├→ PrSafetyCoordinator DO (one per PR) ─→ PrSafetyWorkflow ─→ rule signals + PrSafetyReviewer agent ─→ Discord
└→ MergeRiskCoordinator DO (one per PR) ─→ MergeRiskWorkflow ─→ MergeRiskAssessor agent ─→ one scored PR comment
```
- **Router** (`src/router.ts`): deterministic and pure. `pull_request.labeled`
→ review; `issues.opened|reopened|closed` and human `issue_comment.created`
→ triage; `issues.assigned`, `pull_request.assigned`, and
`pull_request.review_requested` → [personas](#personas);
`repository_advisory.reported` → advisory triage; `pull_request.opened|reopened|synchronize`
from non-maintainers and dependency bots → [PR safety](#pr-safety-srcpr-safety); human reviews,
review comments, comments, and failed checks on Factory pull requests → the
code author; `check_run.completed` on Factory's own `Factory Review` check
→ [merge risk](#merge-risk-srcmerge-risk). Bot activity is dropped at the
door to prevent self-trigger loops.
- **Coordinators** (`src/coordination/queue-coordinator.ts`): a Durable Object
per entity serializes work — one active workflow, one pending (newest wins),
delivery-id dedupe, and a reconcile alarm for self-healing. This replaces
GitHub Actions' `concurrency` groups.
- **Workflows**: every side effect is a checkpointed, retried step. The triage
workflow re-reads issue labels when it runs and routes through the FSM
(`src/triage/fsm.ts`), so queued events always act on fresh state.
- **Agents**: Flue agents, defaulting to Workers AI (Kimi) through a shared
**Cloudflare AI Gateway**. Repositories can name a different gateway-routed
model per capability, including Anthropic models (see [Models](#models)).
The reviewer gets read-only GitHub tools; the triage
classifiers get no tools at all, only the conversation text. Only trusted
workflow code writes to GitHub.
## Capabilities
### Review (`src/review/`)
Adding the configured trigger label to a pull request runs the
bundled review skill, or a repository-provided override, and publishes
validated findings as a PR review (inline comments anchored against the real
diff, the rest in the body, always with an LLM disclosure). Repository config
and skill overrides are read at the target branch's tip SHA captured at webhook
time — never from the PR head. When review is triggered again, the agent also
rechecks unresolved inline threads from its latest prior review and resolves
only those it determines have been addressed. If GitHub does not allow the App
installation identity to resolve a thread, Factory leaves it unresolved without
failing the new review.
### Adversary (`src/adversary/`)
Adding the configured adversary label starts an independent alternative-design
exercise for a public pull request. The submitted PR is red, a blue agent starts
from the exact base commit without access to red's implementation, and a purple
agent evaluates both exact trees in a fresh container. Purple qualifies blue
only when it solves the same problem, is materially different, is verified,
preserves relevant safeguards, and remains appropriately scoped.
Blue and purple receive credential-free Cloudflare Sandbox containers and
discover the repository's own install, build, and test tooling. No commands are
configured in `factory.yml`. Blue's binary patch is streamed through a private
R2 artifact between isolated containers. Only after purple qualifies it does a
third clean container receive a short-lived contents token. When purple selects
blue, that container pushes the alternative branch, Factory opens a draft pull
request using the caller repository's pull request template, and the original PR
receives a comment linking to it. Every other purple verdict produces only a
concise decision comment. The maintainer then chooses which proposal to pursue.
The first version supports public repositories only.
```mermaid
flowchart TB
L[Adversary label] --> W[Cloudflare Workflow]
W --> B[Blue container<br/>Implement from base]
B --> D[git diff creates blue.patch]
D --> A[(R2 stores blue.patch)]
H[GitHub] -->|Clone PR into /red| P[Fresh Purple container]
H -->|Clone base into /blue| P
A -->|Workflow downloads and git applies patch to /blue| P
P --> T[Test and compare /red and /blue]
T --> G{Blue qualifies?}
G -- No --> R[Check and comparison]
G -- Yes --> S{Purple selects Blue?}
S -- No --> R
S -- Yes --> U[Clean publisher container]
A -. Same verified patch .-> U
U --> C[Alternative branch]
C --> O[Factory opens draft PR]
O --> R
R --> M[Maintainer chooses Red or Blue]
```
### Triage (`src/triage/`)
A label-driven state machine over issues, with all state living in GitHub
labels (visible, maintainer-overridable):
- Issue opened/reopened → the full pipeline (reproduce → diagnose → verify →
fix) runs in a **Cloudflare Sandbox container** holding a real checkout of
the repository: a hardened blobless clone of the default branch, a
`factory/fix-N` branch, the skill seeded into the workspace, an
[install and optional build](#bootstrapping-the-checkout) of that checkout,
and a shell for building and testing. The workflow then commits and
force-pushes any
changes (the contents-scoped token exists only inside that one step and
never reaches the agent), optionally opens a PR (`autoPrOnFix`), publishes a
[preview release](#preview-releases) when configured, generates the triage
comment from the pipeline's `report.md`, applies the resolved state label,
and selects priority/package labels.
- While the pipeline runs, the issue carries `triage: in progress` and one
delivery-scoped comment updates a checkbox list as each durable stage
completes. The same comment becomes the final report, so workflow retries do
not create duplicate status comments.
- Comment on `triage: fix pending` → the FixVerifier agent classifies the
reporter's response: confirmed → open the fix PR + `fix verified`;
rejected → `fix rejected`.
- Comment on a re-triageable label → the RetriageJudge agent decides whether
new actionable information warrants a re-run.
- Issue closed → the fix branch is deleted. A closed issue is then out of
scope whatever its triage label says: comments on it neither verify a fix nor
re-triage, so nothing pushes a branch or opens a pull request for an issue a
maintainer has already decided about. Reopening it resumes normal routing.
- Unexpected failures post a marked comment; three strikes parks the issue in
`triage: failed` until a maintainer clears it.
Missing labels are created automatically with sensible colors, so installing
on a fresh repository requires no setup.
### Code author (`src/author/`)
The code author persona owns any same-repository pull request assigned to
it. Triage assigns it to every fix pull request it opens. See
[Personas](#personas) for how it is addressed and how it works with the
reviewer persona.
- **Adoption.** Assigning the persona to a pull request posts an ownership
status comment and immediately addresses any outstanding requested changes
and failing checks.
- **Rounds.** A round starts when a maintainer or Factory's reviewer
**requests changes**, or when a check suite or workflow run on the branch
fails. Comments and review threads are read as context but never start a
round on their own. The workflow re-reads the pull request when it runs,
so feedback that arrived while a round was running collapses into the next
one. A round clones the branch into a credential-free sandbox (bootstrapped
with the triage `installCommand` / `buildCommand`), stages the failing
Actions job log tails, and hands the agent the round's feedback. The
workflow then commits the agent's working tree and pushes it **without
force**, rebasing once onto commits a maintainer pushed meanwhile and
aborting rather than resolving a conflict. It replies to the review threads
the agent answered, resolves the ones it says are fully addressed, marks
the ones it declined, and posts a round summary. The persona never merges.
- **Handing back.** After a round, it requests review again from everyone
whose requested changes it answered, even when it disagreed and pushed
nothing. Any push also re-requests the reviewer persona.
- **Memory.** The agent keeps one durable Flue conversation per pull request
(`author:<repositoryId>:<pullNumber>`), so each round sees what earlier
rounds tried.
- **Trust.** Only feedback from maintainers (`OWNER`, `MEMBER`,
`COLLABORATOR`) and from this Factory installation's reviewer reaches the
agent; everything else is withheld, not merely labelled. Other bots never
start a round.
- **Budget.** A pull request gets `personas.author.maxRounds` rounds (default
5), failed rounds included. When changes are requested after that, the
persona comments that it is handing off, labels the pull request
`factory: needs human`, and unassigns itself. Reassigning it starts a
fresh budget, and unassigning it cancels queued rounds. Ownership state
lives in the status comment, and it is trusted only when this GitHub App
wrote the comment.
### Release security (`src/release-security/`)
Factory privately reviews same-repository `withastro/astro` release PRs from
`changeset-release/<base>` when they are opened, reopened, or synchronized. A
smoke-only path uses `release-security-test/<base>` with the exact title
`[test] release security reviewer`; it checks model health without performing a
release review. Maintainers can rerun either managed check from GitHub.
Each PR has one durable coordinator. The active review is terminated when a
new head arrives, only the newest pending head runs next, and stalled work is
terminalized as `INCOMPLETE`. The model receives a credential-free, read-only
checkout and one isolated CodeMode analysis tool. Private report and
best-effort transcript copies are stored in the `PRIVATE_REPORTS` R2 bucket;
Flue's private durable agent state also retains the structured model output.
GitHub receives only a check result and a sanitized comment containing the
verdict and reviewed SHA. `BLOCK` and `INCOMPLETE` both fail the check.
### PR safety (`src/pr-safety/`)
An opt-in check (`prSafety.enabled`) that asks whether an untrusted pull
request is an **attack**, not whether its code has bugs. Examples: a
planted vulnerability, exfiltration or phoning home, GitHub Actions abuse,
supply-chain tampering, hidden payloads, or text aimed at AI agents. It runs
on every push to a pull request in a public repository whose author isn't an
`OWNER`, `MEMBER`, or `COLLABORATOR`, and on pull requests from renovate and
dependabot. Other bots are skipped, and release pull requests are left to
release security.
Each head is reviewed in two tiers:
1. **Rule signals** (`signals.ts`): deterministic checks over the merge-base
diff. They cover workflow changes and dangerous workflow constructs,
install scripts, non-registry dependency and lockfile sources,
package-manager and git-hook configuration, Trojan Source and invisible
Unicode, encoded blobs and binaries, network, credential, and
process-execution code, new hosts, removed security logic, and
prompt-injection text. No model is involved.
2. **Reviewer agent** (`skills/pr-safety/`): a model in a credential-free,
read-only checkout (the same CodeMode tools as release security; no shell,
no network). It starts from the signals, measures every hunk against the
pull request's stated purpose, and returns `CLEAN`, `SUSPICIOUS`,
`MALICIOUS`, or `INCOMPLETE`.
Results are private. The full report and signal table go to the
`PRIVATE_REPORTS` R2 bucket under `pr-safety/`. `SUSPICIOUS` and `MALICIOUS`
verdicts are posted to the Discord security channel with the report attached,
and so is an `INCOMPLETE` review that had high-severity signals. On GitHub the
`Factory PR safety` check always concludes `neutral` with the same text, so it
never tells a contributor whether their change was flagged. Nothing in the
checkout is executed.
### Merge risk (`src/merge-risk/`)
Assessment only — it never approves, blocks, or merges anything. Once
Factory's own `Factory Review` check completes on a pull request (triggered
by the configured review label or by requesting the reviewer persona, never
on every push, to keep unreviewed pull requests free), a lightweight agent
reads the pull request's metadata, full changed-file list, check
conclusions, and review states — no repository checkout, no tools — and
scores five categories from `skills/merge-risk/SKILL.md`: alignment, impact
(size, breadth, and sensitive areas), verification, rollback cost, and error
detection. Only trusted workflow code turns those `good`/`ok`/`poor`
verdicts into an overall low/medium/high score, with fixed weights, the same
way a review's verdict is decided from structured findings rather than the
model's own words. One comment per pull request, found and updated by a
marker on every re-assessment rather than piling up new ones.
### Advisory triage (`src/advisory/`)
When someone privately reports a vulnerability (`repository_advisory.reported`),
Factory triages the report and posts the result to a private Discord channel.
GitHub's API can't comment on an advisory, so Discord is where maintainers read
it; nothing is ever said to the reporter automatically.
The bundled `skills/advisory-triage/` skill starts from the assumption that the
report is not a vulnerability and requires proof that it can be exploited in a
realistic application. The agent works in a credential-free triage sandbox with
the repository's default branch installed and built (using the `triage`
section's `installCommand` and `buildCommand`), builds a reproduction, and
checks the repository's other advisories for duplicates and precedent. It
doesn't fix anything; when a change is warranted, it writes a fix brief another
agent can implement, worded as an ordinary bug fix.
In Discord, an announcement is posted when the report arrives and a thread is
started from it. The thread receives the verdict (`vulnerability`,
`not-vulnerability`, `duplicate`, or `needs-information`) with the full
assessment, a draft reply to the reporter, and the fix brief attached, followed
by the reply inline so it can be copied into the advisory. The announcement is
edited to show the verdict. Mentions are disabled on every message. The complete
result is also stored in the `PRIVATE_REPORTS` R2 bucket under `advisories/`.
## Personas
Personas are assignable GitHub identities in front of capabilities. You
assign an issue or pull request to a persona the way you would to a teammate,
and the assignment is the signal to act:
| Persona | Signal | Action |
| ---------- | -------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- |
| `triage` | Issue assigned to it | Runs the triage pipeline, whatever the current label, then unassigns itself. Reassign it to run triage again. |
| `reviewer` | Review requested from it, or pull request assigned to it | Runs the review capability (a label isn't needed), then withdraws the request or assignment. Re-request it to review again. |
| `author` | Same-repository pull request assigned to it | Takes ownership and addresses requested changes and failing checks until unassigned (see [Code author](#code-author-srcauthor)). |
### The review loop
The reviewer and author personas play GitHub's own review loop, and every
hop between them is a visible GitHub state change a maintainer can step
into:
1. Triage opens a fix pull request, assigns the author persona, and requests
the reviewer persona.
2. The reviewer reviews. With no findings above the least severe configured
severity (by default: nothing but `low`), it **approves** and the loop
ends; a human merges. Otherwise it **requests changes**.
3. Requested changes start an author round. The author fixes what it agrees
with and declines what it doesn't, explaining why in the thread. Then it
re-requests the reviewer.
4. The reviewer re-reviews, weighing each declined finding. If it accepts the
author's reasoning it resolves the thread. If every blocking finding left
is one the author declined and the reviewer still stands by, that is a
**stand-still**: the reviewer says so, labels the pull request
`factory: needs human`, and unassigns the author so nothing proceeds.
5. The author's round budget bounds the loop; when it runs out, the author
hands the pull request to a human the same way.
A human "Request changes" review starts an author round too, and failing
checks do as well. GitHub doesn't let an App approve or request changes on
a pull request it opened itself, which covers every triage fix. There the
reviewer's verdict is posted as a comment review that states the verdict,
and the author acts on it just the same. On other pull requests the
verdict is a real GitHub approval or change request, and a change request
may block merging until the reviewer approves or someone dismisses it.
Label-triggered reviews stay plain comments with no verdict.
GitHub App accounts can't be assigned or asked for review, so each persona
is a plain GitHub **user account** with enough repository access to be
assignable: `astro-triage`, `astro-reviewer`, and `astro-author` by default.
The accounts are inert handles: Factory never signs in as them and holds no
credentials for them. Everything is still written by the GitHub App, signed
with the persona's name.
Personas need no configuration. All three are on by default, except that
the reviewer only exists where a `review` section configures what it
reviews with (its skill, model, and vocabulary). A repository can rename a
persona's account, tune the author, or switch a persona off:
```yaml
personas:
triage:
login: my-triage-bot # defaults to astro-triage
reviewer: false # switch a persona off
author:
# login: astro-author
# skill: .agents/skills/author # overrides the bundled default skill
# model: cloudflare-ai-gateway/claude-opus-4-6
# thinkingLevel: medium # minimal, low, medium, high, xhigh, or max (default: high)
# maxRounds: 5
```
A queued run checks that the persona is still assigned (or still requested)
when it starts, so withdrawing the assignment cancels it. Assignments to
anyone who isn't a persona are ignored.
## Repository configuration
Target repositories may add `.github/factory.yml` (all sections optional; no
file at all means triage-on with defaults and review off). Configuration is
always read from maintainer-controlled content.
```yaml
version: 1
adversary:
trigger:
label: ai-adversary
blueTeam:
# skill: .agents/skills/adversary-blue
# model: cloudflare-ai-gateway/claude-opus-4-6
# thinkingLevel: medium
purpleTeam:
# skill: .agents/skills/adversary-purple
# model: cloudflare-ai-gateway/claude-opus-4-6
# thinkingLevel: high
review:
trigger:
label: ai-review
# skill: .agents/skills/astro-review # overrides the bundled default skill
# model: cloudflare-ai-gateway/claude-opus-4-6 # overrides the built-in reviewer model
# thinkingLevel: high # minimal, low, medium, high, xhigh, or max (default: high)
# severity: [critical, high, medium, low]
# areas: [correctness, security, ...]
triage:
# enabled: true
# autoPrOnFix: false
# skill: .agents/skills/triage # overrides the bundled default skill
# prWriterSkill: .agents/skills/pr-writer # adds repository-specific PR guidance
# model: cloudflare-ai-gateway/claude-opus-4-6 # reproduce/diagnose/fix pipeline
# thinkingLevel: medium # default: high
# verificationModel: cloudflare-ai-gateway/claude-haiku-4-5 # classifiers
# verificationThinkingLevel: low # omitted means use the model/provider default
# installCommand: pnpm install --no-frozen-lockfile # [] to install nothing
# buildCommand: pnpm build # one command, a list, or a block scalar
# previewRelease:
# workflow: factory-preview.yml # opt in to preview releases
# check: factory/preview-release # check run the workflow reports to
# checkApp: github-actions # app that must have created that check
# allowedHosts: [pkg.pr.new] # hosts trusted to serve preview packages
# labels:
# inProgress: bot-working
# fixPending: awaiting-confirmation
personas:
author:
# thinkingLevel: medium # default: high
releaseSecurity:
# thinkingLevel: high # release-security reviewer; default: high
advisories:
# enabled: true # triage privately reported advisories (needs Discord; see GitHub App setup)
# skill: .agents/skills/advisory-triage # overrides the bundled default skill
# model: cloudflare-ai-gateway/claude-opus-4-6 # default: triage.model
# thinkingLevel: high # default: high
prSafety:
# enabled: false # review untrusted pull requests for malicious intent (needs Discord)
# model: cloudflare-ai-gateway/claude-opus-4-6
# thinkingLevel: high # default: high
```
Skills resolve as **bundled default, repository override wins**: the factory
ships generic adversary, review, triage, and advisory triage skills; repository overrides live
under `.agents/skills/`. Blue and purple have separate adversary overrides, so
implementation guidance does not leak into judging guidance. A purple override
may add domain-specific criteria but cannot weaken the built-in correctness and safety gate. The
factory's review and triage defaults live in `skills/review/` and `skills/triage/`;
a repository can replace either one by committing a skill under
`.agents/skills/` and pointing the capability's `skill` setting at it.
Triage pull requests use Factory's built-in `Changes`, `Testing`, and `Docs`
format. A repository can add its own PR writing guidance with
`triage.prWriterSkill`; that skill is applied in addition to the built-in
format.
(The bundled entry file is stored as `skill.md` — Flue's vite plugin treats
imports literally named `SKILL.md` as packaged skills, and we need the raw
text; it's seeded into the sandbox as `SKILL.md`.)
## Bootstrapping the checkout
The triage sandbox starts as a plain checkout of the default branch. Two
optional stages run before the agent does, each a list of commands executed in
order from the repository root:
```yaml
triage:
installCommand:
- pnpm install --no-frozen-lockfile
- git clone --depth 1 https://github.com/withastro/compiler.git .compiler || true
buildCommand: pnpm build
```
`installCommand` **defaults to `pnpm install --no-frozen-lockfile`**, because
nearly every repository the factory runs on is a pnpm workspace and an
uninstalled checkout can't reproduce anything. The lockfile is deliberately not
frozen: the agent may add a dependency while building a reproduction, and a run
that dies on a lockfile mismatch has failed for a reason unrelated to the bug. A
repository that isn't a pnpm workspace has to say so — `installCommand: []`
switches the default off.
`buildCommand` is empty by default. A repository whose packages resolve through
built output needs it: in a monorepo where a reproduction project links `astro`
to `packages/astro`, whose `main` points into `dist/`, nothing runs until the
workspace is built, so without it the agent reports "could not reproduce" about
its own unbuilt workspace rather than about the bug.
### Writing commands
All three YAML spellings mean the same thing — one command per line, no `&&`
needed to sequence them:
```yaml
buildCommand: pnpm build # a single command
buildCommand: [pnpm install, pnpm build] # a list
buildCommand: | # a block scalar
pnpm install
pnpm build
```
Each line runs as its own command and the stage stops at the first failure, so
lines have `&&` semantics without the punctuation. The cost is that a single
command can't span lines: write multi-line shell constructs on one line, or as
a script in the repository.
Commands are maintainer-authored content read from the default branch and run in
a sandbox holding no credentials, so they're otherwise unrestricted — they grant
nothing the repository's own CI doesn't already have.
### Failure
A checkout that won't bootstrap is an environment problem, not a triage verdict.
Failure stops the remaining commands and parks the issue in the re-triageable
`triage: failed` state, with the failing command's output in the failure
comment, so fixing the repository and commenting is enough to resume. Failure
messages name the stage and position (`install 2/3`).
Install gets 15 minutes per command and two retries, being the most
network-dependent part of a run; build gets 30 minutes and one retry, which
guards against a flaky container rather than a deterministically failing build.
Commands should avoid modifying tracked files: they run in the same checkout the
agent later commits from, so their edits become part of whatever it pushes as
the fix.
## Models
A model is named as `<provider>/<model>`. Factory bundles only the
`cloudflare-ai-gateway` provider, which can route to multiple upstreams while
ensuring every inference request passes through the shared gateway:
- Workers AI model ids include a routing prefix and their own vendor segments,
for example
`cloudflare-ai-gateway/workers-ai/@cf/moonshotai/kimi-k2.7-code`.
- Anthropic models use their normal model id, for example
`cloudflare-ai-gateway/claude-opus-4-6`. They use the gateway's native
Anthropic endpoint rather than calling Anthropic directly.
The direct Workers AI and Anthropic providers are not bundled, and the Worker
has no `AI` binding, so repository configuration cannot bypass the gateway.
Existing `anthropic/…` and `cloudflare/…` configuration values remain accepted
as migration aliases, but Factory rewrites them to their gateway equivalents
before an agent sees them. Gateway authentication is resolved inside the Flue
runtime from encrypted Worker secrets and is never available to target
repositories or agents. Each request disables prompt and response payload
retention while leaving gateway usage analytics available.
Thinking level is independently configurable for each agent in
`.github/factory.yml`. Supported values are `minimal`, `low`, `medium`, `high`,
`xhigh`, and `max`. The substantive coding/review agents default to `high`,
preserving their existing behavior. `triage.verificationThinkingLevel` is
optional and omitted by default so lightweight classifiers retain the model
provider's default. The release-security reviewer also defaults to `high`.
Model settings are configurable and default to gateway-routed Workers AI
models:
| Setting | Used by | Default |
| ---------------------------- | --------------------------------------- | -------------------- |
| `adversary.blueTeam.model` | independent alternative implementation | `CODE_MODEL` |
| `adversary.purpleTeam.model` | qualification and red/blue comparison | `CODE_MODEL` |
| `review.model` | the pull request reviewer | `CODE_MODEL` |
| `triage.model` | the reproduce/diagnose/fix pipeline | `CODE_MODEL` |
| `triage.verificationModel` | fix verification and retriage decisions | `VERIFICATION_MODEL` |
The matching `thinkingLevel` fields are available for each model-backed agent:
`adversary.blueTeam`, `adversary.purpleTeam`, `review`, `triage`,
`triage.verificationThinkingLevel`, `personas.author`, and `releaseSecurity`.
Defaults live in `src/models.ts`. The verification agents only classify
conversation text and hold no tools, so they do not need a coding model.
Providers are bundled at build time by the `providers` array in
`flue.config.ts`, and `MODEL_PROVIDERS` in `src/models.ts` mirrors it. New
configuration should name `cloudflare-ai-gateway`; the legacy `anthropic` and
`cloudflare` prefixes are syntax aliases only. Every other provider is rejected
when configuration is parsed rather than failing at the first model call
partway through an agent run. `src/ai-gateway.ts` overrides the bundled
provider's auth with Factory-scoped secrets and adds metadata for current
Workers AI models that have not reached Pi's gateway catalog yet.
## Preview releases
A preview release is an installable build of a candidate fix, so the person who
reported the bug can verify it before a maintainer merges anything. It's what
moves an issue into `triage: fix pending` and unlocks the FixVerifier loop;
without one, a fix either opens a PR directly (`autoPrOnFix`) or lands as a
pushed branch + `needs triage`.
Publishing has to happen in the target repository's own CI — pkg.pr.new
authenticates with that repository's Actions OIDC identity, which a Worker
cannot present. So the factory:
1. dispatches a maintainer-owned `workflow_dispatch` workflow, and
2. polls for a check run on the pushed fix branch commit to collect the result.
Copy [`templates/factory-preview.yml`](templates/factory-preview.yml) into the
target repository's `.github/workflows/`, adapt the build and publish steps,
then set `triage.previewRelease.workflow`. The workflow reports back by
creating a check run named `factory/preview-release` on the built commit whose
summary contains a fenced `json` block:
```json
{
"packages": [
{
"name": "astro",
"url": "https://pkg.pr.new/withastro/astro/astro@abc1234"
}
]
}
```
Three deliberate design choices:
- **The dispatch targets the default branch**, not the fix branch, so the
workflow _definition_ is always maintainer-controlled and an agent can never
rewrite the CI that runs its own code. The fix branch travels as an input.
The workflow still builds LLM-authored code, which is why the whole
capability is opt-in per repository.
- **Results are polled, not delivered by `workflow_run` webhooks.** Polling
keeps preview releases inside one durable workflow instance with no
cross-instance event correlation, and `workflow_run` carries no step outputs
anyway. Sleeping between polls is durable and costs no compute. The budget is
30 minutes; every failure mode (unconfigured, undispatchable, failing build,
malformed report, timeout) degrades to "no preview" and never fails triage.
The sandbox is released before the wait starts, so a preview release never
holds a container while the repository's CI runs.
- **The reported result is untrusted input.** The publishing workflow executes
agent-authored build scripts, so it can influence what it reports. The check
run must come from `checkApp`, every URL must be https on an `allowedHosts`
host with no embedded credentials, and one bad entry rejects the whole
payload. The install instructions are then rendered by the factory rather than
by the comment agent, so the comment and the label can't disagree and the URLs
never enter a model prompt.
## GitHub App setup
- **Permissions**: Contents (read/write — also required by GitHub's
`resolveReviewThread` mutation), Issues (read/write), Pull requests
(read/write), Checks (read/write), Actions (read/write — dispatching preview
release workflows), Repository security advisories (read).
- **Events**: Pull request, Check run, Issues, Issue comment. The code author
persona also needs Pull request review, Check suite, and Workflow run.
Advisory triage needs Repository advisory.
- **Webhook URL**: `https://<worker>/channels/github/webhook`.
- **Secrets** (`wrangler secret put` / `.dev.vars`): `GITHUB_APP_ID`,
`GITHUB_APP_PRIVATE_KEY` (PKCS#8 — convert with
`openssl pkcs8 -topk8 -nocrypt`), `GITHUB_WEBHOOK_SECRET`,
`FACTORY_AI_GATEWAY_TOKEN`, `FACTORY_AI_GATEWAY_ACCOUNT_ID`, and
`FACTORY_AI_GATEWAY_ID`. The gateway connection values deliberately use
Factory-scoped names so Wrangler cannot mistake the remote gateway account
or token for Factory's deployment credentials.
Advisory triage posts to Discord as a bot. Create a Discord application with
a private bot, add it to the server, and allow it to view, send messages,
create public threads, send messages in threads, embed links, attach files,
and read message history in the target channel. Set the channel id as the
`DISCORD_SECURITY_CHANNEL_ID` variable in `wrangler.jsonc` and the bot token as
the `DISCORD_BOT_TOKEN` secret. Without both, reported advisories are skipped.
Public and private repositories are both supported. Public repositories get
an anonymous blobless clone (the triage sandbox holds no credentials at all);
private repositories get a full single-branch clone authenticated with a
short-lived contents-read token passed as a one-shot git header — never
persisted to git config — after which the origin remote is removed, so the
agent still runs credential-free.
Before deploying release security, create the private bucket declared in
`wrangler.jsonc`:
```sh
pnpm exec wrangler r2 bucket create astro-release-securitybot-reports
```
Before enabling adversary runs, create its transient artifact bucket:
```sh
pnpm exec wrangler r2 bucket create factory-adversary-artifacts
```
For cutover, deploy Factory while the previous reviewer remains available,
open the smoke PR described above, and confirm the `Astro release security smoke
test` check completes. Then disable the previous reviewer's webhook or workflow
before opening or synchronizing a release PR, so only Factory publishes the
managed check and comment.
## Development
```sh
pnpm install
pnpm dev # local dev (vite + workerd); triage sandboxes need Docker running
pnpm exec biome ci . # formatting and linting
pnpm test # vitest
pnpm check:types # tsc
pnpm deploy # vite build && wrangler deploy
```
`vite build` is mandatory before deploy: the Flue vite plugin compiles each
`'use agent'` module into a Durable Object class and generates the merged
wrangler config. Never hand-author `FLUE_*` bindings; do declare migrations
for generated classes (see `wrangler.jsonc`).
## Roadmap
1. **PR feedback agent** — respond to maintainer reviews with code changes.
2. **Pluggable routing** — the router is a pure `event → dispatch` function
precisely so a markdown-configured LLM router can slot in later.