weft-cloud-install

# Putting weft on GCP A cloud install has no machine of its own. weft's parts and programs' workers are Cloud Run services that scale to zero. Triggers that keep a connection open run on holders, which weft starts only while at least one such trigger is on. Builds run on Cloud Build, timers on Cloud Tasks, and infrastructure on Compute Engine machines. Everything weft keeps lives in a Postgres database the user brings, by its address. So an install with no trigger that keeps a connection open and no infrastructure up runs nothing between calls and, within Google's and the database's free tiers, costs next to nothing: what it does pay for is storing weft's own images. It is made by the "install on GCP" workflow in the user's own fork of weft. The user owns the GCP project, the billing, the database and the fork; you do the work, through their `gh` and `gcloud`. ## First: the two CLIs, logged in Everything after this step you run yourself, so this step comes first and you do not move on until both work. Check before asking anything: `gh auth status` and `gcloud auth list` (plus `gcloud config get project`). Whatever already works, you skip. - **GitHub.** No account: send them to `https://github.com/signup` and wait. No `gh`: install it for their system (the instructions are at `https://cli.github.com`). Then they run `gh auth login` themselves (it opens a browser; GitHub.com, HTTPS, log in with a web browser), since only they can sign in. Check with `gh auth status`. - **Google Cloud.** No account: send them to `https://console.cloud.google.com`, where they sign in with a Google account, accept the terms and add a billing account (a card is required even for the free tier). No `gcloud`: install it for their system (`https://cloud.google.com/sdk/docs/install`). Then they run `gcloud auth login`. Check with `gcloud auth list`. The only things the user does by hand are what needs them in a browser: signing up, the card, and the two logins. Say each step in plain words and wait for them to say it is done. ## Then you do the install The steps are in the cloud guide (`https://weavemindai.github.io/weft/running/cloud.html`, section "Install weft on GCP"). Run them with the user's CLIs, telling them what you are about to do before anything that creates or costs something: 1. **The project.** A new project made just for weft (weft's own accounts get broad rights inside it, so nothing else should live there): `gcloud projects create <id>`, then link billing (`gcloud billing accounts list`, `gcloud billing projects link <id> --billing-account <account>`). If they already made one for weft, use it. A billing account links only a few projects (five on a new one): if the link fails with `Cloud billing quota exceeded`, offer to reuse a project made for weft before, or to unlink one they no longer use (`gcloud billing projects unlink <id>`), and let them choose. 2. **The region.** If the user is on the free tier and has no reason to be anywhere in particular, the install goes in `us-west1` (zone `us-west1-b`) and its database in Neon's `aws-us-west-2`: both are in Oregon, so every call the install makes to its database stays short. Google's always-free machine and 5 GB of storage exist only in `us-west1`, `us-central1` and `us-east1`, and Neon runs on AWS and Azure, not Google, so that pair is the one place both free tiers sit side by side. If they need another region, take theirs, and the Neon region nearest it. 3. **The fork.** `gh repo fork WeaveMindAI/weft --clone=false`. A new fork has its workflows turned off: turn them on with `gh api -X PUT repos/<fork>/actions/permissions -F enabled=true`. 4. **The `gcloud` block** below, as written, with the three values on its first line filled in (`FORK` is the fork's `owner/repo`, with GitHub's capitals, because Google compares it letter for letter); never invent a flag. Its IAM steps retry on their own while IAM catches up on a brand new project, so a few "waiting for IAM" lines are normal. ```bash PROJECT=my-project REGION=us-west1 FORK=me/weft NUMBER=$(gcloud projects describe $PROJECT --format='value(projectNumber)') # On a new project IAM can take a minute to catch up (with the services # just enabled, with the account just made), so each IAM step below is # tried again, quietly, for up to three minutes; a last try shows its # own error. retry() { for _ in $(seq 18); do "$@" >/dev/null 2>&1 && return 0 echo "waiting for IAM to catch up, trying again in 10 seconds" sleep 10 done "$@" } gcloud services enable --project $PROJECT \ iam.googleapis.com iamcredentials.googleapis.com sts.googleapis.com \ cloudresourcemanager.googleapis.com serviceusage.googleapis.com gcloud storage buckets create gs://$PROJECT-weft-state --project $PROJECT \ --location $REGION --uniform-bucket-level-access gcloud iam service-accounts create weft-installer --project $PROJECT retry gcloud projects add-iam-policy-binding $PROJECT --role roles/owner \ --member serviceAccount:weft-installer@$PROJECT.iam.gserviceaccount.com pool() { gcloud iam workload-identity-pools describe weft-install --project $PROJECT \ --location global >/dev/null 2>&1 \ || gcloud iam workload-identity-pools create weft-install --project $PROJECT \ --location global } provider() { gcloud iam workload-identity-pools providers describe github --project $PROJECT \ --location global --workload-identity-pool weft-install >/dev/null 2>&1 \ || gcloud iam workload-identity-pools providers create-oidc github \ --project $PROJECT --location global --workload-identity-pool weft-install \ --issuer-uri https://token.actions.githubusercontent.com \ --attribute-mapping google.subject=assertion.sub,attribute.repository=assertion.repository \ --attribute-condition "assertion.repository=='$FORK'" } retry pool retry provider retry gcloud iam service-accounts add-iam-policy-binding \ weft-installer@$PROJECT.iam.gserviceaccount.com --project $PROJECT \ --role roles/iam.workloadIdentityUser \ --member principalSet://iam.googleapis.com/projects/$NUMBER/locations/global/workloadIdentityPools/weft-install/attribute.repository/$FORK ``` 5. **The fork's variables**, with `gh variable set <NAME> --repo <fork> --body <value>`: `GCP_PROJECT_ID`, `GCP_REGION`, `GCP_ZONE`, `TF_STATE_BUCKET`, `GCP_WORKLOAD_IDENTITY_PROVIDER` and `GCP_INSTALL_SERVICE_ACCOUNT`. A project's frontend is added later, from the project (read weft-deploying), never here. 6. **The database.** If the user has no Postgres, read weft-database, which also says when to offer the install a Postgres of its own on Cloud SQL instead. If they already have a Postgres they want, put its address in the `WEFT_DATABASE_URL` secret with `gh secret set WEFT_DATABASE_URL --repo <fork>` (piping it in, never echoing it into the chat), and when that address goes through a pooler, a direct address of the same database in `WEFT_DATABASE_LISTEN_URL`. 7. **The workflow.** `gh workflow run "install on GCP" --repo <fork>`, then follow it with `gh run watch` in the background. It runs for a while. Its last step prints the install's address (a Cloud Run address, `https://weft-role-dispatcher-<number>.<region>.run.app`, working at once) and the line that logs in with the first operator key, which you read with `gh run view <id> --log` (plain `gh run view` leaves it out). Run that line from the project, after `weft target add` (read weft-deploying): it reads the key from Secret Manager straight into `weft login --key-stdin`, so the key never passes through the chat or a file. Never print the key itself. The install emails when weft's own work is stuck (what `weft status --on prod` lists under `waiting on weft:`). It writes to the fork's repository secret `WEFT_ALERT_EMAIL` if there is one, and otherwise to the first person, by address, holding the Owner role directly on the project. If the user wants the emails somewhere else, set that secret before running the workflow (`gh secret set WEFT_ALERT_EMAIL --repo <fork> --body <address>`). If the workflow stops in "create the cloud" with `no email to send the alerts to was found`, neither exists: set the secret, or, if the user wants no emails, set `alerts_enabled` to `false` in `deploy/terraform/gcp/variables.tf`, then run the workflow again. `alert_after_minutes` in the same file is how many minutes weft keeps retrying before it emails (5 by default). For the rest (what triggers an email, how often, what it costs), go and read the cloud guide's section "What you get". The install needs no domain: its own address is HTTPS and free. A domain costs money, so it is never part of the install; read weft-deploying when the user asks for one. Upgrading weft on the cloud is `weft maintenance on --on prod` (every trigger that is on taken down and the infra that runs stopped, so the old weft and the new one never run side by side; the workflow refuses to replace anything while something is on), merging upstream into the fork (`gh repo sync <fork>`), running the workflow again, rebuilding the user's CLI from that commit (`./setup.sh --cli`; an older CLI cannot read what `off` answers), then `weft maintenance off --on prod` (those projects built again on the new weft, that infra started on that build, then those triggers back). If something in a project does not come back, `off` says what and why, brings everything else back, and still ends the maintenance. If the message says the project's copy of the standard nodes is older than this weft, run `weft catalog update` in its folder once the user's checkout and CLI are on the new commit, since it copies from that checkout (it deletes any edits the user made under `nodes/base_catalog/`, so check with them first); otherwise fix what the error says. Then `weft resync --on prod` in that folder brings back what the upgrade took down. Each program on it with infrastructure then needs `weft infra upgrade --on prod`, which you run yourself: the new weft builds each piece's image anew, and `weft status --on prod` marks each infra node `behind the source` until it is done. A program's deploy workflow needs nothing: it asks the install which weft it runs on every run. If weft's own database is the install's own on Cloud SQL, its size is the fork's repository variable `WEFT_CLOUD_SQL_TIER` (`db-g1-small`, a small shared machine, when unset; a `db-custom-<cpus>-<MiB>` one for a busier program). Editing `deploy/terraform/gcp/variables.tf` does nothing for it, because the install workflow always passes the variable. Its companion `WEFT_CLOUD_SQL_MAX_CONNECTIONS` is optional: unset, Cloud SQL's own number for the tier applies (50 on `db-g1-small`); once set, it stays at that number whatever the tier, so raise it together with the tier. Two more variables cap how many copies of weft's dispatcher and broker Cloud Run runs at once: `WEFT_DISPATCHER_MAX_COPIES` (2 when unset) and `WEFT_BROKER_MAX_COPIES` (4 when unset). Every copy takes an equal share of the database's connections, so more copies means a smaller share each. If the user wants more copies running at once, give the database more connections first (a bigger tier), then raise a ceiling. If a copy refuses to start because its share is too small, raise the tier or lower a ceiling. On a database the user brought, more connections means a bigger plan at their provider. For how the share is worked out and the least each copy needs, go and read the cloud guide's section "How many connections the install takes". Changing the tier, or setting or changing `WEFT_CLOUD_SQL_MAX_CONNECTIONS`, restarts weft's database once, and the workflow replaces nothing while anything on the install is on, so ask the user first, saying both. Then `weft maintenance on --on prod`, `gh variable set <name> --body <value> --repo <fork>` for each variable you change, the install workflow run again, and `weft maintenance off --on prod`. Repository variables live in the fork's GitHub settings, so updating the fork keeps them. The rest of the sizing (how many triggers one holder takes, a holder's CPU and memory, the machine builds compile on) is a default in `deploy/terraform/gcp/variables.tf` in the fork: change it there, commit, and run the workflow again. The build machine defaults to the one Cloud Build's free minutes cover; a bigger one (`build_machine = "E2_HIGHCPU_8"`) compiles a program's image faster and is paid by the minute, once per version of the program, never per worker. Removing weft from GCP is deleting the project made for it, since weft and everything it started live there and nowhere else: `gcloud billing projects unlink <id>`, then `gcloud projects delete <id>` (Google keeps it 30 days, `gcloud projects undelete <id>` brings it back). It deletes everything in it, so say so and wait for the user's yes. The database is theirs, at its own provider, and stays. ## Moving an app from this machine When the user built an app here and now runs it on the cloud install, you never ask them to make its keys or sign-ins again, or to upload its files: you copy them. From the project's folder, once its target is added and logged in (read weft-deploying): ```bash weft migrate connections --to prod --all --dry-run weft migrate connections --to prod --all weft migrate files --to prod --prefix project/<project id>/ ``` `--to` names a target of the project; what is copied comes from the install the command acts on (this machine's, unless `--on <other>`). A selection is required: `--all`, or any mix of `--id <id>` (repeatable), `--since` and `--until` (a date like `2026-10-01`, or an RFC 3339 time), `--service <name>` or `--project <id>` for connections, and `--prefix <key prefix>` for files. `--dry-run` lists what would be copied, and `--json` gives the same as data. Each line says copied, already there, or skipped and why. Secrets travel sealed to the cloud install and keep their ids. Which connection each node uses is picked per install, so the command ends with `weft connect --node <node> --grant <id> --on prod` lines: run those yourself, in the project folder. A connection that spends the install's own shared key copies only when the cloud install holds that key; for one that is skipped, the user connects it there themselves. A run's files stay behind unless they were kept. A file keeps its key, version and lifetime, so a value that names it still finds it on the cloud. For deploying projects to the install once the key is in `weft login`, read weft-deploying.