update

name	update
description	Update and restart everything after a change — installed binaries, the long-running services, the mecha-graph MCP server, the benchmark binary, the factory client, and the droplet. Use when asked to "update everything", "deploy", "restart the services", after cutting a release, or whenever something running looks older than the repo.

Updating everything

There is no single command, and that is the whole problem. "Update" names six independent surfaces on two machines from three repositories, each with its own version line. Every one of them can be stale while the others are current, and none of them complain.

The failure this skill exists to prevent, from 2026-08-15: mecha 0.1.4 was tagged, pushed, and published to crates.io, and the services were restarted — and mecha --version still said 0.1.3 afterwards, because nothing had run cargo install. Four surfaces were stale at once that day: the installed binaries, mecha-triggers (left out of the restart), the benchmark's musl binary (eight days old, so a benchmark would have measured old code and reported it as new), and the factory client.

The one rule

Verify the running thing, never the repo.

A tag proves a commit exists. A green CI run proves it compiles. git log proves someone wrote it. None of them prove that the bytes executing on this machine contain it. Ask the process what it is, not the source tree: mecha --version, ls -l ~/.cargo/bin/, /proc/<pid>/exe.

Three specific confusions worth naming, because each one has actually happened:

• A release is not an install. Tagging and publishing changes crates.io, not ~/.cargo/bin.
• A restart is not a reinstall. systemctl restart re-executes the same file on disk. Restarting before installing accomplishes nothing.
• A debug build is not the one that runs. Both mecha's MCP config and the benchmark point at release paths.

The six surfaces

Work them in this order — later ones depend on earlier ones.

1. Installed binaries (~/.cargo/bin)

From ~/Github/mecha:

cargo install --path mecha-cli  --locked --force   # mecha
cargo install --path mecha-mail --locked --force   # mecha-mail, mecha-google, mecha-outlook

And the graph, from ~/Github/personalized_knowledge_graph — installed with mecha since 2026-08-16, because ~/.mecha/config.toml runs ~/.cargo/bin/mecha-graph-mcp, not a repo path.

That path is right, and it is not the mecha-graph checkout beside it. The project is called mecha-graph and its crates are mecha-graph*, so ~/Github/mecha-graph looks like the obvious source and is not one: it is a generated artifact, git archive HEAD from the private repo minus a hardcoded exclusion list, run through check-public-denylist.sh — a gate that deletes the tree it refuses rather than flagging it. The two histories are disjoint. Building from the public mirror would drop eval/gold.jsonl, the operational docs and the export tooling, and would put authoring on the far side of the gate that keeps life-derived text out of a public repo. Develop in the private repo; export to publish. Confirmed 2026-08-22, after this parenthetical's earlier wording ("the mecha-graph repo") sent a session looking for which of the two was authoritative.

cargo install --path mecha-graph     --locked --force   # mecha-graph (CLI)
cargo install --path mecha-graph-mcp --locked --force   # mecha-graph-mcp (MCP server)

Both lines, every time — and the second one is the one that matters at runtime. They are separate crates that both link mecha-graph-core, so installing mecha-graph alone leaves the MCP server running whatever library code it was last built against, silently. Found 2026-08-24: a session developing in the graph repo had been running cargo install --path mecha-graph all evening and left mecha-graph-mcp an hour stale — across an arc whose central fix was in fact::normalize_predicate, which had been auto-registering any predicate it could not alias (49 of 83 predicates arrived unreviewed that way). The CLI stopped doing it; the MCP server, which reaches that code through kg_upsert → assert_fact, would have gone on minting predicate fragments through the one write path nobody watches. A stale MCP binary is worse than a stale CLI, because everything mecha does at runtime goes through it and nothing about it announces its version.

Verify: mecha --version matches mecha's workspace Cargo.toml, and the graph server answers from the installed path:

echo '{"jsonrpc":"2.0","id":1,"method":"tools/list","params":{}}' \
  | ~/.cargo/bin/mecha-graph-mcp | python3 -c 'import json,sys; print(len(json.load(sys.stdin)["result"]["tools"]), "tools")'

1b. The web app's assets

mecha serve serves the UI from [web] assets in the live config, which points at ~/.mecha/web/dist — a stable home, deliberately not a repo or worktree path (production once served from a Claude session's worktree, which is the deferred-failure shape: fine until the session directory is cleaned). The dist is a build artifact, not in git, so a binary install does not update it:

cd <clean worktree>/web && npm ci && npm run build
rsync -a --delete dist/ ~/.mecha/web/dist/

Before that rsync, ask what is deployed — the dist may be another lane's live test. git tag -l deployed-local in the main checkout names the commit whose build is on the box when a session deployed something other than main (re-point it whenever you deploy; delete it when main is deployed). And read the mismatch correctly even when the tag is missing: a served bundle hash that matches no tree you can see is far more likely to be another session's deployment than a leftover — on 2026-08-29 it was exactly that, filed as "stale", and the rsync that followed silently reverted a surface the owner was live-testing. If the dist is not what you are about to install, announce to the live sessions before replacing it.

Then restart mecha-serve.service (step 2). Verify the served page, not the directory: the 8443 door returning 200 with the new bundle hash.

When a header probe echoes a value back (If-Modified-Since from a Last-Modified you just grepped), strip carriage returns first — | tr -d '\r' — or the CR rides into the outgoing header and the server answers 400, which reads exactly like a server bug and is your curl. Two sessions hit it independently within an hour on 2026-08-29.

2. The long-running services

These hold an open file handle on the old binary and must be restarted after step 1:

systemctl --user restart mecha-slack.service mecha-triggers.service \
                         mecha-drain.service mecha-serve.service \
                         mecha-voice-worker.service

mecha-serve and mecha-voice-worker were missing from this list until 2026-08-25, which is worth naming because the omission is the shape this skill exists to catch. mecha-serve is three surfaces in one process — the web app, the chat agent, and the mounted voice facade — so it holds a handle on the new mecha binary and serves [web] assets; step 1b mentions restarting it, but only in the branch a session reaches when it changed the assets, so a pure-Rust change would have installed a binary nothing re-executed. mecha-voice-worker is the other direction and easier to miss: it runs scripts/voice/worker.py from the repo working tree, so it is the one unit here that goes stale on a change that never touched Rust at all and never appears in cargo install output.

mecha-parakeet is deliberately not in the list. It runs scripts/voice/parakeet_server.py, so restart it when that file changes — and only then, because coming back costs a model load and voice is deaf until it finishes.

The timer-driven units need nothing. mecha-frontdoor, mecha-ruminate and mecha-slots are .timer-fired and exec fresh on each firing, so they pick up a new binary by themselves. Knowing which list a unit is on is the difference between a restart that matters and cargo-culting six of them.

Verify — and take a startup line, never is-active, since a unit that crashes on its first request is active for a while first. journalctl --user -u mecha-slack.service --since "2 minutes ago" should show the reconnect line ("Connected to … N owner(s)"), mecha-triggers should log "N trigger(s), N enabled · ticking every minute", mecha-serve should print both its doors ("voice facade on http://127.0.0.1:8990" and "mecha serve on http://127.0.0.1:63242"), and mecha-voice-worker should reach "Uvicorn running on http://127.0.0.1:7860". A service that comes back but logs nothing is not evidence of success.

3. The graph MCP server — folded into step 1 since 2026-08-16

~/.mecha/config.toml runs ~/.cargo/bin/mecha-graph-mcp (server alias graph, prefix_tools = false, tools are bare kg_*), so the graph updates through the same cargo install ritual as everything else — step 1 covers it, and step 2's restart is what hands the running Slack connector the new server (its MCP children are spawned at connector start). What this surface still owns: a repo build is no longer an install here either — a cargo build --release in the graph repo changes nothing mecha can see, which is the same trap as mecha's own binaries wearing a different repo. And there is a seventh binary, which no cargo install reaches. This paragraph used to say the graph repo's own nightly "builds and runs from its repo tree and is not mecha's concern". It does not build, and it is: scripts/nightly.sh sets PKG="$REPO_DIR/target/release/mecha-graph" and executes whatever is sitting there. Its link --auto step runs the kNN linker, straight into the owner's graph, at 01:30.

Found 2026-08-26, hours after a session repaired a linker bug out of the live graph — 30 placeholder nodes merged, 121 payloads rewritten, 23 accepted facts re-pointed. That binary was dated Aug 25. Left alone, the nightly would have re-run the old linker and re-staged the same damage while every version string on the machine read current. So:

cd ~/Github/personalized_knowledge_graph && cargo build --release
ls -l target/release/mecha-graph     # must be today

Do not assume cargo install --path refreshed it — where cargo puts an install's intermediate artifacts is not a promise about that path, so check the date rather than reason about it. This is the skill's own thesis failing inside the skill's own text: verify the running thing, never the repo — and a cron job's binary is a running thing that answers to no --version anybody types.

mecha itself needs no change when graph tools change — it discovers tools via tools/list.

4. The benchmark binary

bench/run.sh uses target-musl/release/mecha, a static build — never the installed one, because the glibc build will not start in most task containers. It is rebuilt by bench/build-portable.sh (which bench/run.sh calls), but check its date before trusting a scorecard: a stale one measures old code and labels the result with today's model.

ls -l target-musl/release/mecha && target-musl/release/mecha --version

5. The factory client (different repository, different version line)

factory-publish comes from ~/Github/mecha-factory, versioned 0.2.x independently of mecha's 0.1.x. mecha-drain-follow is a hand-installed bash wrapper around it and is not in that repo, so it needs nothing.

Fetch before installing. On 2026-08-15 the local checkout was 16 commits behind origin and still said 0.2.1 while the installed binary was 0.2.2 — installing from that tree would have been a silent downgrade of a running service.

cd ~/Github/mecha-factory && git fetch origin --tags && git status -sb
factory-publish --version          # compare against the newest tag

6. The factory server (the DigitalOcean droplet)

Not this machine, and not covered by anything above. The server runs on a DigitalOcean droplet in NYC1 (ubuntu-s-1vcpu-1gb-nyc1, a 1 vCPU / 1 GB box) behind mecha-factory.ai; everything local is a client of it. Note the apex domain resolves to Squarespace, not the droplet — the origin you want is gate.mecha-factory.ai (compute. is the same host).

spark reaches it as root, using a dedicated key generated here for this purpose (~/.ssh/mecha_factory_deploy, commented with the box's hostname). No ~/.ssh/config entry exists, so the key has to be named explicitly:

# Read-only: what is actually deployed right now
ssh -i ~/.ssh/mecha_factory_deploy [e-mail] \
    'factory --version; systemctl is-active mecha-factory.service'

# Deploy a released version
ssh -i ~/.ssh/mecha_factory_deploy [e-mail] \
    'factory-deploy vX.Y.Z'      # download, checksum, prove binary+config, swap, health-check
ssh -i ~/.ssh/mecha_factory_deploy [e-mail] \
    'factory-deploy --rollback'  # reinstall /usr/local/bin/factory.prev

The unit is mecha-factory.service; the binary is /usr/local/bin/factory, with the previous one kept beside it as factory.prev, which is what --rollback restores. Adding a Host factory block to ~/.ssh/config would make all of this shorter and is worth doing.

Check before assuming it is behind. On 2026-08-15 the droplet was already serving factory 0.2.4 — the newest release — while the local client was two releases back at 0.2.2. The staleness ran the opposite direction from the guess, and one read-only factory --version settled it in seconds.

This is production, on a 1 GB box, serving people who are not the owner. The read-only check above is always fine. Ask before deploying, and never infer that "update everything" included the droplet — it is a different machine from a different repository on a different version line.

Ordering constraints

• Install before restarting. Reversing them restarts the old binary and looks like success.

• Install before editing ~/.mecha/config.toml, and restart anything long-lived after. A key a binary does not know is a fatal parse error — ProviderConfig and friends carry deny_unknown_fields, deliberately, so a typo'd setting fails loudly rather than being silently ignored. The cost is that a new key breaks every older binary, including one already running.

  That failure is deferred and partial, which is what makes it expensive. A long-lived process parsed the config at startup, before the edit, so it keeps working — until it hits a path that re-reads config, which then fails in a subsystem with no apparent connection to what changed. Measured 2026-08-21: a TUI session started at 01:20 kept running after the 03:07 install (/proc/<pid>/exe showed mecha (deleted) — a process executes the inode it was launched from, so an install is invisible to it), and the symptom was show_file reporting a config parse error two hours later, which sent the model reading config.toml to investigate a version skew.

  /proc/<pid>/exe ending in (deleted) is the check, and it is the one that catches a stale process where mecha --version catches a stale file:

  Walk exe, never argv. A process launched off $PATH has argv mecha tui, so any pgrep for the install path misses precisely the interactive session most likely to be stale — which the first version of this check did:

  for p in $(ls /proc | grep -E '^[0-9]+$'); do
    t=$(readlink /proc/$p/exe 2>/dev/null) || continue
    case "$t" in
      "$HOME"/.cargo/bin/*"(deleted)")  ;;
    
  
  

7. The sandbox image

~/.mecha/config.toml's [sandbox] runs confined shell in the mecha-sandbox docker image (built from scripts/sandbox.Dockerfile). It bakes in its own Rust toolchain, so it goes stale independently of everything above — after a host toolchain bump, rebuild it or confined cargo quietly diverges from the cargo that CI and the host run:

docker build -t mecha-sandbox -f scripts/sandbox.Dockerfile .
docker run --rm mecha-sandbox bash -lc 'cargo --version'   # compare to host

(bwrap would be the cheaper backend but is blocked on this host: kernel.apparmor_restrict_unprivileged_userns=1. The config comment above [sandbox] records the targeted fix if that's ever wanted.)

What no amount of this reaches

~/.mecha/ is not a git repository. config.toml and triggers/*.toml live on one disk and are in no repo's history, so nothing above updates, restores, or reviews them. When a change lands there — a trigger's tool allowlist, a provider block — say so explicitly in the handoff, because the next clone will not have it.

Related coupled values that no build step checks, each silent when wrong: a llama-server -c against context_window in config.toml, a --alias against that provider's model, and temperature — which mecha sends, so the config overrides the server's --temp.