orchestrate

name

orchestrate

description

Babysit a sibling Claude Code session in another tmux pane through a long-running plan. On every idle, ask the session if /clear is useful; if yes, sibling Writes its self-contained next-prompt body to /tmp/orchestrate-next.txt, orchestrator runs /clear and tells sibling to Read+execute that file (no paste-buffer). Halt on design questions or unexpected deviations. On every /orchestrate invocation it FIRST reads the handoff doc /srv/grappa/.orchestrate/orchestrator-resume.md (the persistent brain) then reconciles against the per-pane daemon state — so /orchestrate alone resumes with zero extra instruction; user can /clear freely to save tokens.

Orchestrate

Drive a sibling Claude Code session in another tmux pane through a long-running plan with hands-off context refresh. The user /clears the orchestrator freely to save tokens; the per-pane state file on /tmp survives /clear so orchestration resumes automatically.

Why /clear, not /compact

Earlier versions of this skill used /compact <prompt-body>. Switched to /clear because:

• The sibling's prompt bodies (the "first action after clear" paragraphs) are exhaustive — file paths, commit SHAs, full state, ordered next steps. The auto-summary /compact adds is mostly redundant.
• /compact keeps the entire prior conversation as a summary on top of the prompt body. Tokens add up across many sub-tasks.
• /clear wipes everything → sibling re-loads CLAUDE.md + active CP + plan from scratch, then acts on the prompt body. Lighter, cleaner restarts.

Tradeoff: no auto-summary safety net. The prompt body MUST be fully self-contained (file paths, commit SHAs, exact next-step). Tell the sibling that explicitly when asking for the prompt.

Architecture (v3 — daemon + log + persistent Monitor)

v1 used a single-shot wait-for-event chain: orchestrator armed one bg-bash, harness fired a notification when it exited, orchestrator re-armed. This was brittle: forgetting to re-arm = silent stall (happened twice in the visitor-parity cluster). 60s tick missed fast clear-ask replies. Permission prompts and design pickers all looked like "IDLE" so orchestrator tried to clear sibling mid-prompt.

v2 kept the one-shot waiter but made the daemon durable, so a missed re-arm only delayed events instead of losing them. That was not enough, and v3 exists because it failed in production.

🔴 Why v3: you cannot notice silence

On 2026-08-02 the orchestrator armed a wait-for-event.sh waiter and a CI poller in the same assistant message. The harness reaps both, so pgrep showed no waiter at all. The daemon kept writing events that nobody read. Meanwhile both workers were halted asking the orchestrator questions — w2 for ~60 minutes, w1 for ~30 — and the orchestrator kept merging PRs and reporting them as "building". vjt had to point it out.

The lesson is structural, not a scolding: an absent listener and a calm worker produce exactly the same observable — nothing. Any design that needs a human-in-the-loop re-arm on every event will eventually skip one, and the skip is invisible by construction.

v3 removes the loop. One Monitor armed once per session streams every event forever. There is no re-arm, so there is nothing to forget.

v3 separates concerns:

• lib/monitor-stream.sh <PANE> [<PANE>...] — the event feed. tail -n0 -F on each pane's daemon log, filtered to the actionable events, each line prefixed with the pane's tmux title. Never exits. Arm it ONCE via the Monitor tool with persistent: true; every subsequent event arrives as its own notification with no action from you. Handles N panes in ONE monitor — one stream, not one per worker. It only tails what the daemon writes, so a dead daemon is still silent: check daemon.sh status on resume.

• lib/daemon.sh start|stop|status|log <PANE> — long-running detached ticker (forked via nohup … & + disown; macOS has no setsid). Calls wakeup-tick.sh every 5s (was 20s, was 60s) and appends events to /tmp/orchestrate-events-<pane>.log. Single-instance per pane via pid file at /tmp/orchestrate-daemon-<pane>.pid. Survives orchestrator /clear, /exit, harness restarts. The orchestrator can't break the chain by forgetting to re-arm anything.

• lib/wakeup-tick.sh <PANE> — the one-shot pane sample. Reads pane via tmux capture-pane, classifies state, emits zero-or-more event lines. State persisted at /tmp/orchestrate-state-<pane>.json for transition diffs across ticks.

• lib/wait-for-event.sh <PANE> — ⚠️ LEGACY (v2), superseded by the Monitor above. Do not use it as the primary listener. Cursor-tracking one-shot log tailer: reads the byte offset from /tmp/orchestrate-cursor-<pane>, waits until the log grows past it, dumps the new events, advances the cursor, exits. Still useful for a deliberate one-off drain ("show me what I missed") — but as a steady-state listener it is exactly the re-arm treadmill that blinded the orchestrator on 2026-08-02. 🔴 Never arm it in the same assistant message as another background command: the harness reaps both and you are left with no listener and no error.

• lib/state.sh <PANE> — query current state without consuming events. Use when orchestrator wakes via user message and needs ground truth.

• lib/resume-check.sh <PANE> — returns FRESH | STALE age=Ns | RESUMING age=Ns daemon=running|stopped.

Event vocabulary (v2 expanded)

Event	Meaning
BOOT state=<idle|busy|prompt|picker> ctx=NN%	First tick after FRESH/STALE
IDLE ctx=NN%	busy → idle (real idle, no prompt/picker pending)
BUSY ctx=NN%	idle → busy
PROMPT-PENDING ctx=NN%	Sibling on a permission/dialog prompt (Do you want to proceed? + 1. Yes) — DON'T act
PROMPT-CLEARED ctx=NN%	User clicked through the prompt — sibling unblocked
PICKER ctx=NN%	Sibling popped a design-Q picker (↑/↓ to navigate, Tab/Arrow keys) — HALT, ping vjt
PICKER-CLEARED ctx=NN%	Picker resolved
USER-TYPED ctx=NN%	vjt typed in pane directly (md5-deduped) — observe only
CTX-BUMP NN% state=<...>	Entered new ≥10%-bucket at ≥30%
CTX-CRITICAL NN% state=<...>	Entered ≥80% — last-chance clear before auto-compact
STALL state=<...> ctx=NN% duration=Ns	Same state ≥300s, possible deadlock — investigate
HEARTBEAT state=<...> ctx=NN%	No event in ≥600s (was 1800) — keepalive
PANE-MISSING	tmux pane gone (2 consecutive misses) — daemon exits

SAME events are swallowed by the daemon, never written to log.

State file fields

/tmp/orchestrate-state-<pane>.json (key=value, not real JSON):

• state — idle | busy | prompt | picker
• ctx — NN or TBD
• bucket — NN (10s)
• prompt_active — 0|1
• picker_active — 0|1
• last_user_typed_hash — md5 of last ❯ <text> line (USER-TYPED dedup)
• last_emit — unix ts of last emitted event
• last_state_change — unix ts of last state transition (STALL gate)

⚠️ One handoff file PER WORKER — never the shared path

This skill was written for ONE sibling, so it says /tmp/orchestrate-next.txt throughout. With two or more workers on the same host that single path is a silent clobber: w2 stages its body, w1 stages its own minutes later, and whichever clears second reads the other's prompt — resuming the wrong branch with total confidence. Caught 2026-07-29 with both grappa workers live on voyager (w2's 00:47 file still sitting there while w1 was being asked to stage its own).

Use /tmp/orchestrate-next-<worker>.txt (-w1, -w2, …) whenever more than one worker exists, and say the exact path in BOTH the clear-ask and the post-clear directive. Read every path in this document as that per-worker form. Check the file's mtime before dispatching it — a stale file from an earlier run looks identical to a fresh one, and re-dispatching yesterday's prompt is worse than not clearing at all.

Setup

Step 0 — read the handoff doc FIRST (always, before anything else)

On EVERY /orchestrate invocation the FIRST action — before tmux, before resume-check, before any tool — is:

Read /srv/grappa/.orchestrate/orchestrator-resume.md

(DURABLE path — survives host reboot, unlike /tmp. The per-pane daemon state files stay in /tmp — they're regenerable per-run; only the handoff brain must be durable.)

The handoff is the orchestrator's persistent brain across /clear. It holds ONLY THIS-RUN STATE: the active issue pack, what's shipped/queued, any pending decision or open halt, and an ## IMMEDIATE NEXT STEP line — plus per-RUN config the user set (autopilot scope, clear-cycle relaxation). PERMANENT rules that apply to EVERY run live in this SKILL (see "Permanent rules" below), NOT the handoff. Reading the handoff top-to-bottom means /orchestrate alone fully restores context — the user should never have to say "read the handoff and resume." If absent, first-ever run — skip to Step 1.

Keeping it current is the orchestrator's job, not optional. Update the handoff at every ship, dispatch, halt, design decision, and run-config change — it is the ONLY thing that survives the orchestrator's own /clear (manual OR the auto-clearer). A stale handoff is the highest-severity bug. Resolve panes BY TITLE, never hardcode %NN (ids are ephemeral): sibling = "grappa-worker", orchestrator = "grappa-orch", ircbot = "vjt-claude".

THE HANDOFF IS BOUNDED — PRUNE DONE WORK, DO NOT APPEND (vjt direct order 2026-07-15). The handoff is a LIVE-STATE snapshot, NOT a log. It must not grow unbounded. Every update is DELETE-then-write, never append-only:

• The instant an issue is shipped + closed (gh issue close done, status:* label removed), DELETE its block from the handoff entirely. The only residue a closed issue may leave is a fact still load-bearing for LIVE work — e.g. the new PROD SHA it produced, or "shipped X, so held branch Y must rebase past it." One line, in the PROD/held section — not its own block.
• DELETE resolved narrative on sight: past dispatch blow-by-blow, superseded plans, "[HISTORICAL]" / "RESUMED + RECONCILED" / "MORNING BRIEFING" / prior-window sections, old timestamped LIVE-NOW blocks. Once the event is over and left no live consequence, it is git/DESIGN_NOTES territory, not handoff territory. The decision log (DESIGN_NOTES) and closed GitHub issues ARE the permanent record — the handoff never duplicates them.
• Held (merge-ready, not-yet-shipped) work stays, but COMPRESSED: SHA + deploy-class + device-verify-or-not + any batch-merge gotcha (e.g. two branches touching the same line). The full merge-ready essay lives in the branch + code-review, not here — one or two lines per held issue is enough to drive the ship.
• Target ceiling: the whole handoff reads in ONE Read (≤~120 lines / well under the 25k-token page cap). If it needs pagination, it's overdue for a prune — prune it THIS turn before doing anything else. A bloated handoff (the 388-line / 260KB states this file hit twice) is itself the bug, not a byproduct.

Permanent rules (apply to EVERY run — do NOT re-paste into the handoff)

• Announce to #grappa on BOTH Azzurra AND Libera (new 2026-07-14; not #it-opers). ONE announce PER BATCHED DEPLOY, not one per issue (vjt 2026-07-17) — since deploys are batched (see the batch rule below), the announce covers all issues in that bundle in one line per network (users get a single BundleRefreshBanner for the batch; tell them what changed). Post via the ircbot pane ("vjt-claude"), its own voice, no vjt-highlight for routine. The bot owns both net connections (2 monitors). The bot may decline "nothing to add" → re-brief explicitly as an unposted ship announce so it posts. See memory [[feedback_announce_ships_to_grappa]]. 🔴 A COLD DEPLOY ANNOUNCES TWICE — BEFORE AND AFTER (vjt order 2026-07-02, RE-STATED 2026-07-29). A cold restart drops every live IRC + web session, so users get a heads-up, not a surprise: • BEFORE (~30–60s ahead): "cold restart starting now, your IRC + web sessions will drop and auto-reconnect in ~1–2 min." • AFTER (post-verify, only once healthz is green): "deploy done, sessions restored" + what shipped. A HOT --cic-only deploy needs NO before-announce (no session drop) — just the after/bundle-refresh note. Both legs go to BOTH networks. Forgetting the BEFORE is the failure mode — it is the only one that costs users anything.
• BATCH ALL DEPLOYS — never deploy per-issue (vjt STANDING ORDER 2026-07-17). A per-issue --cic bundle deploy (OR cold restart) spams live users with a BundleRefreshBanner every ~20min. So: as each issue completes, worker MERGES + pushes to origin/main (the CI-green gate still gates the merge) — but does NOT deploy. Accumulate in soon. Ship ONE batched deploy only when ~4–5 issues are resolved (merged, awaiting deploy), carrying all of them in a single bundle broadcast, then ONE announce covering the batch + close all + strip their status:soon. Merge ≠ deploy: the m42 jail only pulls origin/main when deploy-m42 runs, so merging freely does not touch prod. This SUPERSEDES per-issue ship-on-green in dispatch briefs — tell the worker to merge+HOLD, not deploy. Deploy rules stack: this batching gate + the CI-green-before-ship gate (integration must be green before ANY merge/ship) + the night-cold-deploy window (cold- classified issues wait for the ~4am restart window; batch them there too). Prefer designing features HOT. See [[feedback_minimize_cold_deploys]]. 🔴 DON'T STOP AT THE COLD DEPLOY, AND SHIP HOT WHAT CAN GO HOT (vjt STANDING ORDER 2026-07-29). Two halves, both explicit: (1) a cold deploy is NOT the end of the night — keep pulling the status:queued set and dispatching, do not idle after the restart; (2) hot-shippable work must NOT be parked waiting for the next cold window — classify honestly and ship it hot. This does NOT repeal the batching gate above: batch hot ships too (a --cic batch is still one banner), just never HOLD a hot-ready batch for a cold restart it does not need. When the two rules pull against each other, the tiebreak is users see one banner per batch, and no work sits waiting on a restart it does not require.
• RELEASE-CUTTING + NEWS.JSON (vjt STANDING ORDERS 2026-07-24). After a batch DEPLOYS to Azzurra + verifies healthy: cut a GitHub release + tag (tag ≡ CTCP VERSION exactly, #391), THEN produce the site's News/Releases news.json entry — bilingual, curated by vjt, and committed+pushed to grappa-www + deployed + CF-purged, NEVER deployed-not-committed (anti-drift; trigger = testimonials left live-but-uncommitted). Full procedure + schema (grappa-www#4) in docs/OPERATIONS.md → "Release-cutting". See [[feedback_release_cut_news_json_committed]].
• Every new feature needs a REAL e2e that asserts the user-visible outcome (not a hollow green spec). A red integration/e2e CI job BLOCKS — never build/ship on red; gh run list to find where it went red, fix/bump-to-front, green it. cic ci job is Elixir-only; integration is the real e2e gate. See [[feedback_e2e_mandatory_and_ci_blocks]].
• Close-out = gh issue close N (+ announce). Ship+announce alone is NOT done.
• WORKTREE HYGIENE — remove merged worktrees (vjt STANDING ORDER 2026-07-17). Once a worktree branch is merged to main, its worktree MUST be removed (git worktree remove, --force only after merged+clean is verified — the submodule blocker needs it) and the merged branch deleted (git branch -d). Removal is part of the merge step, not a someday-cleanup — tens of stale worktrees had piled up eating disk (chore #296). EVERY dispatch brief MUST tell the worker to remove its worktree after merging. NEVER force-remove an UNmerged or DIRTY worktree — it belongs to a concurrent session's in-flight work (also the source of the "sibling stashed my changes" pitfall). Codified in CLAUDE.md Development Cycle too.
• status:* label discipline (WIP board — grappa-irc #258, mandatory 2026-07-15). The grappa.chat WIP board renders directly from three mutually-exclusive grappa-irc labels — status:queued (accepted, in build queue, not started), status:cooking (worker STILL ON IT — building, in code-review, waiting on CI including post-merge CI polling, addressing findings: ANY active worker attention on the issue), status:soon (worker FULLY DONE + handed off, no active work and NO CI-wait remaining, purely awaiting a deploy window). The board's two plain-link columns are derived: backlog = open issues with NO status:* label (shown before Queued), closed = closed issues (after Soon) — both exclude status:*. The orchestrator OWNS keeping these labels truthful, or the board drifts from reality:
  • cooking → soon fires ONLY at the worker's HAND-OFF, NEVER at merge (vjt order 2026-07-18). Waiting on CI — PR checks OR post-merge main CI — is STILL cooking, not soon. A merged issue whose worker is still polling its post-merge run stays cooking. The ORCHESTRATOR flips it to soon in the SAME turn it processes the worker's DONE hand-back (worker idle, CI settled, moved on) — the worker does NOT self-flip to soon at merge. (Prior rule "worker merge+soon" flipped prematurely during CI-wait → the exact drift vjt caught. Worker now: merge+HOLD, STAYS cooking.)
  • Enqueue (→ status:queued) is done by the ircbot or vjt, NOT you — that label is how work enters the queue (the ircbot no longer pings you to hand issues over; the label IS the handover). Your first touch is status:queued → status:cooking when the worker starts building. Move, don't add — mutually exclusive (gh issue edit N --remove-label status:X --add-label status:Y).
  • On deploy/close → REMOVE the status:* label entirely (a shipped+closed issue leaves the board's Soon column and shows only under the closed link). Removing it is part of the ship/close-out step, alongside gh issue close + announce.
  • A newly-filed backlog issue gets NO status:* label (it lives under the backlog link until triaged into the queue). The board is a shared artifact — keep it honest every transition.
  • ANTI-DRIFT (vjt caught two misses 2026-07-16 — stale cooking on closed #268; forgotten queued→cooking on the #273 dispatch). The label move is NOT a separate step you remember — it is ATOMIC with the action:
    • The queued→cooking edit goes in the SAME Bash block as the clear-and-dispatch send-keys (dispatch and label move as one tool call — you cannot dispatch without moving the label).
    • The strip status:* edit goes in the SAME handling turn as processing the worker's shipped/closed report (alongside gh issue close + the announce brief).
    • lib/board-check.sh [--cooking N] is the STANDING GUARD. Run it at EVERY handoff-flush and EVERY /orchestrate resume (Step 0). It fails (exit 1) on: a CLOSED issue with a status:* label, any issue with >1 status label, or (with --cooking N) a cooking set that doesn't match the in-flight issue you believe is building. It bakes in --limit 300 — plain gh issue list defaults to 30 and silently truncates older issues (that truncation masked the drift twice). If it prints DRIFT, fix it BEFORE doing anything else.
• Pull the queue at end of each round (2026-07-15). The status:queued label set IS the execution queue — there is no hand-managed list. When the worker is free and nothing is in flight, read the open queued set (gh issue list --state open --label status:queued --json number,title,labels) and dispatch the next per the placement rules in /srv/grappa/docs/ISSUE_PIPELINE.md (P0 first / never preempt in-flight, then similarity-group, else lowest number), moving it status:queued → status:cooking. This REPLACES waiting for an ircbot handover. Only when the queued set is EMPTY do you ping vjt "what next?" — don't invent work.
• Auto-clearer: lib/auto-clear-watch.sh start|status grappa-orch runs an external watchdog that, at ctx≥40% (idle+quiet, 60s debounce), FIRST prompts the orchestrator to flush its handoff, WAITS for that flush turn to settle (polls busy→idle, capped at AUTOCLEAR_FLUSH_MAX=180s), and only THEN /clears + /orchestrates. The flush-before-clear step (added on vjt's order) means an auto-clear no longer races your unsaved in-flight state. Still: keep the handoff current proactively — the watchdog's flush-prompt is a safety net, not a substitute (a wedged/slow flush past the cap clears anyway; and you may be mid-halt on something the prompt can't fully capture). ALWAYS flush any open decision before going idle.
• Halt + ESCALATE on: design picker, plan deviation, real breakage, CI regression (2nd recurrence), ambiguous scope, daemon/pane death, PACK COMPLETE. Don't auto-pick design/ product choices; orchestration mechanics MAY be auto-defaulted.
• WHEN YOU NEED VJT'S INPUT, PING HIM VIA THE IRCBOT — ALWAYS. vjt lives on IRC, NOT in the orchestrator conversation; a reply typed only into this session can sit unseen for hours. Any time you need his decision/answer (escalation, design picker, scope question, ambiguous call, PACK COMPLETE, "what next?"), brief the ircbot pane ("vjt-claude") to post a #grappa message HIGHLIGHTING his nick vjt (push) with the concise question — THEN hold. Posting the question in the conversation alone does NOT count as pinging him. (Routine ship announces still go without the highlight; the highlight is specifically for "I need your input".) This is non-negotiable — vjt set it as a standing order 2026-06-29. See [[feedback_orchestrator_ping_vjt_via_ircbot]].
• PERMISSION DIALOGS ARE VJT'S, WITH EXACTLY ONE STANDING EXCEPTION (vjt, 2026-07-26). You do NOT answer a worker's permission prompt on your own — that dialog is his control point, and your own judgement that an action "looks harmless" is precisely what it exists to not rely on. The single exception he granted: removing a STALE GIT LOCK FILE inside the worker's own worktree. Its two conditions are non-negotiable and he stated both explicitly: (1) "verifica sempre prima" — every single time, first confirm no git process is running (pgrep -fl "git " on the worker's host, PATH exported) and inspect the lock; never once-and-for-all. (2) "e SOLO per git lock / non altri files" — git lock files ONLY. Anything else, however similar it feels (a stale submodule .git, an object file, a scratch artifact), goes back to him. Always answer option 1, NEVER option 2 — option 2 is a permanent directory allowlist he has not granted. When you do use the exception, say so in the turn so the click is on the record.

After reading the handoff, proceed to Step 1 (resume-check) to reconcile it against live daemon/pane state.

Step 1 — check for existing state (resume case)

.claude/skills/orchestrate/lib/resume-check.sh <SIBLING_PANE_ID>
# → "RESUMING age=NNs daemon=running"   (state file fresh + daemon up — pick up live)
# → "RESUMING age=NNs daemon=stopped"   (state file fresh but daemon died — restart needed)
# → "STALE   age=NNs"                    (state file ≥600s old — treat as fresh)
# → "FRESH"                              (no state file → first invocation)

If RESUMING daemon=running:

• Do not wipe the state file or stop the daemon.
• Do not clear or interrupt the sibling pane.
• Re-read the active plan + active checkpoint so you know what "as planned" means.
• Query current sibling state: lib/state.sh <PANE>.
• Arm wait-for-event.sh (Step 2.4) and resume the decision tree.

If RESUMING daemon=stopped:

• Restart daemon: lib/daemon.sh start <PANE>. Cursor + state file preserved.
• Re-arm wait-for-event.sh.

If STALE or FRESH, fall through to Step 2.

Step 2 — first invocation

1. Identify panes:

   tmux list-panes -F '#{pane_index} #{pane_id} #{pane_active} #{pane_current_command}'
   

   The OTHER pane (not the one this session runs in) is the target. Note its %id.

2. Read the active plan: invoke /start to get the workflow context, then read the relevant GitHub issue(s) for the task (gh issue view <n>) — plus the feature's ephemeral plan file under docs/plans/ if one exists this session — so you know the sub-task order. Read docs/checkpoints/*.md with status: active for current state. gh issue view <n> plain is BROKEN by the classic-projects deprecation — always pass --json: gh issue view <n> --json number,state,title,body,labels -q .... Same for closing: gh issue close <n> -c "<note>".

3. If STALE, wipe stale files: rm -f /tmp/orchestrate-state-<id>.json /tmp/orchestrate-cursor-<id> /tmp/orchestrate-events-<id>.log /tmp/orchestrate-daemon-<id>.pid. (The leading % from the pane id is stripped in the filenames.)

4. Start the daemon — it ticks every 5s and emits a BOOT event on first tick:

   .claude/skills/orchestrate/lib/daemon.sh start <SIBLING_PANE_ID>
   

   Wait ~3s, then verify: .claude/skills/orchestrate/lib/daemon.sh status <SIBLING_PANE_ID> should report last_event: BOOT state=....

5. Arm the event stream — ONCE, for the whole session, covering EVERY pane:

   Monitor(
     command: "/srv/grappa/.claude/skills/orchestrate/lib/monitor-stream.sh %16 %28",
     description: "grappa worker pane events (w1 %16, w2 %28)",
     persistent: true,
     timeout_ms: 3600000
   )
   

   Every event the daemons write now arrives as its own notification. There is no re-arm. Do not arm a wait-for-event.sh alongside it — one listener, and it is this one.

   Pass all worker panes in the single call. One monitor for N panes beats N monitors: fewer things to lose track of, and the pane label is already in every line ([grappa-worker %16] IDLE ctx=24%).

   The stream is filtered to what you act on — IDLE, PROMPT-*, PICKER*, USER-TYPED, CTX-*, BOOT, PANE-MISSING, HEARTBEAT, STALL state=idle. BUSY and STALL state=busy are deliberately excluded: a working worker is the common case, and Monitor auto-stops a stream that gets too chatty — losing the whole feed to keep the least useful events would be a bad trade. When you need busy-state ground truth, capture the pane or use lib/state.sh.

   🔴 Verify it took: the tool returns a task id. If the monitor is ever auto-stopped for volume, or the session's monitors are cleared, you get no error — you just stop hearing anything. So on resume, and any time both panes have seemed quiet for a while, confirm the feed is alive rather than assuming calm (see "Resume", and the 2026-08-02 entry under Pitfalls).

Detector internals (in lib/wakeup-tick.sh)

Busy detector: a line in the last 30 (was 15 in v1 — permission modals push the spinner offscreen) must carry … ( (the spinner shape: ellipsis + space + open-paren that introduces the parenthesized status — (NNs · ...) once the timer arms, (thinking) / (almost done ...) in the pre-timer phase) — OR an explicit Press up to edit / esc to interrupt prompt. Bare … is NOT enough: truncated task descriptions (tok…, … +N completed, … +N pending) used to produce false-busy events for ~30 minutes during CP10 S6.

Prompt detector: Do you want to proceed? AND a 1. Yes numbered list. Emits PROMPT-PENDING instead of IDLE so the orchestrator doesn't try to clear sibling mid-prompt. (v1 lesson: visitor-parity cluster wasted ~10 turns trying to clear sibling that was waiting on a CDP cp permission click.)

Picker detector: ↑/↓ to navigate OR Tab/Arrow keys to navigate OR Enter to select (the design-Q multi-choice modal Claude Code pops). Emits PICKER — orchestrator MUST halt + ping vjt.

USER-TYPED detector: hashes the last ❯ <text> line; if it changes vs prior tick (md5), emits USER-TYPED so orchestrator knows vjt typed in pane directly. Observe-only — don't intervene.

ctx parse: tries 🧠 NN%, falls back to TBD (post-/clear empty). v1 emitted ctx=% (broken parse) when status line wrapped offscreen; v2 always returns a valid value.

Idle debounce: a single idle read after a busy read can be a transient tool-call gap (between Read/Bash result rendering and the next spinner line). The tick re-captures after 5s and only classifies as idle/prompt/picker/busy on the second read.

Decision tree per event

A wait-for-event.sh exit may emit MULTIPLE event lines (events queued during a no-waiter window). Process each in turn:

Event	Action
BOOT state=idle	Capture pane (tail -50), orient on what just landed, then re-arm
BOOT state=busy	Sibling mid-work; re-arm, no intervention
BOOT state=prompt	Sibling on a permission prompt — halt + ping
BOOT state=picker	Sibling on a design-Q picker — halt + ping
IDLE ctx=NN%	Run the IDLE decision tree below
BUSY ctx=NN%	Sibling started new work; re-arm
PROMPT-PENDING ctx=NN%	Sibling needs vjt's permission click — halt + ping. Do NOT send keys, do NOT clear, do NOT investigate the prompt content (it's typically a cp script approval — vjt clicks 1 or 2). Wait for PROMPT-CLEARED.
PROMPT-CLEARED ctx=NN%	Sibling unblocked, re-arm
PICKER ctx=NN%	Sibling popped a design-Q multi-choice — halt + ping vjt with the choice options. Capture pane, identify the question + choices, present them concisely. Optionally include your recommended pick + 1-line reasoning, but the call is vjt's.
PICKER-CLEARED ctx=NN%	vjt picked, sibling processing — re-arm
USER-TYPED ctx=NN%	vjt typed in pane directly. Capture, note what they said, re-arm. Do not respond on vjt's behalf — sibling will.
CTX-BUMP NN% at ≥30%	Proactively suggest clear-cycle (don't wait for IDLE). At ≥30% the next chunk of work likely won't fit before auto-compact.
CTX-CRITICAL NN% at ≥80%	Aggressive clear posture — ask sibling to flush + clear at next safe checkpoint, even mid-bucket if needed. Auto-compact lurks.
STALL state=busy duration=Ns	Long-running busy state. Capture pane to confirm legit progress (long doc-write, large compile, multi-step subagent). If pane shows real progress → re-arm, false alarm. If genuinely stuck → halt + ping.
STALL state=idle duration=Ns	Orchestrator is the bottleneck, not sibling. Sibling has been waiting on you. Capture pane: (a) if sibling self-issued CLEAR and staged /tmp/orchestrate-next.txt → auto-dispatch immediately (do NOT ping vjt — autopilot mandate), (b) if sibling left a free-form question or design choice → ping vjt with the question, (c) if sibling looks done with nothing pending → ping vjt to ask "next?". Don't just re-arm and wait — STALL idle MEANS act now.
HEARTBEAT state=<...>	Long quiet period (≥600s no event). Capture pane to confirm legit progress vs invisible deadlock; re-arm
PANE-MISSING	Halt + ping user. Daemon has exited — manual restart needed.

On IDLE event:

1. Capture: tmux capture-pane -t <PANE_ID> -p | tail -50

2. Inspect last assistant message. Categorize:

   Pane state	Action
   Step landed cleanly + offers next step from plan order	Ask clear
   Sibling already self-issued CLEAR + staged /tmp/orchestrate-next.txt	Skip the ask, go straight to clear-and-dispatch
   Session asks design question (X vs Y, which approach?)	Halt + ping user (note: should have been caught by PICKER event; if a free-form ask shows up post-IDLE the picker detector missed it — investigate)
   Plan deviation (sub-task skipped or reordered without OK)	Halt + ping user
   Codebase review gate fires (per CLAUDE.md threshold)	Halt + ping user
   Background agents still running (e.g. parallel review agents — general-purpose / Plan row visible)	False idle — ignore, re-arm
   User typed in pane directly	Watching only — don't intervene

   Live deploys / pushes / shared-infra writes default to halt; if the user has explicitly authorized autopilot for the run, treat them as plan-aligned and let sibling proceed.

3. Ask clear path: send to pane:

   orchestrator: same drill before <next step>. /clear or no? if yes WRITE the full prompt body (fully self-contained for /clear, no auto-summary safety net — explicit file paths + commit SHAs + first action) to /tmp/orchestrate-next.txt and reply with literally "CLEAR". if no reply with literally "NO CLEAR". do NOT print the body inline in chat.
   

   Why file handoff, not pane scrape: the prompt body is large + can be many KB. Going through tmux scrollback (sibling prints body → orchestrator captures → reconstructs from line-wrap → loads into paste-buffer → pastes back) is fragile (line-wrap concat ambiguity, ANSI artifacts, <system-reminder> bleed) and bloats both sessions' context. File handoff: sibling Writes once, orchestrator instructs sibling to Read it post-clear. Zero paste-buffer, zero scraping.

4. On reply:

   • Reply contains literal NO CLEAR → send go on with <next step> per plan.
   • Reply contains literal CLEAR → run /clear, then send a short directive: read /tmp/orchestrate-next.txt and execute it. Sibling Reads + acts. No paste-buffer.

   The 5s tick (was 20s, was 60s in v1) catches fast NO-CLEAR / CLEAR replies near-instantly — you'll get the IDLE event within ~10s of the sibling answering.

5. Always re-arm wait-for-event.sh before returning. (Fail-soft: even if you forget, the daemon keeps ticking; next call to wait-for-event.sh resumes from cursor with all queued events.)

Sending text to the sibling pane

Submit a normal message:

tmux send-keys -t <PANE_ID> '<text>' Enter
sleep 1
tmux send-keys -t <PANE_ID> Enter   # second Enter — sometimes needed to actually submit

The first send-keys often leaves the text queued without submitting; the second Enter flushes. Verify with tmux capture-pane | tail -5 showing a spinner appearing.

Running /clear with a fresh prompt

/clear is a slash command — the / MUST be TYPED, not pasted. /clear takes no argument: it wipes the conversation, then the next sent message is the new turn-1 user prompt.

After sibling has Written the body to /tmp/orchestrate-next.txt (and replied CLEAR), the orchestrator's job is just three short sends — no paste-buffer, no scraping:

# 1. Clear any leftover input
tmux send-keys -t <PANE_ID> C-u
sleep 1

# 2. TYPE /clear + Enter (wipes the conversation)
tmux send-keys -t <PANE_ID> '/clear' Enter
sleep 3

# 3. Verify clear landed: status line should show `🧠 TBD` (fresh, no tokens).
tmux capture-pane -t <PANE_ID> -p -S -25 | grep -E "🧠 TBD|🧠 [0-9]+%" | tail -2

# 4. One short directive — sibling reads the file and executes.
tmux send-keys -t <PANE_ID> 'read /tmp/orchestrate-next.txt and execute it.' Enter
sleep 1
tmux send-keys -t <PANE_ID> Enter   # second Enter — sometimes needed to actually submit

After sibling Reads and starts working, ctx jumps from TBD to a small % (Read of a few KB) and the spinner appears, confirming turn 1 of the clean session is underway.

Why this is safer than paste-buffer: the prompt body never traverses the tmux paste buffer or pane scrollback. No line-wrap reconstruction, no ANSI/<system-reminder> bleed, no quoting hazards. The orchestrator never needs to read the body — only the sibling does, and Read gives it a clean, file-rooted view.

If you ever fall back to the legacy paste-buffer path (sibling printed the body inline by mistake), see git history of this skill before 2026-04-27 for the scrape-and-paste-buffer recipe — it was retired because file handoff is strictly better.

Halt protocol

When you halt:

• PING VJT VIA THE IRCBOT (vjt-claude pane): brief it to post a #grappa message HIGHLIGHTING vjt with the concise question — what landed, what's pending, what the Q is. This is the REAL escalation; a reply only in the orchestrator conversation does NOT reach him (he's on IRC, not watching this session).
• Also drop the one-line summary in the conversation (for the record), but the ircbot ping is what gets his attention.
• Do not send anything to the sibling pane.
• Do not run /clear.
• Do not reschedule the next tick — wait for user direction. (Decide explicitly: if you want passive monitoring to continue while you halt, schedule the next tick and just don't act on its events until the user replies.)

After user direction:

• Translate into the appropriate send-keys sequence to the sibling pane.
• Resume normal tick-event handling (re-arm ScheduleWakeup if you stopped).

Resume after /clear (orchestrator side)

The daemon at /tmp/orchestrate-daemon-<pane>.pid runs independently of the orchestrator's Claude session. State + cursor + event log persist in /tmp. The user clears the orchestrator session freely to save tokens. On /orchestrate invocation post-/clear:

1. Run lib/resume-check.sh <PANE_ID>. Branch on output:
   • RESUMING daemon=running → daemon kept ticking. Skip to step 4.
   • RESUMING daemon=stopped → state file fresh but daemon died. Restart: lib/daemon.sh start <PANE>. Cursor preserved.
   • STALE → daemon is gone or never ran. Treat as fresh: Setup Step 2.
   • FRESH → first invocation: Setup Step 2.
2. Re-read the active plan + active CP so you have the "as planned" frame again.
3. Query current state: lib/state.sh <PANE> — gives you ground truth (state, ctx, last_state_change age, etc.) without consuming events.
4. Capture every worker pane once for orientation: tmux capture-pane -t <PANE_ID> -p | tail -40. Do this for ALL of them, not just the one you were last thinking about — a worker halted on a question looks identical to a worker you simply forgot.
5. Re-arm the persistent Monitor on lib/monitor-stream.sh with ALL panes (Setup step 5). A monitor does NOT survive the orchestrator's /clear — the daemons do, the monitor does not. tail -n0 means it starts from now, so anything the daemons wrote while you were away is not replayed: step 4's captures are what recover that window, which is why they are not optional. For a precise diff of what you missed, lib/wait-for-event.sh <PANE> as a deliberate one-off drain still works (cursor-tracked), or read the tail of /tmp/orchestrate-events-<id>.log.

The daemon-survives-clear design means the record is never lost. The listener is what you must re-establish, and its absence is silent — so re-arming the Monitor is step 5 of every resume, not an optional flourish.

Pitfalls (learned in S29 of CP07 + CP08/CP09 Phase 2/3 + CP10 S6 + visitor-parity cluster v2 rewrite)

• Don't interrupt the session mid-generation. If the sibling is still writing the prompt body and you ask another question, you destroy the prompt. Wait for full IDLE.
• Spinner words vary wildly. Cooked, Crunched, Sautéed, Churned, Baked, Cogitated, Worked, Whipped, Brewing, Stewed, Boondoggling, Mulling, Quantumizing, Forging, Spinning, Befuddling, Undulating, Zigzagging, Proofing, Osmosing, Transfiguring, Crystallizing, Reticulating, Billowing, Calculating, Discombobulating, Imagining, Hullaballooing, Pouncing, Channeling, Spelunking, Thundering, Smooshing — don't match words; match the spinner shape … ( paired with parenthesized status.
• Bare … is NOT a busy signal. Truncated task descriptions (tok…, M3, H11, M2, M12 — already organic…), task-list compaction (… +N completed, … +N pending), and sibling-printed punctuation all carry … while the session is fully idle. The fixed regex requires … ( (ellipsis + space + open-paren) on the same line. The earlier "match … ellipsis, not specific words" rule (CP08-era) was too loose — fixed CP10 S6 after a stalled sibling reported BUSY for ~30 min while truly idle.
• Permission prompts and design pickers are NOT idle states. v1 conflated them with IDLE → orchestrator would try to clear sibling mid-prompt or send keys to dismiss the modal. v2 detects them as PROMPT-PENDING / PICKER events with their own halt semantics. If you ever add a new modal class to Claude Code (multi-step wizard, inline diff confirm, etc.), extend wakeup-tick.sh to detect it.
• paste again to expand is just a hint, not an error. (Legacy paste-buffer path only — file handoff avoids the warning entirely.)
• /clear confirmed by 🧠 TBD in status line (fresh conversation, no tokens). After the sibling Reads the prompt file and starts working, ctx jumps to a small % (e.g. 5–10%), confirming turn 1 landed in a clean session. If you still see the pre-clear ctx %, /clear didn't fire — re-run the sequence.
• Background agents leave the spinner gone but work continues. If pane shows N local agents or task list with ◻/◼ items, it's a false idle even if no spinner is up. Don't propose clear, wait. With the v2 busy detector this is mostly handled (no spurious BUSY) but the IDLE event after the agents finish IS the right signal — just don't act if you see active agent rows.
• Halt at human-required steps even on autopilot. iPhone/device tests, explicit user-tagged tasks (◼ HALT for ...), real-credential operations the user hasn't pre-authorized.
• Self-contained prompt files only. With /compact an auto-summary covers gaps. With /clear, the prompt body in /tmp/orchestrate-next.txt is the ENTIRE context the sibling has after wipe. Sibling MUST bake in: every sub-task SHA so far, file paths, exact first action, all carried-forward state from any "deferred to next sub-task" notes. Tell sibling that explicitly when asking for the file.
• Daemon survives orchestrator restarts but NOT host reboots. State + log + pid file in /tmp — fine across /clear + /exit + harness restart. If the box reboots, /tmp may be wiped (depends on OS); resume-check returns FRESH and you start over. Not a bug, just a constraint.
• Sibling can stash YOUR working-tree changes during its own deploy. Visitor-parity V9 cluster: orchestrator was rewriting lib/orchestrate/* while sibling was prepping V9 deploy from a clean tree; sibling correctly stashed orchestrator's changes as orchestrator-infra-pre-v9-deploy. Untracked new files were lost (default git stash skips untracked — use -u if you care). Fix: stage + commit infra changes onto a separate branch BEFORE letting sibling deploy, OR pause infra work during sibling's deploy windows.
• Stale task IDs surface back as notifications. The harness sometimes re-fires completion events for old task-ids. Don't treat them as new events — verify the cursor advanced before processing. v2 cursor-tracking makes this safe (re-reading the same byte range yields nothing).
• Recurring same-triplet flake = real regression, not flake (per feedback_recurring_e2e_not_flake). The visitor-parity cluster failed CI on the SAME 2 specs (network-circuit-ets-leak + push-server-fires-30s) for 6+ buckets in a row. Each bucket "documented as pre-existing flake and proceeded" — this is exactly the retry-mask pattern the rule warns against. Halt + investigate after the SECOND consecutive recurrence, not the sixth.
• STALL state=idle means YOU forgot to dispatch. Don't ping vjt with "sibling stalled" — sibling is waiting on you. If the pane shows sibling's CLEAR + a staged /tmp/orchestrate-next.txt, auto-dispatch immediately under the autopilot mandate. Origin: visitor-parity cluster CLOSE → Images dispatch — orchestrator pinged vjt twice asking "Images dispatch a/b/c?" while sibling sat idle for 600+ seconds. The autopilot rule from cluster open already covered "dispatch staged next-cluster prompts without asking" — STALL idle is the signal that you missed the cue.

Project standing rules — grappa (moved out of the handoff 2026-07-29)

These are PERMANENT: they were living in .orchestrate/orchestrator-resume.md, which is a live-state snapshot that gets pruned every flush — the wrong home for rules that must outlive the pruning. The handoff now carries state only and points here.

🚢 DEPLOY POSTURE (prod = m42 bastille jail)

Worker MERGES + pushes, never deploys; stays cooking until its DONE hand-off; ORCH flips to soon. ONE batched deploy (~4–5 issues), ONE dual-net announce, then close all + strip labels.

• COLD: /srv/grappa/scripts/deploy-m42.sh --force-cold · HOT: --force-hot THEN --cic — a HOT deploy is TWO runs; one alone ships half the range. ABSOLUTE path, redirect to a file (a pipe SIGPIPEs the remote deploy).
• 🔴 RUN DEPLOYS DETACHED (nohup + disown). Tonight the --cic run was HARNESS-REAPED mid-vite build (status killed, no rc); detached, it completed. Same rule as long gates.
• 🔴 WORKERS SYSTEMATICALLY MIS-CALL SERVER CHANGES "COLD" — CHECK IT YOURSELF. The test: git diff --name-only <prod-sha>..<branch> | grep -E '^config/|priv/repo/migrations/|mix.exs|mix.lock|Dockerfile|infra/|lib/grappa/application.ex' — empty ⇒ HOT. ⚠️ The ^infra/ arm over-triggers: a shell script under infra/freebsd/ is git-pulled and run at deploy time, no restart needed. Let the CONTENT decide, not the grep.
• 🔴 PROVE A HOT DEPLOY by the reload {"failed":[]} list + the served cic bundle hash (curl https://irc.sindro.me/). /api/config stays STALE after a hot deploy — valid for COLD only. A release rpc from root fails :noconnection — use service grappa status + fetch http://127.0.0.1:4000/healthz.
• 🔴 grappa.chat is the MARKETING SITE; the APP is irc.sindro.me.
• 🔴 main MOVED FIVE TIMES tonight under in-flight branches (a THIRD session pushes shottino every few minutes, authored Your Name <[e-mail]> — an unconfigured git identity landing on main; worth telling vjt). The rule that worked every time: verify the landed diff yourself and let the CONTENT, not the SHA, decide whether a re-gate is owed. Twice it saved a pointless 25-min re-run. For a starved --ff-only push: rebase + push as ONE immediate sequence, retry ≤5, and STOP if an incoming commit touches lib/, test/, cicchetto/, priv/, config/, mix*.

🚦 SEMAPHORES — I AM THE ALLOCATOR (probe the HOST, never take a worker's word)

COMPILE = anything touching the shared _build (check.sh, mix.sh …, any mix compile). STACK = docker / e2e / integration.sh. Cic-only gates (bun.sh run check|test) need NEITHER — never make a worker queue for those (both workers ask anyway; just say no lane needed). Grant them SEPARATELY and say which. 🛑 NEVER RECORD A LANE VERDICT HERE — PROBE BEFORE EVERY GRANT. This line used to read "LANE IS CURRENTLY FREE" and that cached verdict is what made me grant an occupied stack (00:1x, cost ~10 min of a run). A handoff records what WAS true; only pgrep on the host records what IS. Probe (non-interactive ssh has no docker on PATH): ssh voyager 'export PATH=$PATH:/usr/local/bin:/opt/homebrew/bin; pgrep -f "check.sh|bats-exec|mix |integration.sh"; docker ps' — check.sh's bats stage shows NO container, so docker ps ALONE LIES; pgrep is the authority. 🔑 REBASE BEFORE GATING. 📟 🧠 NN% is the CONTEXT gauge (40%-clear rule); ⚗️ NN％ is NOT context. CLEAR WORKERS AT 40%, at a CLEAN BOUNDARY (after a commit, or while a long gate runs) — gate FIRST, then clear: clearing on unverified edits leaves the next session unable to tell whether they hold.

🧷 KNOWN RED / caveats

• 🔴 FALSE-GREEN TRAP scripts/_lib.sh:34 — run scripts from the worktree ROOT or you gate MAIN's tree.
• 🔴 HOLLOW GREEN: reconcile the tick COUNT against the summary AND confirm BOTH projects (~440 chromium + ~112 webkit). Read the Playwright SUMMARY, never the exit code — tonight's proof gate exited 1 on a tolerated flake while PASSING its pre-registered criteria.
• 🥇 PRE-REGISTER pass/fail criteria BEFORE a run when a tolerated red is expected, and HOLD them when the result is inconvenient. 🥇 ESTABLISH THE BASELINE BEFORE BLAMING A BRANCH. 🥇 A red that reproduces beats any code-path argument; a red that does not reproduce beats any statistic.
• 🔴 NEVER weaken an assert to get green. Tonight vindicated this twice: the issue496 spec was RIGHT and the branch was wrong — after the revert those three went green untouched.
• 🔴 A GATE IS A SAMPLE, NOT A LIST — scope a sweep from a systematic scan across every spelling, never from the failures you happened to see.
• ⚠️ check.sh aborts at the first failing stage — "check red" does NOT mean "only style is broken".
• 🔴 HARNESS REAP looks like infra death — tell is the missing rc / task killed. Long gates + deploys DETACHED.
• 🔴 e2e serves a PRE-BUILT cic dist (runtime/e2e/cicchetto-dist) — a cic fix needs a bundle rebuild.
• 🔴 check is src-scoped ⇒ gates neither e2e/ nor cic vitest (#484 tracks the ~20 pre-existing e2e type errors).
• 🔴 CROSS-WORKTREE _build CONTAMINATION: a gate naming a module absent from your source = the neighbour's branch; scripts/mix.sh --env=dev compile --force.
• 🔁 Healthy integration ≈ 19–25 min, ~24 tests/min. A sub-5-min failure is registry/network death — re-run once.
• 🔴 A main integration gets CANCELLED by the next push (concurrency group). cancelled ≠ failure, but nothing settled. Only a settled green at the FINAL SHA gates a deploy. THE ORCHESTRATOR WATCHES CI, NOT THE WORKERS.
• 🛑🛑 NEVER SEND A BARE Enter WITHOUT CAPTURING THE PANE FIRST — if a picker opened meanwhile, that Enter SELECTS the highlighted option. Never Esc a picker either. (The guard caught exactly this tonight.)
• 🔴 LONG send-keys GET SWALLOWED — short one-line orders, one constraint each; often needs a THIRD Enter.
• 🔴 IRCBOT: cd /home/vjt/code/IRC/vjt-claude && ./bot.say '#grappa' <<'EOF' … EOF. 🔴 FLAGS GO BEFORE THE TARGET — bot.say -f …/bot.send.libera '#grappa', NEVER '#grappa' -f …: the parse loop stops at the first non-flag arg, so a trailing -f is silently ignored and the message goes to AZZURRA. 🔴 bot.say exits 0 even when wedged — VERIFY the PRIVMSG in bot.log / bot.libera.log. 🔴 THE BOT LOGS SPAN DAYS, ARE NOT SORTED, AND CARRY NO DATE — anchor to TZ=Europe/Rome date before reading any line as a reply; a stale "faccio io" from another day nearly read as authorization.
• 🔴 ci.yml triggers ONLY on push-to-main or a PR targeting main, and is Elixir-only ⇒ GitHub CI cannot see a red cic vitest. After ANY change to a shared cic verb, run the FULL vitest.
• CI flakes (tracked): #277 #279 #254 #291/#339 #519 #520 #522 #506, bahamut IP-autokill. OTP29 pair #355/#185 HELD; bats #44 pre-existing red; hex.audit/deps.audit CVE wall NON-FATAL.
• 🔴 Never cite DESIGN_NOTES as current behaviour without confirming it in the code first.

🏷️ LABEL DISCIPLINE

lib/board-check.sh [--cooking "N M"] at EVERY flush + resume. Moves are ATOMIC: queued→cooking rides the SAME Bash block as the dispatch send-keys; strip status:* rides the SAME turn as processing the shipped report. cooking→soon only at the worker's DONE hand-off — CI-wait is still cooking. A closed issue carries NO status:*. Enqueue is vjt's or the ircbot's — except when he says "fix it" in conversation: that IS the enqueue (#526, #522).

🔀 PR / MERGE / GATING MECHANICS (learned the hard way 2026-08-01 — permanent)

• 🔴 A CONFLICTING PR RUNS NO CI AT ALL. GitHub cannot build refs/pull/N/merge for a conflicting PR, so pull_request workflows never fire — zero runs, zero check-runs, and gh pr checks says "no checks reported", which reads like "not started yet" and strands a poller forever. When an expected run never appears, check in this order: gh pr view --json mergeable,mergeStateStatus FIRST, then the workflow paths: filter, then [skip ci]. Cure = rebase onto current main + --force-with-lease; CI restarts by itself. Neither ci.yml nor integration.yml has workflow_dispatch, so for a PR whose run NEVER STARTED, fixing mergeability is the only route.
• 🔧 gh run rerun <run-id> --failed re-runs just the failed jobs of an EXISTING run, and needs no workflow_dispatch. Use it when a settled run went red on a diagnosed-transient cause — it beats pushing an empty commit (no history pollution) and beats close/reopen (which does nothing). The "no manual lever" rule above applies ONLY when no run exists to re-run.
• 🥇 A FIXTURE-LEVEL FLAKE MUST BE MATCHED BY MECHANISM, NEVER BY SPEC NAME. An auto-fixture (_vjtReset) runs for every test, so its race surfaces in whatever spec happens to be running — #195 originally, #263 on 2026-08-01, both the same bug (#277: resetSubject 500 → {:nick_rejected, 433, "vjt-grappa"}). Checking the tracked-flake list by spec name will never match it. Key the triage off the ERROR SIGNATURE + the fixture frame in the stack (fixtures/test.ts in the trace = not your test's fault), not off which spec went red.
• 🥇 Verify a rebase with git merge-base --is-ancestor origin/main <branch> — never with GitHub's mergeable field (async, lags a push) and never with the worker's belief that it rebased. Rebase + force-push must be ONE immediate sequence: main moving mid-rebase puts the PR straight back to CONFLICTING (cost two rounds on PR #600).
• 🥇 After a rebase-then-direct-merge, judge "did it land?" by COMMIT CONTENT (git log origin/main --grep '#NNN'), never by merge-base --is-ancestor on the PR head — a rebase gives the landed commits NEW shas, so the PR head is legitimately not an ancestor.
• 🥇 THE STRUCTURAL CURE FOR THE STALE-PR PROBLEM: FORCE-PUSH THE REBASED BRANCH BEFORE THE FF-MERGE. Order that works (w2, #600, verified): rebase onto origin/main → push --force-with-lease the BRANCH → ff-merge into main → push main with an explicit refspec. Because the remote PR head is now the rebased commit, the ff-merge makes it a genuine ancestor of main and GitHub marks the PR MERGED by itself — no manual close, and no stale pre-rebase head left behind. The leak below happens when the rebase stays LOCAL and only main is pushed: the PR keeps its pre-rebase head forever. Prefer this order; treat "remember to close it" as backup.
• 🔴 CLOSING THE PR IS PART OF THE MERGE STEP, NOT A LATER SWEEP — this leaked FIVE times in one day (#587 #586 #583 swept 05:10; #602 #603 swept 12:0x, all on already-shipped issues). A rebase-then-ff-merge leaves the PR open with its pre-rebase head still reading mergeable — a standing invitation to ship the same work twice. Put "close the PR" in every merge brief; the periodic sweep is the symptom, the merge step is the fix. Audit method — use it, never eyeball: git log origin/main --grep '#NNN' to find the landed commit, then git patch-id --stable on both heads. Identical ids ⇒ definitively landed. Differing ids do NOT mean unlanded — a rebase legitimately rewrites context lines. Then diff the PR's touched files against main and look ONLY for lines the PR has that main LACKS; none ⇒ landed. (#603 differed by exactly one comment terminator that a sibling PR had extended.)
• 🥇 "ancestor of main" does NOT prove a worktree's work merged — it equally matches a branch with NO commits. Check for commits before concluding a worktree is disposable.
• 🥇 Gate via PR CI, not the local STACK, whenever a lane is contended — the PR runs the full suite for free and leaves the host lane for whoever actually needs a testnet.
• 🥇 Diff the e2e test COUNT across the gate: +1 proves a new spec really ran; an UNCHANGED count is expected only when the change is server-side with ExUnit coverage. State which case applies before calling a green real.

🧪 FLAKE FORENSICS

• 🥇 A fixed identifier in a shared namespace is the classic flake: a hard-coded nick/channel/port collides with a ghost from a prior run, so re-running is exactly what triggers it, and it does not fail where it is caused. Suspect that before suspecting the code under test. (#600: fixed peer nick m591peer → 433 → irc-framework registers under an ALTERNATE nick while the fixture keeps the requested one → /ping DMs a phantom → 15 s timeout.)
• 🔴 Fix a flake by making the SETUP deterministic (unique per-run identifier, wait on an observable ready signal) — never by bumping a timeout blindly and never by weakening the assert.
• 🥇 Give every diagnosis a falsification condition and let the worker run it. A dispatch body is a HYPOTHESIS: mine was killed by one git merge-base call, and the worker was right. Accept it plainly and move on.

🪟 TMUX VIEWPORT / PICKER MECHANICS

• 🔴 A -S capture of a pane SHORTER than its content reads SCROLLBACK, not the live view — keystrokes then appear to no-op against a picker you "can see". tmux capture-pane -p with NO -S, plus a Down probe that visibly moves ❯, is the liveness proof.
• 🔴 NEVER pin a window's size to un-cramp a pane — the window is almost certainly being watched. I did exactly this (window-size manual + resize-window -x 200 -y 60 on 0:2) on the theory that "no client is viewing it". FALSE, and vjt had to revert it (set-window-option -t 0:2 -u window-size): 0:2 was the ACTIVE window of a session with THREE attached clients, including his phone at 71x60 — the pin broke his resize-to-viewport. tmux list-clients tells you who is attached to the SESSION, and if the window is the active one those clients ARE viewing it. Never infer a window is free just because you are not in it. A cramped pane is the user's terminal geometry: report it and let him fix it (detach / resize on his side) — geometry is his environment, not yours.
• 🥇 Picker input: number keys select in a SINGLE-select; in a MULTI-select they do nothing — ↑/↓ to the row, Enter toggles [ ]→[✔], then navigate to Submit and Enter, then 1 on the confirm screen.

🔁 RECURRING WORKER-BRIEF CORRECTIONS (say these in EVERY dispatch)

Workers regress to these every time, and a worker's OWN staged resume file is written from its memory, not from these rules — read a worker's /tmp/orchestrate-next-<w>.txt for wrong rules before you dispatch it (w2's said "ask vjt for the STACK lane", which is flatly wrong: lanes are MINE).

• STACK (docker/e2e/integration.sh) and COMPILE (mix/check.sh, shared _build) are EXCLUSIVE and I allocate them. Ask ME, never vjt, never self-serve. Cic-only gates (bun.sh run check|test) need NEITHER.
• The worker MERGES + pushes ONLY on my word; the DEPLOY is always held. No gh issue close at merge. CLOSE THE PR at merge (see PR/MERGE MECHANICS). Remove the worktree + delete the branch at merge.
• No Closes #NNN in a PR body — it auto-closed #540 while prod lacked the code. board-check.sh after EVERY merge. No CI polling — the ORCHESTRATOR watches CI.
• A flake is fixed by making the SETUP deterministic, never by weakening an assert or bumping a timeout.
• ALWAYS push with an explicit refspec (git push origin refs/heads/X:refs/heads/X) — the bare-refspec trap landed a branch on main twice in one day.
• 🔴 VOYAGER'S LOCAL main IS PERMANENTLY STALE — "branch from local main" SILENTLY BRANCHES FROM ANCIENT HISTORY THERE (caught 2026-08-02, PR #651 based on 654f158f, FOUR commits behind). Workers live in worktrees and nobody ever fast-forwards voyager's main, so CLAUDE.md's "branch from LOCAL main, never origin/main" — a rule written to protect UNPUSHED local commits — inverts into a bug on that host. The result is a CONFIRMED CONFLICTING PR, which runs NO CI AT ALL (zero runs, and gh pr checks reads "no checks reported", i.e. exactly like "not started yet" — a poller strands forever). The correct instruction, and it must be in EVERY dispatch brief: git fetch origin FIRST, verify git log origin/main..main is EMPTY (proving local main holds nothing unpushed — I check this myself, from the orchestrator, via ssh), THEN branch/rebase onto origin/main. 🥇 A rule's rationale, not its wording, decides whether it applies on a given host — check which of the two mains is actually ahead.
• | tail && echo OK MASKS the exit code — redirect to a file and capture $?.

🧷 ORCHESTRATOR TRAPS (mechanics of driving the panes)

• 🔴🔴 THE BLINDING, 2026-08-02 — the worst failure this skill has had, and the reason v3 exists. The orchestrator armed a wait-for-event.sh waiter and a CI poller in the same assistant message. The harness reaps both, so there was no listener at all — pgrep -fl 'wait-for-ev''ent.sh' returned nothing. The daemons kept writing events nobody read. Both workers were halted on questions addressed to the orchestrator — w2 for ~60 minutes, w1 for ~30 — while the orchestrator merged PRs and reported them as "building". vjt had to notice and say so. 🥇 The insight worth keeping: you cannot notice silence. A dead listener and a calm worker are the same observable — nothing. So never rely on "I'd have heard something by now". 🥇 The cure is structural, not vigilance: ONE Monitor with persistent: true on lib/monitor-stream.sh, armed once per session, covering every pane. No re-arm ⇒ nothing to forget. ⚠️ It is still not self-verifying: a monitor does not survive the orchestrator's /clear, and can be auto-stopped for volume — both silently. So re-arm it on every resume, and if both panes have seemed quiet for a stretch, prove the feed is alive instead of enjoying the calm.
• 🔴 Never background a waiter with & inside a foreground Bash — it detaches, advances the cursor and eats events. Arm ONLY via run_in_background: true, one per assistant message (two in one message = both killed, observed 3×). This is the legacy v2 path; prefer the Monitor above.
• 🔴 On resume, orphan waiters from the PRE-CLEAR session keep running and EAT EVENTS while notifying a dead session (their cmdline carries the old /tmp/claude-<id>-cwd). Kill them and re-arm fresh — cursor-tracking loses nothing. Verify with pgrep -fl 'wait-for-ev''ent.sh' (the unsplit pattern kills its own shell).
• 🔴 API Error: Stream idle timeout looks exactly like IDLE. Cure = a SHORT riprendi. — do not clear.
• 🔴 QUEUED INPUT ≠ SWALLOWED ≠ DELIVERED. Proof of delivery is a -S capture showing ❯ <text> as a TURN. ℹ️ A picker about LANES or a BRANCH BASE is addressed to ME; escalate only DESIGN/product pickers.
• 🔴 A worker's redirect log / rc file can belong to a DEAD run — ls -lat and match the mtime, never cat. Same for a staged /tmp/orchestrate-next-<w>.txt: stat it before dispatching, a stale body looks identical.
• 🔴 The harness's own "background command completed (exit code 0)" is the COMPOUND's last command, i.e. the trailing echo, NOT the gate's rc. Only a redirected rc FILE counts.
• 🔴 NEVER column-split gh pr checks — TAB-separated and the check name itself contains spaces (cicchetto + grappa + azzurra-testnet), so awk '{print $2}' returns + and a poller "settles" instantly. It has no --json; poll the run: gh run view <id> --json status,conclusion. 🥇 Key off a structured field, never off a column position.
• 🔴 gh needs a git repo to resolve the base repo — from a scratchpad dir it dies with "failed to determine base repo". Run from the repo, or pass -R vjt/grappa-irc.
• 🥇 A background gate SURVIVES /clear. DIAGNOSE-THEN-CLEAR-THEN-FIX when a worker hits 40% mid-debug on a red — a bare clear strands the next session on a red it must re-derive. The cheapest clear is the one taken while the worker is already blocked, at a boundary where its output is durable (pushed, or posted to the issue).
• 🥇 When a worker asks a question, answer it where it will be SEEN. vjt lives on IRC; check the bot log with a WIDE tail (tail -40, not -6) — a reply 10 lines back reads as "no reply" and idles a worker for nothing.
• ℹ️ The Pi has no git credential helper — git push --delete dies on "could not read Username"; prune remote branches with gh api -X DELETE.