| name | orchestrate |
| description | Coordinate multiple tickets in parallel across worktrees with wave-based execution, worker dispatch, and adversarial verification |
| disable-model-invocation | false |
| allowed-tools | Read, Write, Bash, Task, Grep, Glob, Agent |
| version | 1.0.0 |
Orchestrate
Coordinate multiple Linear tickets in parallel across git worktrees. The orchestrator creates
worktrees, dispatches /oneshot workers, tracks progress via a dashboard, and enforces quality
gates through adversarial verification. The orchestrator NEVER writes application code — it only
coordinates, monitors, and verifies.
Two dispatch modes. With the default catalyst.orchestration.dispatchMode: "oneshot-legacy",
each ticket gets one long claude -p /catalyst-dev:oneshot worker that runs the full lifecycle.
With dispatchMode: "phase-agents", the orchestrator dispatches nine short-lived claude --bg
phase skills (phase-triage … phase-monitor-deploy) per ticket, advancing on
phase.<name>.complete.<ticket> broker events. The phase-agents mode is the post-2026-06-15 path
that keeps worker dispatch on the subscription pool. See
Phase agents for the
pipeline, model assignment, cost economics, and the end-to-end runbook.
Prerequisites
source "${CLAUDE_PLUGIN_ROOT:-plugins/legacy}/scripts/require-catalyst-dev.sh" \
"${CLAUDE_PLUGIN_ROOT:-plugins/legacy}" || exit 1
if ! command -v git &>/dev/null; then
echo "ERROR: Git is required"
exit 1
fi
if ! command -v linearis &>/dev/null; then
echo "ERROR: Linearis CLI required for ticket intake"
echo "Install: npm install -g linearis"
exit 1
fi
if ! command -v gh &>/dev/null; then
echo "ERROR: GitHub CLI required for PR/CI monitoring"
exit 1
fi
if ! command -v claude &>/dev/null; then
echo "ERROR: Claude CLI required for worker dispatch"
exit 1
fi
if [[ -f "${CATALYST_DEV_SCRIPTS}/check-project-setup.sh" ]]; then
"${CATALYST_DEV_SCRIPTS}/check-project-setup.sh" || exit 1
fi
Invocation
/catalyst-dev:orchestrate PROJ-101 PROJ-102 PROJ-103 # explicit tickets
/catalyst-dev:orchestrate --project "Q2 API Redesign" # pull from Linear project
/catalyst-dev:orchestrate --cycle current # pull from current cycle
/catalyst-dev:orchestrate --file tickets.txt # read ticket IDs from file
/catalyst-dev:orchestrate --auto 5 # auto-pick top 5 Todo tickets
Flags
| Flag | Description |
|---|
--project <name> | Pull tickets from a Linear project |
--cycle current | Pull tickets from the current Linear cycle |
--file <path> | Read ticket IDs from a file (one per line) |
--auto <N> | Auto-pick top N Todo tickets: urgent/high priority first, newer first. Default N=3. |
--auto-merge | Workers auto-merge PRs when CI + verification pass |
--max-parallel <n> | Override config maxParallel (default: 3) |
--base-branch <branch> | Base branch for worktrees (default: main) |
--interactive | Include PM intake phase before orchestration |
--prd <path> | Run PRD review panel + ticket creation before orchestration |
--dry-run | Show wave plan without executing |
--state-on-merge <name> | Linear state to set on PR merge. Default: stateMap.done (typically "Done") |
--stop | Print how to deregister an execution-core project (edit the central registry) and exit. Works regardless of the configured dispatchMode. |
Configuration
Reads orchestration config from .catalyst/config.json (or .claude/config.json if .catalyst/
doesn't exist). Falls back to sensible defaults if no orchestration block exists.
{
"catalyst": {
"orchestration": {
"worktreeDir": null,
"maxParallel": 3,
"hooks": {
"setup": [],
"teardown": []
},
"workerCommand": "/catalyst-dev:oneshot",
"workerModel": "opus",
"dispatchMode": "phase-agents",
"thoughts": {
"profile": null,
"directory": null
},
"testRequirements": {
"backend": ["unit"],
"frontend": ["unit"],
"fullstack": ["unit"]
},
"verifyBeforeMerge": true,
"allowSelfReportedCompletion": false
}
}
}
dispatchMode (CTL-452):
"phase-agents" (default after Phase 6 lands) — the orchestrator dispatches 9 short-lived phase
agents per ticket via claude --bg (subscription pool). Phase 4 monitor subscribes a broker
phase_lifecycle interest per ticket and advances on
phase.<name>.{complete,skipped}.<TICKET> events via the orchestrate-phase-advance helper
(CTL-512: skipped is a monitor-deploy terminal-no-deploy status routed the same as complete)."oneshot-legacy" — the orchestrator dispatches one long claude -p oneshot worker per ticket.
Kept for rollback safety; flipping a single config key reverts to the pre-CTL-452 behavior."execution-core" (CTL-554, CTL-582) — daemon-served. /orchestrate runs no wave loop and no
Phase 4 session: it just ensures the single machine-level execution-core daemon is running and
exits. Enrolled projects are the central registry ~/catalyst/execution-core/registry.json
(maintained by setup-execution-core-states.sh); the daemon watches that file and serves each
team by composing the CTL-535 monitor, CTL-536 scheduler, and CTL-539 recovery modules.
The workerCommand field is still honored in legacy mode. In phase-agents mode it is unused (each
phase has its own canonical skill).
See config template for full schema documentation.
Core Principle: The Orchestrator Never Writes Code
The orchestrator operates ONLY from its own worktree. It:
- Creates/removes worker worktrees
- Dispatches worker sessions (via
claude CLI with streaming JSON output)
- Reads status from worker signal files and git
- Writes dashboard and briefing documents for the human
- Runs adversarial verification agents in worker worktrees
- Updates Linear ticket states when workers fail to do so
It NEVER:
- Modifies application code
- Writes test code
- Changes configuration in worker worktrees
- Commits to worker branches
Workflow
Phase 1: Intake & Dependency Analysis
-
Resolve tickets: Based on invocation mode, use the Linearis CLI to fetch ticket data. For
exact CLI syntax, run linearis issues usage or linearis cycles usage — do not guess.
- Explicit IDs: read each ticket's full details
--project: list issues filtered by project name--cycle current: list the active cycle, then list its issues--file: read IDs from file, then read each ticket's details--auto <N>: list status=Todo issues, then select the top N. Ranking: urgent/high priority
first (Linear priority 1 = Urgent → 4 = Low, with 0 = "No priority" sorted LAST), then newest
createdAt first. Example jq after linearis issues list --status Todo:
sort_by((if .priority == 0 then 5 else .priority end), (-(.createdAt | fromdateiso8601))) | .[:N]
Present the auto-picked tickets to the user as part of the wave plan before proceeding.
-
Read ticket details: For each ticket, extract:
- Title, description, estimate
- Dependencies (linked blocking/blocked-by issues)
- Labels (to detect scope: backend, frontend, fullstack)
-
Build dependency graph: Identify which tickets in the set depend on each other.
-
Group into waves:
- Wave 1: tickets with no dependencies on other tickets in the set
- Wave 2: tickets that depend only on Wave 1 tickets
- Wave N: tickets that depend on Wave N-1 tickets
- Tickets with circular dependencies → flag to user, do NOT proceed
-
Present wave plan for approval:
Orchestration Plan — "api-redesign"
Total: 6 tickets | 3 waves | Max parallel: 3
Wave 1 (parallel, 3 workers):
PROJ-101: Auth middleware rewrite [backend, 3pt]
PROJ-102: Rate limiting service [backend, 2pt]
PROJ-103: Email templates [frontend, 1pt]
Wave 2 (after Wave 1, 2 workers):
PROJ-104: OAuth integration [fullstack, 5pt] — depends on PROJ-101
PROJ-105: API usage dashboard [frontend, 3pt] — depends on PROJ-102
Wave 3 (after Wave 2, 1 worker):
PROJ-106: Self-service API keys [fullstack, 5pt] — depends on PROJ-104, PROJ-105
Estimated waves: 3 sequential rounds
Proceed? [Y/n]
- If
--dry-run: Print wave plan and exit.
Phase 2: Provision Worktrees
Determine worktree base directory (in priority order):
catalyst.orchestration.worktreeDir from config${GITHUB_SOURCE_ROOT}/<org>/<repo>-worktrees/ (from env var + git remote)~/wt/<repo>/ (fallback)
Read config for orchestration settings:
CONFIG_FILE=""
for CFG in ".catalyst/config.json" ".claude/config.json"; do
if [ -f "$CFG" ]; then CONFIG_FILE="$CFG"; break; fi
done
WORKTREE_DIR=$(jq -r '.catalyst.orchestration.worktreeDir // empty' "$CONFIG_FILE" 2>/dev/null)
MAX_PARALLEL=$(jq -r '.catalyst.orchestration.maxParallel // 3' "$CONFIG_FILE" 2>/dev/null)
SETUP_HOOKS=$(jq -c '.catalyst.orchestration.hooks.setup // []' "$CONFIG_FILE" 2>/dev/null)
TEARDOWN_HOOKS=$(jq -c '.catalyst.orchestration.hooks.teardown // []' "$CONFIG_FILE" 2>/dev/null)
WORKER_COMMAND=$(jq -r '.catalyst.orchestration.workerCommand // "/catalyst-dev:oneshot"' "$CONFIG_FILE" 2>/dev/null)
if [[ ! "$WORKER_COMMAND" =~ ^/[a-z][a-z0-9_-]*:[a-z][a-z0-9_-]*$ ]]; then
echo "ERROR: catalyst.orchestration.workerCommand=\"$WORKER_COMMAND\" must be plugin-namespaced (/<plugin>:<skill>), e.g. /catalyst-dev:oneshot. Update $CONFIG_FILE." >&2
exit 2
fi
WORKER_MODEL=$(jq -r '.catalyst.orchestration.workerModel // "opus"' "$CONFIG_FILE" 2>/dev/null)
VERIFY_BEFORE_MERGE=$(jq -r '.catalyst.orchestration.verifyBeforeMerge // "true"' "$CONFIG_FILE" 2>/dev/null)
ALLOW_SELF_REPORTED=$(jq -r '.catalyst.orchestration.allowSelfReportedCompletion // "false"' "$CONFIG_FILE" 2>/dev/null)
DISPATCH_MODE=$(jq -r '.catalyst.orchestration.dispatchMode // "oneshot-legacy"' "$CONFIG_FILE" 2>/dev/null)
Linear App-Actor Identity (CTL-550, CTL-749)
catalyst.monitor.linear.botUserId is required for the Linear app-actor comms channel — i.e.
when the execution-core daemon mirrors phase-agent output to Linear and wakes on human replies
(CTL-550 / CTL-549 / CTL-749). It must be set before the daemon starts. The daemon reads it
from the project's Layer-1 committed config at the flat path
<project>/.catalyst/config.json → catalyst.monitor.linear.botUserId (not the Layer-2 home
config, not a team-keyed sub-object). This is the Linear user UUID of the Catalyst app-actor (the
"Linear for Agents" app user) — workspace-specific, and null in the committed config template.
See /catalyst-foundry:setup-catalyst for how to obtain it (query viewer.id with the app-actor
token). (The execution-core daemon is the pull-based runner of the phase-agent pipeline; it is not
a dispatchMode enum value — those are phase-agents and oneshot-legacy.)
The daemon reads botUserId only at startup and uses it to filter the agent's own mirror activity
out of the bidirectional Linear comms channel (CTL-749/CTL-549): it suppresses bot-authored comments
and description updates so they are not written into workers/<ticket>/inbox.jsonl as input. Without
it, the daemon cannot tell the agent's own comments/updates apart from a human's, so every mirror
comment lands in the worker inbox as a false "human replied" signal (noise / write loops). It is
the self-echo / loop-prevention guard.
Create ALL worktrees using create-worktree.sh — both orchestrator and workers go through the
same script so they all get .claude/, .catalyst/, dependency install, thoughts init, and custom
hooks:
SCRIPT="${CATALYST_DEV_SCRIPTS}/create-worktree.sh"
WT_DIR_FLAG=""
if [ -n "$WORKTREE_DIR" ]; then
WT_DIR_FLAG="--worktree-dir ${WORKTREE_DIR}"
fi
HOOKS_FLAG=""
if [ "$SETUP_HOOKS" != "[]" ] && [ -n "$SETUP_HOOKS" ]; then
HOOKS_FLAG="--hooks-json '${SETUP_HOOKS}'"
fi
ORCH_FLAG="--orchestration ${ORCH_NAME}"
"$SCRIPT" "${ORCH_NAME}" "${BASE_BRANCH}" ${WT_DIR_FLAG} ${HOOKS_FLAG} ${ORCH_FLAG}
ORCH_WORKTREE="${WORKTREES_BASE}/${ORCH_NAME}"
ORCH_DIR="$("${CATALYST_DEV_SCRIPTS}/catalyst-state.sh" ensure-run-dir "${ORCH_NAME}")"
for TICKET_ID in "${WAVE_TICKETS[@]}"; do
"$SCRIPT" "${ORCH_NAME}-${TICKET_ID}" "${BASE_BRANCH}" ${WT_DIR_FLAG} ${HOOKS_FLAG} ${ORCH_FLAG}
done
Where worktrees actually land — the create-worktree.sh script resolves the base directory in
this priority order:
--worktree-dir <path> flag (from catalyst.orchestration.worktreeDir config)~/catalyst/wt/<projectKey>/ (default — reads catalyst.projectKey from config)~/catalyst/wt/<repo>/ (fallback if no config)
So for a project with projectKey: "acme" and no worktreeDir override, all worktrees land in:
~/catalyst/wt/acme/
├── api-redesign/ # orchestrator
├── api-redesign-ACME-101/ # worker
├── api-redesign-ACME-102/ # worker
└── api-redesign-ACME-103/ # worker
With worktreeDir: "~/catalyst/api" explicitly configured:
~/catalyst/api/
├── api-redesign/ # orchestrator
├── api-redesign-ACME-101/ # worker
├── api-redesign-ACME-102/ # worker
└── api-redesign-ACME-103/ # worker
Recommended: Add ~/catalyst to Claude Code's additionalDirectories in
~/.claude/settings.json so all worktrees across projects are automatically trusted:
{
"permissions": {
"additionalDirectories": ["/Users/you/catalyst"]
}
}
What create-worktree.sh does for EACH worktree (orchestrator and workers alike):
git worktree add -b <name> <path> <base-branch> — creates the worktree- Copies
.claude/ directory (Claude Code native config, plugins, rules)
- Copies
.catalyst/ directory (Catalyst workflow config, if it exists)
- Runs
catalyst.worktree.setup commands from config — dependency install, thoughts init,
permission grants, or any project-specific setup (like a worktree-manager's lifecycle
hooks)
- If no
catalyst.worktree.setup configured, falls back to auto-detected setup: make setup or
bun/npm install, then humanlayer thoughts init + sync
- Runs additional orchestration hooks from
--hooks-json (from
catalyst.orchestration.hooks.setup)
Available variables in setup commands: ${WORKTREE_PATH}, ${BRANCH_NAME}, ${TICKET_ID},
${REPO_NAME}, ${DIRECTORY}, ${PROFILE}
After worktree creation, set up the orchestrator's status directory:
mkdir -p "${ORCH_DIR}/workers/output"
Render the initial DASHBOARD.md (CTL-230) — the renderer reads state.json + workers/*.json
signals + the events log every cycle, so calling it now produces a real header + empty worker
table + waves outline rather than a template skeleton:
"${CATALYST_DEV_SCRIPTS}/update-dashboard.sh" \
--orch "${ORCH_NAME}" --orch-dir "${ORCH_DIR}" --roll-usage \
>/dev/null 2>>"${ORCH_DIR}/.update-dashboard.log" || true
--roll-usage is the wire that fulfils the "Worker usage / cost is rolled in by the monitor pass"
contract below (CTL-487): the dashboard helper iterates ${ORCH_DIR}/workers/*.json and invokes
orchestrate-roll-usage.sh -v per worker before rendering, with stderr captured to
${ORCH_DIR}/.roll-usage.log. The per-worker call is bounded — already-rolled workers short-circuit
on signal.cost != null and cost a single jq read.
Create the orchestrator's status directory:
${ORCH_DIR}/ # ~/catalyst/runs/${ORCH_NAME}/
├── DASHBOARD.md # human-readable status (re-rendered each Phase 4 cycle)
├── state.json # machine-readable orchestration state
├── wave-1-briefing.md # per-wave briefings
├── SUMMARY.md # final run summary (post-Phase 5)
└── workers/
├── ${TICKET_1}.json # worker signal (schema: worker-signal.json)
├── ${TICKET_2}.json
└── output/ # claude CLI output (streams, stderr)
├── ${TICKET_1}-stream.jsonl # streaming JSON events from claude
├── ${TICKET_1}-stderr.log # worker stderr (silent exits diagnosable)
├── ${TICKET_2}-stream.jsonl
└── ${TICKET_2}-stderr.log
Note on the runs/ split (CTL-59): ORCH_DIR lives at ~/catalyst/runs/${ORCH_NAME}/ and is
decoupled from the git worktree at ${ORCH_WORKTREE} (e.g.
~/catalyst/wt/${PROJECT_KEY}/${ORCH_NAME}/). This lets state survive worktree cleanup and keeps
git status clean. Claude CLI output (stream + stderr) lands in workers/output/ to keep file
watchers that scan workers/*.json free of noise from large stream files.
Debugging silent worker exits: If workers/output/${TICKET_ID}-stream.jsonl is 0 bytes AND
workers/output/${TICKET_ID}-stderr.log is 0 bytes, the worker exited before emitting its first
event — check git -C ${WORKER_DIR} log --oneline -5 and the worktree's .claude/ directory for
setup issues. A non-empty stderr log will identify permission, path, or environment errors.
Initialize state.json:
{
"orchestrator": "<name>",
"startedAt": "<ISO timestamp>",
"baseBranch": "main",
"totalTickets": 6,
"totalWaves": 3,
"currentWave": 1,
"worktreeBase": "<path>",
"waves": [
{
"wave": 1,
"status": "provisioning",
"tickets": ["PROJ-101", "PROJ-102", "PROJ-103"]
},
{
"wave": 2,
"status": "blocked",
"tickets": ["PROJ-104", "PROJ-105"],
"dependsOn": [1]
}
],
"workers": {}
}
Register with global state (immediately after local state initialization):
STATE_SCRIPT="${CATALYST_DEV_SCRIPTS}/catalyst-state.sh"
COMMS_BIN="${CATALYST_DEV_SCRIPTS}/catalyst-comms"
[ -x "$COMMS_BIN" ] || COMMS_BIN="$(command -v catalyst-comms 2>/dev/null || true)"
if [ -z "$COMMS_BIN" ] || [ ! -x "$COMMS_BIN" ]; then
echo "warn: catalyst-comms not found — comms disabled (install: plugins/dev/scripts/install-cli.sh)" >&2
COMMS_BIN=""
fi
source "${CATALYST_DEV_SCRIPTS}/lib/linear-read-replica.sh"
WORKERS_JSON="{}"
for TICKET_ID in "${ALL_TICKETS[@]}"; do
TITLE=$(linear_read_ticket "$TICKET_ID" | jq -r '.title')
WORKERS_JSON=$(echo "$WORKERS_JSON" | jq \
--arg tid "$TICKET_ID" --arg title "$TITLE" \
'. + {($tid): {ticketId: $tid, title: $title, status: "dispatched", phase: 0, branch: null, pr: null, updatedAt: "'$(date -u +%Y-%m-%dT%H:%M:%SZ)'", needsAttention: false, attentionReason: null}}')
done
REPO=$(git remote get-url origin 2>/dev/null | sed 's|.*github.com[:/]||;s|\.git$||')
"$STATE_SCRIPT" register "${ORCH_NAME}" "$(jq -nc \
--arg id "${ORCH_NAME}" \
--arg pk "$(jq -r '.catalyst.projectKey // "unknown"' "$CONFIG_FILE")" \
--arg repo "$REPO" \
--arg bb "${BASE_BRANCH}" \
--arg wtd "${ORCH_DIR}" \
--arg sf "${ORCH_DIR}/state.json" \
--argjson total "${#ALL_TICKETS[@]}" \
--argjson waves "$TOTAL_WAVES" \
--argjson workers "$WORKERS_JSON" \
'{
id: $id, projectKey: $pk, repository: $repo, baseBranch: $bb,
status: "active", startedAt: (now | strftime("%Y-%m-%dT%H:%M:%SZ")),
worktreeDir: $wtd, stateFile: $sf,
progress: {totalTickets: $total, completedTickets: 0, failedTickets: 0, inProgressTickets: 0, currentWave: 1, totalWaves: $waves},
usage: {inputTokens: 0, outputTokens: 0, cacheReadTokens: 0, cacheCreationTokens: 0, costUSD: 0, numTurns: 0, durationMs: 0, durationApiMs: 0, model: null},
workers: $workers, attention: []
}')"
The CATALYST_ORCHESTRATOR_ID is set to ${ORCH_NAME} for use by workers (passed via environment
variable alongside CATALYST_ORCHESTRATOR_DIR).
Capture event-log baseline (CTL-491): Before any worker dispatch, snapshot the catalyst event
log's current line count and file path. The Phase 4 replay step uses this to find any
phase.*.{complete,failed,turn-cap-exhausted,skipped}.<TICKET> events that landed during the
dispatch-to-monitor handoff. The new state.json.race object is opaque to older state.json
consumers (they ignore unknown top-level fields via .field // default jq patterns).
EVENTS_DIR="${CATALYST_DIR:-$HOME/catalyst}/events"
mkdir -p "$EVENTS_DIR"
BASELINE_FILE="${EVENTS_DIR}/$(date -u +%Y-%m).jsonl"
[[ -f "$BASELINE_FILE" ]] || : > "$BASELINE_FILE"
BASELINE_LINE=$(wc -l < "$BASELINE_FILE" | tr -d ' ')
TMP="${ORCH_DIR}/state.json.tmp.$$"
jq --arg cursor "$BASELINE_LINE" --arg file "$BASELINE_FILE" --arg ts "$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
'.race = {startLineCursor: ($cursor | tonumber), startEventsFile: $file, capturedAt: $ts}' \
"${ORCH_DIR}/state.json" > "$TMP" \
&& mv "$TMP" "${ORCH_DIR}/state.json" || rm -f "$TMP"
Create the shared comms channel (CTL-111): the orchestrator creates a file-based channel that
every worker will auto-join via CATALYST_COMMS_CHANNEL in its dispatch env. Best-effort — the
orchestrator does not crash if catalyst-comms is missing.
if [ -n "$COMMS_BIN" ]; then
"$COMMS_BIN" join "${ORCH_NAME}" \
--as orchestrator \
--capabilities "coordinates workers" \
--orch "${ORCH_NAME}" \
--ttl 7200 >/dev/null 2>&1 || true
fi
Start session tracking (alongside the global state registration above):
SESSION_SCRIPT="${CATALYST_DEV_SCRIPTS}/catalyst-session.sh"
ORCH_STATUS_SCRIPT="${CATALYST_DEV_SCRIPTS}/orchestrate-status.sh"
if [[ -x "$SESSION_SCRIPT" ]]; then
CATALYST_SESSION_ID=$("$SESSION_SCRIPT" start --skill "orchestrate" \
--label "${ORCH_NAME}" \
--workflow "${CATALYST_SESSION_ID:-}")
export CATALYST_SESSION_ID
"$SESSION_SCRIPT" phase "$CATALYST_SESSION_ID" "dispatching" --phase 3
fi
execution-core dispatch fork (CTL-554, CTL-582)
dispatchMode: "execution-core" and the --stop flag both short-circuit the wave loop. Evaluate
this before Phase 3 — when it applies, no workers are dispatched and no Phase 4 monitor session
starts.
CTL-582 (D4) made the central registry ~/catalyst/execution-core/registry.json the single source
of enrolled projects, maintained by setup-execution-core-states.sh. /orchestrate no longer
writes per-project enrollment records — there is nothing to enroll here.
-
Invoked with --stop — execution-core projects are deregistered by editing the central
registry, not by /orchestrate. Print the guidance and exit 0:
echo "execution-core: enrolled projects are the central registry" \
"~/catalyst/execution-core/registry.json — remove the team's entry there to deregister."
-
DISPATCH_MODE is execution-core (no --stop) — ensure the machine-level daemon is
running, then exit:
"${CATALYST_DEV_SCRIPTS}/orchestrate-execution-core-route.sh"
-
Any other DISPATCH_MODE — continue to Phase 3 below.
Phase 3: Dispatch Workers
For each provisioned worker worktree, dispatch a /oneshot session.
Register broker filter interests BEFORE dispatch (CTL-491): Phase-agent workers can finish in
well under a second when their work is a no-op (e.g. an already-triaged ticket). Emitting
filter.register events AFTER dispatch opens a race window where the broker's interest map is still
empty when phase.<name>.complete.<TICKET> arrives — processEvent early-returns and the event is
dropped silently (broker/index.mjs:1782). Run the registration helper here so all four
deterministic + per-ticket interests are durable in the broker BEFORE any claude --bg invocation.
Idempotent at the broker (upserts by interest_id); the Phase 4 entry re-invokes the same helper as
a belt-and-suspenders.
"${CATALYST_DEV_SCRIPTS}/orchestrate-register-interests.sh" \
--orch-dir "${ORCH_DIR}" \
--orch-id "${ORCH_NAME}" \
--config "${REPO_ROOT:-$(git rev-parse --show-toplevel 2>/dev/null || echo .)}/.catalyst/config.json"
Emit dispatching status (CTL-405): Before launching workers, announce the phase:
TOTAL_WORKERS=$(jq -r '.progress.totalTickets // 0' "${ORCH_DIR}/state.json" 2>/dev/null || echo 0)
[[ -x "$ORCH_STATUS_SCRIPT" ]] && "$ORCH_STATUS_SCRIPT" emit \
--orch "${ORCH_NAME}" --phase dispatching --wave "${CURRENT_WAVE:-1}" \
--total "$TOTAL_WORKERS" \
--summary "wave ${CURRENT_WAVE:-1} dispatching" 2>/dev/null || true
Preferred entrypoint — orchestrate-dispatch-next (CTL-116):
The canonical dispatcher drains state.json's .queue.waveNPending for every N (dynamically, so
wave 1/2/3/…/N all work without code changes), respects maxParallel - currentlyRunning, writes
dispatched/phase-0 signal files, launches workers via nohup, updates global state, removes
dispatched tickets from whichever waveNPending list they lived in, and runs the post-dispatch
healthcheck:
"${CATALYST_DEV_SCRIPTS}/orchestrate-dispatch-next" \
--orch-dir "${ORCH_DIR}" \
--orch-id "${ORCH_NAME}" \
--config "${REPO_ROOT}/.catalyst/config.json"
Call this once when the current wave is ready to dispatch, and again whenever a worker slot frees
up. It supersedes the hand-rolled dispatch-next.sh pattern from pre-CTL-116 orchestration runs
(which hardcoded wave1Pending + wave2Pending + wave3Pending). The inline block below is preserved
as reference for the underlying machinery.
Phase advancement on phase.<name>.complete.<TICKET> wake (CTL-452):
"${CATALYST_DEV_SCRIPTS}/orchestrate-phase-advance" \
--orch-dir "${ORCH_DIR}" \
--orch-id "${ORCH_NAME}" \
--ticket "${TICKET}" \
--completed-phase "${COMPLETED_PHASE}"
Phase failure on phase.<name>.failed.<TICKET> wake (CTL-452):
"${CATALYST_DEV_SCRIPTS}/orchestrate-revive" \
--orch-dir "${ORCH_DIR}" \
--orch-id "${ORCH_NAME}"
Phase turn-cap exhaustion on phase.<name>.turn-cap-exhausted.<TICKET> wake (CTL-484):
"${CATALYST_DEV_SCRIPTS}/orchestrate-revive" \
--orch-dir "${ORCH_DIR}" \
--orch-id "${ORCH_NAME}"
Dispatch mechanism — claude CLI with streaming JSON:
WORKER_DIR="${WORKTREES_BASE}/${ORCH_NAME}-${TICKET_ID}"
WORKER_STREAM="${ORCH_DIR}/workers/output/${TICKET_ID}-stream.jsonl"
WORKER_STDERR="${ORCH_DIR}/workers/output/${TICKET_ID}-stderr.log"
SIGNAL_FILE="${ORCH_DIR}/workers/${TICKET_ID}.json"
(
cd "${WORKER_DIR}" || exit 1
CATALYST_ORCHESTRATOR_DIR="${ORCH_DIR}" \
CATALYST_ORCHESTRATOR_ID="${ORCH_NAME}" \
CATALYST_COMMS_CHANNEL="${ORCH_NAME}" \
CATALYST_SESSION_ID="${CATALYST_SESSION_ID:-}" \
exec nohup claude \
-n "${ORCH_NAME}-${TICKET_ID}" \
--output-format stream-json \
--verbose \
--dangerously-skip-permissions \
-p "${WORKER_COMMAND} ${TICKET_ID} --auto-merge"
) > "$WORKER_STREAM" 2> "$WORKER_STDERR" &
WORKER_PID=$!
if [ -f "$SIGNAL_FILE" ]; then
jq --argjson pid "$WORKER_PID" '.pid = $pid | .lastHeartbeat = .updatedAt' \
"$SIGNAL_FILE" > "${SIGNAL_FILE}.tmp" && mv "${SIGNAL_FILE}.tmp" "$SIGNAL_FILE"
fi
Streaming JSON output (--output-format stream-json --verbose) emits NDJSON to stdout, one
event per line, in real-time as the worker runs. The monitor can tail the stream file to show live
worker activity. Key event types:
| Event | What it signals |
|---|
{"type":"system","subtype":"init"} | Worker session started; contains session_id |
{"type":"stream_event","event":{"type":"content_block_start","content_block":{"type":"tool_use","name":"..."}}} | Worker is now invoking a specific tool (Bash, Read, Edit, etc.) |
{"type":"stream_event","event":{"type":"content_block_delta","delta":{"type":"text_delta"}}} | Worker is generating reasoning/response text |
{"type":"assistant"} | Complete assistant turn with all content blocks |
{"type":"system","subtype":"api_retry"} | Worker hit rate limit / error; shows attempt and delay |
{"type":"result"} | Worker finished; contains final answer and usage stats |
Worker usage / cost is rolled in by the monitor pass (CTL-115, wired through
update-dashboard.sh --roll-usage since CTL-487). The dispatch shell backgrounds workers with &
and never waits on them, so usage/cost extraction cannot happen here. Phase 4's
update-dashboard.sh call passes --roll-usage, which iterates ${ORCH_DIR}/workers/*.json and
invokes plugins/dev/scripts/orchestrate-roll-usage.sh -v per worker before rendering. The helper
parses the final result event from the worker's stream file and:
- Mirror cost into the worker's signal file (
.cost = USAGE) — the dashboard reads signal files
(not global state) for per-worker cost columns.
- Write
state.workers[ticket].usage.
- Roll the delta into
state.usage for the orchestrator-level aggregate.
The helper is idempotent (gated on signal.cost == null) and safe to call every cycle. Because
every dashboard render now sweeps every worker, update-dashboard.sh --roll-usage doubles as the
periodic safety net for any worker whose stream contains a result event but whose signal.cost is
still null. Audit trail lands at ${ORCH_DIR}/.roll-usage.log.
Emit dispatch event and update global state after each worker dispatch:
"$STATE_SCRIPT" worker "${ORCH_NAME}" "${TICKET_ID}" '.status = "dispatched" | .phase = 0'
"$STATE_SCRIPT" update "${ORCH_NAME}" '.progress.inProgressTickets += 1'
"$STATE_SCRIPT" event "$(jq -nc \
--arg ts "$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
--arg orch "${ORCH_NAME}" --arg w "${TICKET_ID}" \
'{ts: $ts, orchestrator: $orch, worker: $w, event: "worker-dispatched", detail: null}')"
Post-dispatch health check (CTL-87):
After the wave's per-worker dispatch loop has completed, run the batch health check. This is the
per-wave invocation (CTL-511 added a second call site — see the reactive scan in Phase 4). It
sleeps briefly (default 15s — configurable via --grace-seconds), then verifies that every worker
still sitting at status="dispatched"/phase=0 has a live PID. Any worker whose PID has already
died is transitioned to status="failed" with failureReason="launch-failure", an attention item
of type launch-failure is raised, and a worker-launch-failed event is emitted. This means
dead-on-arrival workers surface in under 30 seconds instead of after the 15-minute stalled-worker
timeout, and the orchestrator can re-dispatch them (via orchestrate-fixup or a manual redispatch)
in the same wave.
"${CATALYST_DEV_SCRIPTS}/orchestrate-healthcheck" \
--orch-dir "${ORCH_DIR}" \
--orch-id "${ORCH_NAME}"
Healthy workers are untouched. Workers that have already advanced past dispatched (e.g. into
researching) are skipped because reaching a later status is itself proof of life. This check
complements the 15-minute stalled-worker detection in Phase 4 — healthcheck catches launch failures,
the stalled-worker scan catches workers that die mid-run.
CTL-511: the same orchestrate-healthcheck script also runs inside the reactive scan (Phase 4)
on every wake-up and every 10-minute idle tick — it is no longer a once-per-wave-only check. That
second call site catches phase agents that die after launch (printing a job id, then exiting
before their terminal emit), bounding detection to one scan interval. The healthcheck is idempotent
— terminal and stalled signals are skipped — so the extra call site adds no duplicate work.
Worker dispatch prompt includes mandatory testing AND lifecycle requirements:
MANDATORY: Before completing your contract:
1. TDD — write failing tests BEFORE implementation for every feature
2. Unit tests — required for all new functions/methods
3. Integration/API tests — required for every new/modified API endpoint
4. Security review — must pass /security-review or equivalent
5. Code review — must pass code-reviewer agent
6. All quality gates in config must pass
Your success contract ENDS at (CTL-252):
✓ PR open (gh pr create succeeded)
✓ Active listen loop resolved all blockers (CI, bot reviews, BEHIND) inline
✓ PR merged with gh pr merge --squash --delete-branch (no --auto)
✓ Optional deployment verified (if catalyst.deploy configured)
✓ pr.mergedAt, deployment.url (if applicable), status="done" written to signal file
✓ Worker process exits cleanly
Listen for PR events using this precedence ladder (matches oneshot Phase 5):
1. PREFERRED — Broker auto-correlation (Pattern 3, lowest cost):
The catalyst-broker daemon classifies events between Claude turns and
emits filter.wake.${CATALYST_SESSION_ID} only on semantic matches.
The agent.checkin event emitted by catalyst-session.sh start already
identifies you to the broker; emit a second agent.checkin carrying
claimed_pr after gh pr create (the oneshot helper broker_claim_pr does
exactly this). The broker auto-derives a pr_lifecycle interest covering
CI, reviews, comments, thread resolution, merge, BEHIND pushes, and
deployment status — all on a single wake event. Wait on:
catalyst-events wait-for \
--filter ".attributes.\"event.name\" == \"filter.wake.${CATALYST_SESSION_ID}\"" \
--timeout 600
Deterministic routes (pr_lifecycle / ticket_lifecycle / comms_lifecycle)
cost zero LLM tokens. See plugins/dev/skills/broker/SKILL.md for the
full protocol and plugins/dev/skills/catalyst-filter/SKILL.md for
prose-based registration when the deterministic routes aren't enough.
2. FALLBACK — catalyst-events wait-for with explicit jq (Pattern 2):
When catalyst-broker status returns non-running, use the two-phase
pattern from plugins/dev/skills/wait-for-github/SKILL.md. Blocking
subprocess call; works reliably in claude -p non-interactive sessions.
One jq predicate covers CI / reviews / push / merge for your PR.
3. FORBIDDEN for workers — Monitor over catalyst-events tail (Pattern 1):
That is the orchestrator's Phase 4 pattern, not a worker pattern.
A short-lived claude -p worker has no long-lived turn loop to consume
Monitor notifications, and tail filters wake on every matching line,
burning context.
NEVER use gh pr view --json in any loop — that burns GraphQL rate limits.
Use gh api REST endpoints only for the authoritative state check that
follows every wake.
WAKE NARRATION (MANDATORY, CTL-369): every Monitor wake and every wait-for
return must be acknowledged with a single short line of assistant text before
returning to the wait. A thinking-only end_turn after a Monitor wake makes the
harness's next <task-notification> XML leak as a confusing "Human:\n<task-id>"
phantom user message. The line shape is:
wake: <event.name> #<pr> [interest=<type>] — <action being taken>
wake: <event.name> — routine, staying in event loop
wake: <event.name> — already addressed, no-op
Include the matched filter clause / interest_id when the wake is from the
broker (.body.payload.interest_id, .body.payload.reason). See
plugins/dev/skills/monitor-events/SKILL.md § Narration for the full rule.
When you write a filter by hand, target the canonical event names listed in
[[event-name-allowlist]] — anything outside that allowlist is either
non-actionable or covered by a different lifecycle group. Do NOT use
`.attributes."catalyst.orchestrator.id"` as a bare clause: CTL-234 stamps it
on every github webhook tied to one of the orchestrator's PRs, so without an
event-type guard it wakes the worker on 60-70% of unrelated webhooks. See
[[wait-for-github]] § Known filter pitfalls.
Write these fields into your signal file as they become available:
pr.number
pr.url
pr.prOpenedAt (ISO timestamp when gh pr create returned)
pr.mergedAt (ISO timestamp when merge confirmed via REST)
pr.mergeCommitSha (from REST response after merge)
pr.ciStatus (pending → merged)
deployment.url (from deployment_status event, if configured)
status (pr-created → done)
On unrecoverable blockers (human changes-requested, DIRTY after attempts,
CI blocked after 3 fix attempts), write status="stalled" with details and
post a `comms attention` message — the orchestrator's Phase 4 dispatches
remediation for stalled workers.
Status transitions you do NOT write (orchestrator-owned fallback only):
done (written by orchestrator Phase 4 ONLY when worker stalled before merge)
COMMS DISCIPLINE: when posting to the shared comms channel, follow the rules in the
catalyst-comms skill (plugins/dev/skills/catalyst-comms/SKILL.md § Posting Discipline):
- info = phase transitions + PR-opened only (default heartbeat, ~5-7 per session)
- attention = orchestrator action required (0-2 per session, MANDATORY on: scope
conflict, missing access, ambiguous spec, 3+ repeated CI failures, status=stalled)
- done = exactly 1, only via the `done` subcommand at terminal success
- never use attention as a heartbeat — it triggers the orchestrator's NEEDS ATTENTION
banner
Write your status to the worker signal file at:
${ORCH_DIR}/workers/${TICKET_ID}.json
Update the signal file at each phase transition using the worker-signal.json schema.
Initialize worker signal file (orchestrator writes the initial state):
{
"ticket": "PROJ-101",
"orchestrator": "<name>",
"workerName": "<orch-name>-PROJ-101",
"label": "oneshot PROJ-101",
"status": "dispatched",
"phase": 0,
"startedAt": "<ISO timestamp>",
"updatedAt": "<ISO timestamp>",
"worktreePath": "<path>",
"pr": null,
"linearState": null,
"definitionOfDone": {
"testsWrittenFirst": false,
"unitTests": { "exists": false, "count": 0 },
"apiTests": { "exists": false, "count": 0 },
"functionalTests": { "exists": false, "count": 0 },
"typeCheck": { "passed": false },
"securityReview": { "passed": false },
"codeReview": { "passed": false },
"rewardHackingScan": { "passed": false }
}
}
Phase 4: Monitor & Track
if [[ -n "${CATALYST_SESSION_ID:-}" && -x "$SESSION_SCRIPT" ]]; then
"$SESSION_SCRIPT" phase "$CATALYST_SESSION_ID" "monitoring" --phase 4
fi
ACTIVE_COUNT=$(jq -rs '[.[] | select(.status != "done" and .status != "failed")] | length' \
"${ORCH_DIR}/workers/"*.json 2>/dev/null || echo 0)
TOTAL_COUNT=$(jq -rs 'length' "${ORCH_DIR}/workers/"*.json 2>/dev/null || echo 0)
[[ -x "$ORCH_STATUS_SCRIPT" ]] && "$ORCH_STATUS_SCRIPT" emit \
--orch "${ORCH_NAME}" --phase monitoring --wave "${CURRENT_WAVE:-1}" \
--active "$ACTIVE_COUNT" --total "$TOTAL_COUNT" \
--summary "wave ${CURRENT_WAVE:-1} monitoring (${ACTIVE_COUNT}/${TOTAL_COUNT} active)" 2>/dev/null || true
Re-register with catalyst-broker daemon (CTL-257, CTL-303, CTL-357, CTL-452, CTL-491): Phase 3
already invoked orchestrate-register-interests.sh BEFORE worker dispatch (the CTL-491 race-fix
hoist). This Phase 4 entry calls the same helper again as a belt-and-suspenders — registration is
idempotent at the broker (upserts by interest_id), and a second invocation ensures the four
interests stay fresh even if the broker restarted between Phase 3 and Phase 4.
The helper emits four deterministic interests (route without Groq):
pr_lifecycle — orchestrator-level aggregation across all worker PRs.ticket_lifecycle — Linear ticket state changes for the orchestrator's tickets.comms_lifecycle — worker-posted attention / done messages on the shared channel.phase_lifecycle (CTL-452, CTL-484, CTL-512) — phase.<name>.complete.<TICKET>,
phase.<name>.failed.<TICKET>, phase.<name>.turn-cap-exhausted.<TICKET>, and
phase.<name>.skipped.<TICKET> events emitted by phase agents. One interest per active ticket,
covering all 9 phase names. The turn-cap status routes through orchestrate-revive's continuation
branch (separate budget from error revives). The skipped status (CTL-512, monitor-deploy
terminal-no-deploy) routes the same as complete via orchestrate-phase-advance — advance no-ops
on monitor-deploy, so the wake's purpose is to free the wave slot via the scheduler's in-flight
predicate. Only registered when catalyst.orchestration.dispatchMode = "phase-agents".
CTL-357 retired the Groq prose interest (~95% false-positive rate). All four interests share the
same notify_event: "filter.wake.${ORCH_NAME}", so the orchestrator's wait-for filter does not
change.
"${CATALYST_DEV_SCRIPTS}/orchestrate-register-interests.sh" \
--orch-dir "${ORCH_DIR}" \
--orch-id "${ORCH_NAME}" \
--config "${REPO_ROOT:-$(git rev-parse --show-toplevel 2>/dev/null || echo .)}/.catalyst/config.json"
Replay race-window phase events (CTL-491): Before entering the event-driven wait loop, catch up
on any phase.<name>.{complete,failed,turn-cap-exhausted,skipped}.<TICKET> events that landed
between the Phase 2 baseline (state.json.race.startLineCursor) and now. With the Phase 3
pre-dispatch registration the window should be zero-length in steady state, but the replay is
idempotent and cheap — it scans only the tail of the current month's event log past the baseline
cursor and routes each match through orchestrate-phase-advance (for complete / skipped) or
orchestrate-revive (for failed / turn-cap-exhausted). Cross-orchestrator events are filtered out by checking
workers/*.json for the ticket.
"${CATALYST_DEV_SCRIPTS}/orchestrate-replay-phase-events.sh" \
--orch-dir "${ORCH_DIR}" \
--orch-id "${ORCH_NAME}" || true
Phase 4 is event-driven, not poll-driven (CTL-210, CTL-243). The orchestrator subscribes to the
unified event log via catalyst-events tail (wrapped in the Monitor tool) and wakes on every
relevant GitHub / Linear / orchestrator-lifecycle event. A 10-minute idle timer is the safety-net
fallback for daemon-down or missed-event scenarios — never the primary mechanism. Do NOT self-pace
with sleeps or "wake in N minutes" framing — that defeats the event-driven contract and burns
context to no purpose. See plugins/dev/skills/monitor-events/SKILL.md for the full pattern.
Launch the Monitor before entering the reactive scan. Wrap this command with the Monitor tool
— each emitted line is a wake-up.
The recommended filter is scope-aware: build it from the orchestrator's worker signal directory
using catalyst-events build-orchestrator-filter, then pass the result verbatim as --filter:
FILTER=$(catalyst-events build-orchestrator-filter "$ORCH_DIR")
catalyst-events tail --filter "$FILTER"
Use this command EXACTLY as shown — do NOT improvise a | grep … or | jq … post-pipe (see "Filter
discipline" below for why, including the specific | grep -v 'filter.wake' anti-pattern).
catalyst-filter alternative (CTL-257): When the filter daemon is running, you can replace the
broad catalyst-events tail with a targeted catalyst-events wait-for on
filter.wake.{ORCH_NAME}. The daemon batches raw events through Groq, classifies which are relevant
to this orchestrator's context, and emits a single wake event — so the reactive scan runs only on
semantically meaningful events rather than on every raw webhook. The 10-minute timeout acts as the
safety-net fallback for daemon-down scenarios.
WAKE_EVENT=$(catalyst-events wait-for \
--filter ".attributes.\"event.name\" == \"filter.wake.${ORCH_NAME}\"" \
--timeout 600 2>/dev/null || true)
if [ -n "$WAKE_EVENT" ]; then
WAKE_REASON=$(echo "$WAKE_EVENT" | jq -r '.body.payload.reason // "unknown"' 2>/dev/null || echo "unknown")
echo "[Phase 4] Filter wake: ${WAKE_REASON}"
fi
Use WAKE_REASON for logging only; the reactive scan below reads authoritative state from
gh pr view, git rev-list, and the signal file regardless of how the wake was triggered.
The helper reads ${ORCH_DIR}/workers/*.json and emits a single jq predicate that matches
catalyst-origin events for this orchestrator, worker lifecycle events for any in-orch ticket, github
events scoped by branch-ref prefix (refs/heads/<orch>-...) or PR-number set,
check_suite/workflow_run events whose detail.prNumbers intersect the PR set, and linear events
for any in-orch ticket. Re-build it after orchestrate-dispatch-next adds new workers so the
PR/ticket sets stay in sync.
Filter discipline (CTL-240, CTL-372). All noise filtering belongs inside --filter. Do NOT
pipe catalyst-events tail through a downstream awk/sed/grep/jq stage for additional
filtering or projection. The primary reason is clarity — --filter is the single place a reader can
look to see what reaches the consumer. The secondary reason is buffering: BSD awk (and unflagged
grep/sed) buffer stdout in 4 KB blocks when stdout is not a TTY, and with the typical ~1–3
events/min orchestrator cadence the buffer never fills and notifications stall silently for 15+
minutes despite live PR activity. grep --line-buffered and jq --unbuffered DO line-flush
mechanically (per their macOS/Linux man pages), but you should still not need either flag because
filtering belongs in --filter.
Anti-pattern (CTL-372): … | grep -v '"event.name":"filter.wake"'. Observed in a real
orchestrator session and is wrong for two reasons: (a) filter.wake.* events are emitted by the
broker as canonical OTel envelopes with no top-level .event, .orchestrator, or .scope field.
build-orchestrator-filter's predicate reads only v1 paths, so canonical filter.wake.* events
never satisfy any clause and never reach the consumer in the first place. (b) The grep pattern would
also strip this orchestrator's OWN intended filter.wake.${ORCH_NAME} wake — the very event
registered with the broker. Since CTL-346 the broker no longer re-classifies its own emissions, so
there is no feedback loop to defend against on the consumer side either. If you find yourself
wanting to remove filter.wake noise from tail output, the answer is to use
build-orchestrator-filter (which already excludes them) and avoid any hand-rolled --filter that
adds a .attributes."catalyst.orchestrator.id" clause without an event-type guard.
GitHub event schema (CTL-240). github.* webhook events carry orchestrator: null and
worker: null on every line — they are scoped only by .attributes."vcs.repository.name",
.attributes."vcs.ref.name", .attributes."vcs.pr.number", .attributes."vcs.revision", and (for
check_suite / workflow_run) body.payload.prNumbers. Predicates that try to scope github events
by .attributes."catalyst.orchestrator.id" == "<orch>" will silently drop every github event.
build-orchestrator-filter handles this correctly — prefer it over hand-rolled filters.
If you need to write a filter by hand (e.g. for one-off wait-for calls), the broad event-type
recommendation is:
catalyst-events tail --filter '
(.attributes."event.name" | startswith("github.pr.")) or
(.attributes."event.name" | startswith("github.pr_review")) or
(.attributes."event.name" | startswith("github.issue_comment")) or
(.attributes."event.name" | startswith("github.check_")) or
(.attributes."event.name" | startswith("github.workflow_run")) or
(.attributes."event.name" | startswith("github.deployment")) or
(.attributes."event.name" == "github.push") or
(.attributes."event.name" | startswith("linear.issue.")) or
(.attributes."event.name" == "orchestrator.worker.phase_advanced") or
(.attributes."event.name" == "orchestrator.worker.status_terminal") or
(.attributes."event.name" == "orchestrator.worker.pr_created") or
(.attributes."event.name" == "orchestrator.worker.done") or
(.attributes."event.name" == "orchestrator.worker.failed") or
(.attributes."event.name" == "orchestrator.attention.raised") or
(.attributes."event.name" == "orchestrator.attention.resolved")
'
This list extends the pre-CTL-240 recommendation with pr_review_comment (Codex review threads land
here — needed for CTL-64 BLOCKED auto-fixup detection), issue_comment (general PR comments), and
workflow_run (the most reliable CI-done signal). The broad form has no scope filter, so events
from sibling orchestrators sharing the repo will also fire wake-ups; prefer
build-orchestrator-filter when you have an $ORCH_DIR to draw on.
Orchestrator-scoped filtering (CTL-234): github.* events now carry
.attributes."catalyst.orchestrator.id" (stamped at receive time by the webhook handler). You may
safely add (.attributes."catalyst.orchestrator.id" == "${ORCH_NAME}") and (...) to narrow the
filter to this run's PRs only.
Wake narration (MANDATORY, CTL-369). Every Monitor wake — including ones classified as routine
or already-addressed — must produce a single short line of assistant text before returning to the
wait. The Claude Code harness wraps each Monitor stdout line in a <task-notification> XML user
message; if the orchestrator's response is end_turn with only thinking blocks and no text
content, the UI renders the next <task-notification>'s <task-id> as a phantom
Human:\n<task-id> line in the transcript. The narration line defeats that artifact and gives the
operator reading the transcript later a record of what fired and what was decided.
Line shape (pick one; keep it under ~120 characters):
wake: <event.name> #<pr> [interest=<type>] — <action being taken>
wake: <event.name> — routine, staying in event loop
wake: <event.name> — already addressed, no-op
wake: idle-timeout — running periodic reconciliation scan
Surface the matched interest when wake came from the broker (filter.wake.${ORCH_NAME}): include
.body.payload.interest_id (or its type: pr_lifecycle / ticket_lifecycle / comms_lifecycle)
and a one-clause restatement of .body.payload.reason. For broad-form catalyst-events tail wakes,
surface the raw event.name and the PR/ticket scope instead.
See plugins/dev/skills/monitor-events/SKILL.md § Narration for the full rule and the good-vs-bad
transcript fixture.
Wake-up classification. When a line arrives on the Monitor, classify it before re-entering the
scan so the response stays proportional. Every reaction reads authoritative state from gh pr view,
git rev-list, or the signal file — events are wake-up triggers, never sources of truth.
| Event | Reaction |
|---|
orchestrator.worker.phase_advanced | Routine in-flight progress; re-render DASHBOARD.md. Coalesced — .body.payload.changes carries the batch (CTL-229) |
orchestrator.worker.status_terminal, orchestrator.worker.done, orchestrator.worker.failed | Terminal transition; re-render DASHBOARD.md and run orchestrate-dispatch-next to fill freed slots. PR-bearing transitions carry .body.payload.pr.{number,url} (CTL-229) |
orchestrator.worker.pr_created | Reconcile the PR number into signal/state; re-render DASHBOARD.md |
orchestrator.attention.raised, orchestrator.attention.resolved | Re-render the DASHBOARD.md NEEDS ATTENTION banner |
github.pr.merged, github.pr.closed | Run the merge-confirmation scan for that PR |
github.pr.synchronize, github.push | Re-evaluate mergeStateStatus for the affected PR; if DIRTY ≥2 min, orchestrate-auto-rebase may dispatch a rebase worker (BEHIND auto-resolves via auto-merge) |
github.check_*, github.workflow_run.completed | Re-check CI; if BLOCKED ≥10 min, orchestrate-auto-fixup may dispatch a fix-up. workflow_run.completed is the most reliable CI-done signal |
github.pr_review*, github.pr_review_comment*, github.issue_comment.created | Re-evaluate mergeStateStatus; surface review activity on the dashboard. Codex review threads land as pr_review_comment.created — required for CTL-64 BLOCKED auto-fixup detection |
github.deployment* | Record deploy outcome on the worker's signal file |
linear.issue.state_changed | Reconcile Linear state with the worker signal |
filter.wake.${ORCH_NAME} (matched on full dotted event.name) | Daemon-filtered semantic wake: read .body.payload.reason for log context, then run the full reactive scan. The reason describes what triggered the daemon (e.g., "CI failed on PR #416") but is never the authoritative source |
phase.<name>.complete.<TICKET> (via phase_lifecycle, CTL-452) | Resolve the next phase via orchestrate-phase-advance --ticket <T> --completed-phase <name>; that helper looks up the next phase in the canonical 9-phase sequence and calls orchestrate-dispatch-next --phase <next> --ticket <T>. If completed-phase=monitor-deploy, no advance (terminal). The advance is idempotent under redundant wakes |
phase.<name>.failed.<TICKET> (via phase_lifecycle, CTL-452) | Run orchestrate-revive once for the affected ticket; on the second failure (reviveCount ≥ MAX_REVIVES), mark worker stalled and post attention to the shared comms channel. Matches the existing one-retry-then-escalate handling for legacy oneshot workers |
phase.<name>.turn-cap-exhausted.<TICKET> (via phase_lifecycle, CTL-484) | Run orchestrate-revive (same script — its continuation branch handles this status). The branch reads handoffPath from the per-phase signal, dispatches a claude --bg --resume continuation with CATALYST_IS_CONTINUATION=true + CATALYST_HANDOFF_PATH + CATALYST_CONTINUATION_COUNT, and bumps .continuationCount on a budget separate from .reviveCount (default 3). On budget exhaustion: stalled + attentionReason="continuation-budget-exhausted" |
phase.<name>.skipped.<TICKET> (via phase_lifecycle, CTL-512) | Same as complete: resolve via orchestrate-phase-advance --ticket <T> --completed-phase <name>. Only emitted by phase-monitor-deploy when no deployment_status event arrived before PHASE_DEPLOY_TIMEOUT_SEC. Because completed-phase=monitor-deploy, the advance no-ops (terminal); the wake's purpose is to free the wave slot via the scheduler's in-flight predicate (scheduler.mjs:isTicketInFlight) |
| 10-minute idle (no event) | Run the full reactive scan as a safety net |
Ground truth is git + PR, not the signal file. The signal file is advisory — it reports the
worker's self-described phase. Authoritative decisions (done, stalled) come from gh pr view /
gh pr list --head <branch> and git rev-list --count <base>..<branch>. A merged upstream PR on a
worker's branch means the worker is done, regardless of what the signal file says. A worker with a
live upstream PR is not stalled even if its signal file is stale. When the signal disagrees with
git/PR, the orchestrator reconciles the signal from the authoritative source.
Reactive scan (per wake-up):
"${CATALYST_DEV_SCRIPTS}/orchestrate-healthcheck" \
--orch-dir "${ORCH_DIR}" --orch-id "${ORCH_NAME}" --grace-seconds 0 \
>/dev/null 2>&1 || true
for WORKER_SIGNAL in ${ORCH_DIR}/workers/*.json; do
TICKET=$(jq -r '.ticket' "$WORKER_SIGNAL")
WORKER_DIR="${WORKTREE_BASE}/${ORCH_NAME}-${TICKET}"
STATUS=$(jq -r '.status' "$WORKER_SIGNAL")
cd "$WORKER_DIR"
BRANCH=$(git branch --show-current)
COMMIT_COUNT=$(git rev-list --count "${BASE_BRANCH}..HEAD" 2>/dev/null || echo 0)
PR_URL=$(gh pr list --head "$BRANCH" --json url --jq '.[0].url' 2>/dev/null || echo "")
if [ -n "$PR_URL" ]; then
CI_STATUS=$(gh pr checks "$BRANCH" --json state --jq '.[].state' 2>/dev/null | sort -u)
fi
done
Update DASHBOARD.md on each wake-up using the dashboard template — every incoming event
re-renders the file. The orch-monitor daemon file-watches DASHBOARD.md and forwards changes to
connected UI clients via SSE, so per-event writes propagate to operators immediately. Include:
- Wave progress (current wave, tickets per wave)
- Per-worker status table (ticket, status, PR, test coverage columns)
- Event log (timestamped significant events)
Update state.json with machine-readable state for crash recovery.
Update global state and heartbeat after each wake-up:
"$STATE_SCRIPT" heartbeat "${ORCH_NAME}"
for WORKER_SIGNAL in ${ORCH_DIR}/workers/*.json; do
TICKET=$(jq -r '.ticket' "$WORKER_SIGNAL")
W_STATUS=$(jq -r '.status' "$WORKER_SIGNAL")
W_PHASE=$(jq -r '.phase' "$WORKER_SIGNAL")
W_BRANCH=$(git -C "${WORKTREE_BASE}/${ORCH_NAME}-${TICKET}" branch --show-current 2>/dev/null || echo "")
W_PR=$(jq -c '.pr // null' "$WORKER_SIGNAL")
"$STATE_SCRIPT" worker "${ORCH_NAME}" "${TICKET}" \
".status = \"${W_STATUS}\" | .phase = ${W_PHASE} | .branch = \"${W_BRANCH}\" | .pr = ${W_PR}"
done
"${CATALYST_DEV_SCRIPTS}/update-dashboard.sh" \
--orch "${ORCH_NAME}" --orch-dir "${ORCH_DIR}" --roll-usage \
>/dev/null 2>>"${ORCH_DIR}/.update-dashboard.log" || true
Merge confirmation fallback (CTL-31, refined by CTL-80, CTL-133, CTL-243, CTL-252):
Workers now exit at status: "done" after actively merging their own PR (CTL-252). The
orchestrator's merge confirmation scan is a safety-net fallback for workers that stalled or
crashed before completing their own merge. Every Monitor wake-up triggered by github.pr.merged,
github.pr.closed, github.push, or github.check_suite.completed runs this scan, and the
10-minute idle fallback re-runs it so daemon-down windows do not block indefinitely. For each worker
whose signal shows pr.number but not yet pr.mergedAt, ping GitHub directly:
for WORKER_SIGNAL in ${ORCH_DIR}/workers/*.json; do
TICKET=$(jq -r '.ticket' "$WORKER_SIGNAL")
W_STATUS=$(jq -r '.status' "$WORKER_SIGNAL")
PR_NUMBER=$(jq -r '.pr.number // empty' "$WORKER_SIGNAL")
PR_URL=$(jq -r '.pr.url // empty' "$WORKER_SIGNAL")
MERGED_AT=$(jq -r '.pr.mergedAt // empty' "$WORKER_SIGNAL")
[ -n "$MERGED_AT" ] && continue
[ "$W_STATUS" = "failed" ] && continue
[ "$W_STATUS" = "stalled" ] && continue
if [ -z "$PR_NUMBER" ]; then
WORKER_DIR="${WORKTREE_BASE}/${ORCH_NAME}-${TICKET}"
BRANCH=$(git -C "$WORKER_DIR" branch --show-current 2>/dev/null || echo "")
[ -z "$BRANCH" ] && continue
REPO_SLUG=$(git -C "$WORKER_DIR" remote get-url origin 2>/dev/null \
| sed -E 's|.*github\.com[:/]([^/]+/[^/.]+)(\.git)?$|\1|')
[ -z "$REPO_SLUG" ] && continue
DISCOVERED=$(gh -R "$REPO_SLUG" pr list \
--head "$BRANCH" --state all \
--json number,state,mergedAt,url --limit 1 2>/dev/null || echo "[]")
PR_NUMBER=$(echo "$DISCOVERED" | jq -r '.[0].number // empty')
PR_URL=$(echo "$DISCOVERED" | jq -r '.[0].url // empty')
[ -z "$PR_NUMBER" ] && continue
jq --argjson n "$PR_NUMBER" --arg u "$PR_URL" \
'.pr = ((.pr // {}) | .number = ($n | tonumber) | .url = $u)' \
"$WORKER_SIGNAL" > "$WORKER_SIGNAL.tmp" && mv "$WORKER_SIGNAL.tmp" "$WORKER_SIGNAL"
fi
REPO=$(echo "$PR_URL" | sed -E 's|https://github.com/([^/]+/[^/]+)/pull/.*|\1|')
PR_JSON=$(gh -R "$REPO" pr view "$PR_NUMBER" \
--json state,mergeStateStatus,mergedAt,mergeable,mergedBy,mergeCommit 2>/dev/null || echo '{}')
PR_STATE=$(echo "$PR_JSON" | jq -r '.state // "UNKNOWN"')
MERGE_STATE=$(echo "$PR_JSON" | jq -r '.mergeStateStatus // "UNKNOWN"')
PR_MERGED_AT=$(echo "$PR_JSON" | jq -r '.mergedAt // empty')
MERGE_COMMIT_SHA=$(echo "$PR_JSON" | jq -r '.mergeCommit.oid // empty')
SKIP_DEPLOY=$(jq -r --arg repo "$REPO" \
'.catalyst.deploy[$repo].skipDeployVerification // true' "$CONFIG_FILE" 2>/dev/null)
PROD_ENV=$(jq -r --arg repo "$REPO" \
'.catalyst.deploy[$repo].productionEnvironment // "production"' "$CONFIG_FILE" 2>/dev/null)
DEPLOY_TIMEOUT_SEC=$(jq -r --arg repo "$REPO" \
'.catalyst.deploy[$repo].timeoutSec // 1800' "$CONFIG_FILE" 2>/dev/null)
case "$PR_STATE" in
MERGED)
if [ "$SKIP_DEPLOY" != "false" ]; then
TARGET_STATUS="done"
TARGET_PHASE=6
else
TARGET_STATUS="merged"
TARGET_PHASE=5
fi
jq --arg ts "$PR_MERGED_AT" --arg sha "$MERGE_COMMIT_SHA" \
--arg status "$TARGET_STATUS" --argjson phase "$TARGET_PHASE" \
'.pr.ciStatus = "merged" | .pr.mergedAt = $ts | .status = $status
| (if $status == "done" then .completedAt = $ts | .phaseTimestamps.done = $ts else . end)
| .phase = $phase
| (if $sha != "" then .pr.mergeCommitSha = $sha else . end)
| (if $status == "merged" then .deploy = ((.deploy // {}) | .startedAt = $ts | .environment = "'"$PROD_ENV"'") else . end)' \
"$WORKER_SIGNAL" > "$WORKER_SIGNAL.tmp" && mv "$WORKER_SIGNAL.tmp" "$WORKER_SIGNAL"
"$STATE_SCRIPT" worker "${ORCH_NAME}" "${TICKET}" \
".status = \"${TARGET_STATUS}\" | .phase = ${TARGET_PHASE} | .pr.ciStatus = \"merged\" | .pr.mergedAt = \"${PR_MERGED_AT}\""
"$STATE_SCRIPT" event "$(jq -nc \
--arg ts "$(date -u +%Y-%m-%dT%H:%M:%SZ)" --arg orch "${ORCH_NAME}" \
--arg w "${TICKET}" --argjson pr "$PR_NUMBER" --arg mt "$PR_MERGED_AT" \
'{ts:$ts, orchestrator:$orch, worker:$w, event:"worker-pr-merged", detail:{pr:$pr, mergedAt:$mt}}')"
