manage-codex-workers

# Manage Codex Workers Use `conveyor ...` as the primary CLI. It is installed by the `agent-conveyor` Python package. The legacy `workerctl` command remains a compatibility alias, but skill-driven flows should prefer `conveyor`. ## One-Prompt Codex App Ralph Loop This is the preferred entry point when the user has installed Agent Conveyor and wants to use another Codex app session without learning the low-level command sequence. User prompt: ```text Use the manage-codex-workers skill. Set up a Codex app Ralph loop for issue CTL. Worker session: <worker-name or choose one> Manager session: <manager-name or choose one> Template: <template name, or choose the best one> Max iterations: <number, default 3> Require adversarial proof before another worker iteration. ``` Skill behavior: 1. Work from `/Users/neonwatty/Desktop/codex-terminal-manager`. 2. Run `conveyor doctor` and `conveyor db-doctor`; fix or report blockers. 3. Choose concise task, worker, manager, and run names when the user does not provide them. Do not ask the user to invent generated names. 4. If running in the Codex app and thread tools are available, create a fresh same-project worker thread with `create_thread`, name it with `set_thread_title`, and keep the returned thread id/title. Use `create_thread` for this flow; do not use `fork_thread` unless the user explicitly asks to fork or resume this exact conversation. 5. Create the no-tmux binding with `conveyor create-disposable-binding` using `--template` when a template is known, `--adversarial`, a bounded `--max-iterations`, and `--json`. When step 4 produced a worker thread id, pass it through `--worker-codex-app-thread-id` and `--worker-codex-app-thread-title` so Conveyor can surface the app identity in `sessions`, `discover`, and setup JSON. If app thread tools are not available, create the binding without those flags and ask the user to open a separate Codex app worker session manually. 6. Ensure Dispatch is running or tell the user the single command to start it: `conveyor dispatch --watch --dispatcher-id dispatch-local`. 7. Read the returned `communication` blocks. A worker or manager with `session_kind=tmux` and `receive_style=push` can receive direct tmux pushes; one with `session_kind=codex_app` and `receive_style=pull` must poll the printed inbox command. 8. Give the worker Codex app session the generated `worker_handoff` prompt. If step 4 created a fresh worker thread, use `send_message_to_thread` only to deliver that bootstrap prompt. Durable manager/worker communication still goes through Dispatch and `worker-inbox`/`manager-inbox`; direct app-thread messages are not Dispatch receipts. The worker should keep polling `conveyor worker-inbox <task> --consume-next --wait --timeout 60 --json` through the bounded loop until no inbox item remains or `max_iterations` is reached. 9. After each worker pass, require concrete evidence and structured `loop-evidence adversarial-check` proof before queueing another `enqueue-continue-iteration`. 10. Use `conveyor loop-status <task> --run <run> --json` and telemetry/audit receipts before declaring the loop ready for manager review. Idle polling rule for Codex app/no-tmux sessions: - When a worker has `session_kind=codex_app` or `receive_style=pull`, its idle/check-in command is `conveyor worker-inbox <task> --consume-next --wait --timeout 60 --json`. - When a manager has `session_kind=codex_app` or `receive_style=pull`, its idle/check-in command is `conveyor manager-inbox <task> --consume-next --wait --timeout 60 --json`. - Repeat the appropriate command whenever the session is idle, after finishing a received instruction, and before deciding there is nothing more to do. A timeout is not completion; it is only a quiet poll interval. - Keep `conveyor dispatch --watch --dispatcher-id dispatch-local` running so Dispatch can route new messages into those inboxes. Reference docs: - `README.md` command reference - `docs/qa/ralph-loop-operator-guide.md` - `docs/agent-evidence-playbook.md` ## Supervision Model Supervision is built on three primitives: **sessions**, **tasks**, and **bindings**. - A **worker session** is a Codex session registered with Agent Conveyor. It may be a tmux-backed session or a Codex app/no-tmux session. Its rollout JSONL on disk (`~/.codex/sessions/.../rollout-*.jsonl`, or a disposable rollout file) is the source of truth for ingest. - A **manager session** is a Codex session that can run anywhere — Ghostty, iTerm2, Terminal.app, a web terminal. The manager does not need tmux. Its job is to call `conveyor` commands, read their JSON output, and decide what to do next. - Registration, `sessions`, `discover`, and disposable binding JSON include a `communication` block. Use it to decide the receive style for both worker and manager: tmux sessions are push-capable, while Codex app/no-tmux sessions receive through `manager-inbox` or `worker-inbox` polling. - App-assisted setup may also record optional `codex_app_thread_id` and `codex_app_thread_title` metadata. This identifies the human-readable Codex app thread; it does not replace rollout ingest or Dispatch inbox receipts. - A **task** is a unit of supervised work with a goal. - A **binding** ties one worker session and one manager session to one task. The manager Codex drives the supervision loop by calling `conveyor cycle <task>` repeatedly. Each cycle ingests new rollout events, captures the worker's tmux pane as a shadow signal, persists a `manager_cycles` row, and returns structured JSON. The manager reads that JSON and decides. Dispatch is core infrastructure for supervised pairs. The `pair` workflow starts a detached Dispatch watch process by default after worker/manager setup and bind. For manually bound pairs, keep Dispatch running in another shell with: ```bash conveyor dispatch --watch --dispatcher-id dispatch-local ``` Dispatch wakes the bound manager on worker completion and executes queued `notify_manager` / `nudge_worker` commands. It does not decide whether the task is correct or finished. ## Preflight 1. Work from the control repo: ```bash cd /Users/neonwatty/Desktop/codex-terminal-manager ``` 2. Verify dependencies: ```bash conveyor doctor ``` 3. Verify the SQLite control plane is healthy: ```bash conveyor db-doctor ``` 4. From the current Codex session, check whether it can register itself: ```bash conveyor doctor-self ``` `supported: true` means the session is inside a live tmux session and can be registered as a worker. A non-tmux session can still be registered as a manager. ## Discovery For Q&A When the user asks which worker, manager, task, or binding to connect, search the control plane first and present likely choices instead of asking for generated names: ```bash conveyor discover <query> conveyor search <query> ``` Use an empty query to list active candidates. Add `--all` only when the user is looking for completed tasks or gone sessions. The JSON output includes `tasks`, `sessions`, `bindings`, `telemetry`, and `suggestions`; use `suggestions` to offer concise next steps such as a `conveyor bind` command or the prompt to register a missing worker or manager. ## Preferred Manual Handoff Workflow When the user wants to hand off an already-open Codex session, do not start with a long `pair` command. Use the skill in each session: 1. In the intended worker session, ask Codex: ```text Use the manage-codex-workers skill. Register this current Codex session as the worker for this dashboard setup. Dashboard setup code: <setup-code> Working directory: <target-repo> Let the skill derive the task and session names from the setup code. Do not ask me to type generated worker, manager, or task names. After registration, wait for the manager. Do not start work until the manager has created or bound the task and provided acceptance criteria. ``` 2. In a separate manager session, ask Codex: ```text Use the manage-codex-workers skill. Register this current Codex session as the manager for this dashboard setup. Dashboard setup code: <setup-code> Working directory: <target-repo> Goal: <goal> Let the skill derive the task and session names from the setup code, find the matching worker, create/configure the task if needed, and bind the worker and manager. Run cycles, inspect criteria and telemetry, nudge only when useful, require evidence, and finish/export the task when done. ``` 3. The manager session should then drive the loop with: ```bash conveyor cycle <task-name> conveyor criteria <task-name> --list conveyor telemetry --summary --task <task-name> conveyor telemetry --task <task-name> conveyor replay <task-name> ``` For `pair`-started workflows, Dispatch is started automatically unless `--no-dispatch` is passed. For manually bound pairs, keep `conveyor dispatch --watch --dispatcher-id dispatch-local` running in a separate shell while the pair is active, or run a bounded verification pass with `conveyor dispatch --watch --watch-iterations 2 --dry-run --json`. The skill should translate those prompts into explicit `conveyor` commands. For the worker, run `doctor-self`; if supported, register the current session with `register-worker`. For the manager, register the current session with `register-manager`, create/configure the task if needed, then `bind`. When the prompt includes a dashboard setup code, derive names without asking the user: ```text task: dashboard-<setup-code> worker: dashboard-<setup-code>-worker manager: dashboard-<setup-code>-manager ``` If there is already a registered worker for the derived worker name, reuse it when binding the manager. If the derived name collides with an active unrelated session, append a short suffix yourself and continue; do not ask the user to invent names. If the prompt has no setup code and no explicit names, choose concise names from the task goal or current date yourself. Ask the user only when the target repo or goal is missing or ambiguous. This is the ergonomic manual workflow. Use `pair` only when the user wants conveyor to spawn both sessions in one automated command. ## Register Sessions Register an already-running Codex worker (rollout JSONL is auto-discovered from the pid via `lsof`): ```bash conveyor register-worker --name foo --pid <WORKER_PID> \ --cwd "$PWD" --tmux-session codex-foo ``` When a Codex app tool created or identified the thread, preserve that identity with optional metadata flags: ```bash conveyor register-worker --name foo --pid <WORKER_PID> \ --cwd "$PWD" --codex-app-thread-id <THREAD_ID> \ --codex-app-thread-title "Human readable title" ``` If `lsof` discovery fails, pass the rollout path explicitly: ```bash conveyor register-worker --name foo --pid <WORKER_PID> \ --cwd "$PWD" --tmux-session codex-foo \ --codex-session ~/.codex/sessions/.../rollout-...-<uuid>.jsonl ``` Register a manager (tmux not required): ```bash conveyor register-manager --name foo-mgr --pid <MGR_PID> --cwd "$PWD" ``` For new manager sessions started by Agent Conveyor, prefer `start-manager` or `pair`. These send a manager bootstrap prompt to Codex so the rollout JSONL is opened during startup and the manager has setup context. In `pair`, the manager prompt includes the task name, goal, worker session, `manager-config <task> --questions`, and `cycle <task>`. For late attach, pass the known task context directly: ```bash conveyor start-manager --name foo-mgr --cwd "$PWD" \ --task foo-task --task-goal "..." --worker foo-worker ``` That bootstrap starts with concrete `manager-config`, `cycle`, `manager-ack`, and `worker-ack` commands instead of `<task>` placeholders. If manager config has already been recorded for the task, the bootstrap tells the manager to start with `cycle`. List registered sessions: ```bash conveyor sessions conveyor sessions --role worker conveyor sessions --role manager ``` ## Create A Task And Bind For automated bootstrap of a fresh supervised worker/manager pair, use `pair` instead of manually starting and binding sessions: ```bash conveyor pair \ --task <task-slug> \ --worker-name <worker-name> \ --manager-name <manager-name> \ --dispatcher-id dispatch-pair \ --cwd <target-repo> \ --codex-profile yolo \ --manager-mode strict \ --task-goal "<one-line goal>" \ --task-prompt "<worker prompt>" \ --manager-objective "<manager objective>" \ --manager-acceptance "<finish criterion>" ``` Use `--no-dispatch` only for isolated tests or manual Dispatch supervision. Use this for external dogfood runs. Keep the control repo as the command cwd, but set `--cwd` to the downstream project: ```bash cd /Users/neonwatty/Desktop/codex-terminal-manager export DOGFOOD_CWD="/path/to/external/project" export TASK="external-dogfood-$(date +%Y%m%d)" export WORKER="dogfood-worker-$(date +%Y%m%d)" export MANAGER="dogfood-manager-$(date +%Y%m%d)" conveyor pair \ --task "$TASK" \ --worker-name "$WORKER" \ --manager-name "$MANAGER" \ --cwd "$DOGFOOD_CWD" \ --codex-profile yolo \ --manager-mode strict \ --task-goal "Complete one small real task in the external project." \ --task-prompt "Pick one small, concrete improvement. Keep changes scoped. Run verification. Report files changed and commands run." \ --manager-objective "Supervise the worker, request acceptance criteria and evidence, and finish only when verified." \ --manager-acceptance "The task is complete, verified, and summarized with files changed and commands run." ``` During external dogfood, review telemetry every few cycles: ```bash conveyor cycle "$TASK" conveyor criteria "$TASK" --list conveyor telemetry --summary --task "$TASK" conveyor telemetry --task "$TASK" conveyor telemetry --search manager --task "$TASK" conveyor replay "$TASK" ``` Finish and export evidence: ```bash conveyor finish-task "$TASK" \ --capture-transcript-before-stop \ --require-transcript-segment \ --require-criteria-audit \ --stop-manager \ --stop-worker conveyor export-task "$TASK" \ --output "/tmp/$TASK-export" \ --zip \ --include-transcripts conveyor sessions --state active conveyor reconcile --stale-cycles-seconds 1 ``` ```bash conveyor tasks --create my-task --goal "Refactor auth" conveyor handoff my-task \ --summary "Worker explored the current auth flow and found middleware drift." \ --next-step "Implement the middleware cleanup from docs/auth-plan.md" conveyor manager-config my-task \ --mode guided \ --objective "Keep the worker aligned to docs/auth-plan.md" \ --reference docs/auth-plan.md \ --acceptance "Tests pass" \ --guideline "Nudge only when the worker is idle, stale, or blocked" conveyor bind --task my-task --worker foo --manager foo-mgr ``` `tasks` lists or creates rows. `bind` ties the worker and manager sessions to the task. The task is now active. Use `handoff` before or during management promotion to save the worker's compact progress summary and likely next steps in SQLite. Use `manager-config` to save what the manager should check against, how structured supervision should be, acceptance criteria, planning/PRD/mockup references, and permissions such as `--allow-pr`, `--allow-merge-green`, and `--allow-worker-compact-clear`. ## Manager Recipes When the user's setup request is broad or freeform, resolve it to one named manager recipe before starting supervision, or explicitly label it `custom`. Use `docs/manager-recipes.md` as the canonical reference, and use `conveyor manager-recipes --list --json` or `conveyor manager-recipes --show <recipe> --json` for stable setup metadata. First-draft recipes: - `GoalBuddy Conveyor` — one parent board, one active child board, PR/CI/merge or `satisfied_on_main` proof, and parent receipt update before the next child. - `Test Coverage Loop` — require test coverage evidence plus structured adversarial proof before another worker pass. - `UX Polish Loop` — require browser/screenshot/visual-diff evidence plus structured adversarial proof before another visual pass. - `Nudge / What's Next Manager` — observe, ask status and criteria questions, nudge sparingly, and keep permissions minimal. - `PR/CI/Merge Ralph Loop` — manage PR readiness, CI, fixes, merge, handoff, and compact/clear receipts. Support patterns: - `Inbox / No-Tmux App Loop` — use `manager-inbox` and `worker-inbox` for Codex app sessions that cannot receive tmux pushes. - `Recovery / Resume / Handoff` — inspect saved config, handoff, replay, audit, telemetry, and inbox state before continuing a task. Before saving config or cutting the manager loose, show a locked setup summary with the selected recipe, mode, permissions, tools, epilogues, cleanup policy,