fleet-orchestrator

name	fleet-orchestrator
description	Fan out a single high-level directive (audit, deprecation cleanup, fix, migration, version bump) across an arbitrary list of repositories as parallel Agent workers, with a supervisor that polls a shared manifest, renders a live status table, and aggregates final results. Use when the user says 'apply X across repos', 'audit N repositories', 'sweep every repo', 'run the same fix in all projects', 'parallel multi-repo', 'fleet-wide change', or provides a list of repositories with a single directive. Preferred over running issue-work in each repo sequentially.
argument-hint	<repos-spec> <directive-spec> [--max-parallel N] [--max-retries N] [--poll-interval SEC] [--reanchor-interval N] [--top-k N] [--agents-dir PATH] [--resume <fleet-id>]
user-invocable	true
disable-model-invocation	false
allowed-tools	Bash(gh *), Bash(flock *), Bash(jq *)
max_iterations	10
halt_conditions	[{"type":"success","expr":"All repos reach a terminal worker status (done, failed-after-retries, skipped)"},{"type":"limit","expr":"--max-parallel worker pool drains with no pending items"}]
on_halt	Render final fleet-status table with per-repo outcome and exit
loop_safe	false
tiers	{"light":{"ref_docs":[],"deep_checks":false},"standard":{"ref_docs":["core"],"deep_checks":false},"deep":{"ref_docs":["core"],"deep_checks":true}}
default_tier	standard
iso_class	none

Fleet Orchestrator -- Parallel Multi-Repo Directive Executor

A supervisor skill that applies one directive across many repositories concurrently. Each repository is handled by an independent worker Agent; progress and outcomes flow through a single JSON manifest (fleet-status.json) updated with flock for atomic, race-free writes.

When to Use This Skill

• Applying the same change (audit, cleanup, migration, dependency bump, policy update) to N repos.
• The user wants wall-clock compression — eight repos serialized take hours, fanned out take the length of the slowest one.
• Failure isolation matters — one flaky repo must not block the other seven.
• The directive is uniform enough that per-repo customization is minor (each worker still adapts to its repo's code style and issue landscape).

Do NOT use this skill when:

• There is only one repo — use /issue-work directly.
• The directive requires tight cross-repo coordination (e.g., a schema change that must merge in strict order) — sequence it manually.
• Repos have incompatible toolchains and the supervisor cannot meaningfully aggregate their outcomes.

Execution Mode: Sub-agent (Parallel) with Supervisor

Agent teams are capped at one active team per session. Fleet work needs N independent workers running concurrently with run-to-completion semantics, so we use the sub-agent mode (Agent(..., run_in_background=true)) plus a file-based manifest for coordination.

Data passing strategy:

Layer	Mechanism	Why
Shared state	_workspace/fleet/fleet-status.json	One JSON blob, race-free via flock, readable by every worker and the supervisor
Per-worker artifacts	_workspace/fleet/{repo-slug}/	Per-worker scratch, logs, diff snapshots — preserved for audit
Supervisor → worker	Agent prompt (one-shot)	Each worker gets the spec and its repo slot at launch; no runtime re-parameterization
Worker → supervisor	Manifest writes	Supervisor polls; workers never call back

Agent Configuration

Role	subagent_type	model	run_in_background	Purpose
supervisor	(main session)	n/a	n/a	Input validation, worker fan-out, manifest polling, aggregation
worker (one per repo)	general-purpose	opus	true	Executes the directive on its assigned repo, updates manifest

All workers use the same worker prompt template (reference/worker-template.md). The supervisor never edits code itself — it only dispatches, monitors, and reports.

Architecture

                    [Supervisor (main session)]
                     | dispatch      ^ poll every POLL_INTERVAL seconds
                     v               |
       +-------------+---------------+-------------+
       |             |               |             |
   [Worker 1]    [Worker 2]      [Worker 3]    [Worker N]
   repo A        repo B          repo C         repo N
       \             \               /             /
        \            flock-guarded   /            /
         \           writes to       /           /
          \------> fleet-status.json <----------/

Workflow

Phase 1: Input Resolution and Validation

The skill accepts inputs in three forms; pick the first that matches:

Input form	Example	How to parse
Inline list	repo-a repo-b repo-c "fix all TODOs"	Positional args: repos until a non-repo-like token, then the directive
File list	@repos.txt "fix all TODOs"	Read repos.txt, one owner/repo per line
Org-wide	--org kcenon "fix all TODOs"	gh repo list <org> --json nameWithOwner,isArchived --jq '[.[] | select(.isArchived == false)] | .[].nameWithOwner'

Validation rules:

1. Every repo must resolve to owner/repo form. Reject entries without a /.
2. Deduplicate repos.
3. Verify each repo exists and the current user has write access: gh api "repos/{owner}/{repo}" --jq .permissions.push.
4. Repos failing step 3 are moved to a pre-excluded list in the manifest with a reason, not dispatched.
5. The directive must be non-empty. If empty, prompt the user for it (one-shot).
6. If N > --max-parallel, workers are launched in waves of --max-parallel. Default: 8 (conservative; most Macs handle this comfortably).

Phase 2: Manifest Initialization

Create _workspace/fleet/ and write the initial manifest:

FLEET_ID="fleet-$(date '+%Y%m%d-%H%M%S')"
mkdir -p "_workspace/fleet"
MANIFEST="_workspace/fleet/fleet-status.json"

# Seed the manifest. Use jq to build the structure to guarantee valid JSON.
jq -n \
  --arg fleet_id "$FLEET_ID" \
  --arg started_at "$(date -u '+%Y-%m-%dT%H:%M:%SZ')" \
  --arg directive "$DIRECTIVE" \
  --argjson repos "$REPOS_JSON" \
  '{
     schema_version: "1.0.0",
     fleet_id: $fleet_id,
     started_at: $started_at,
     directive: $directive,
     max_parallel: '$MAX_PARALLEL',
     max_retries: '$MAX_RETRIES',
     workers: ($repos | map({
       repo: .,
       status: "queued",
       started_at: null,
       updated_at: null,
       pr_url: null,
       ci_conclusion: null,
       merge_status: null,
       retry_count: 0,
       error: null,
       phase: "queued"
     })),
     summary: null
   }' > "$MANIFEST"

Full schema: reference/manifest-schema.json (JSON Schema Draft 2020-12). Changes to the manifest shape must bump schema_version in both the seed above and the schema file.

Phase 2.5: Top-K Agent Routing (per worker)

Motivation: the agents/ directory contains 8 specialized agents. Spawning every applicable agent per work item costs roughly 64 K tokens of parallel context and frequently invokes agents that are irrelevant to the change (for example test-strategist on a documentation-only PR). This phase selects only the K highest-scoring agents per work item, inspired by Top-K expert routing in Mixture-of-Experts transformers.

Controlled by the --top-k N flag (default 2, 0 disables routing and preserves the prior "all-applicable" behavior).

Agent Metadata Contract

Each agent definition under the repository's agents directory carries two frontmatter fields used for scoring:

Field	Type	Purpose
applies_to	list of glob patterns	File patterns this agent is relevant for (for example **/*.ts, **/test_*.py, **/*.md)
keywords	list of strings	Task-type triggers (for example security, performance, refactor)

Canonical source of agent definitions: plugin/agents/*.md (the distributable copy) with a mirror at project/.claude/agents/*.md. The orchestrator reads from --agents-dir (default plugin/agents). Both directories are kept in sync by the agent-sync policy; the scorer only needs one of them.

Scoring Function

For each work item (issue + PR draft for a repo) and each agent:

score(agent, work_item) =
    2 * count(glob in agent.applies_to where any changed_file matches glob)
  + 1 * count(keyword in agent.keywords where keyword appears in work_item.title or work_item.body)

Rules:

1. Glob matches use case-sensitive gitignore-style matching (the same semantics git ls-files uses). **/*.ts matches src/a.ts and a.ts.
2. Keyword matches are case-insensitive substring matches against the concatenation of the issue title and body. Word-boundary matching is NOT required — security matches security-audit intentionally.
3. changed_file is sourced from git diff --name-only <base>..<head> when a draft PR exists, otherwise from the issue body's "Where" section file list.
4. Ties are broken by alphabetical agent name (stable ordering).
5. Agents scoring 0 are never selected regardless of K.

Selection

positive = [a for a in agents if score(a) > 0]
selected = sort_by_score_desc(positive).take(K)
if len(selected) == 0:
    # Nothing scored > 0; fall back to a single documentation-writer or the
    # caller-specified default agent. Record "no-match" in the worker log.
    selected = [DEFAULT_FALLBACK_AGENT]

Zero-score agents are never selected, even when K >= len(agents). To restore the pre-Top-K "all-applicable" behavior, set --top-k 0, which disables routing entirely.

The worker records its scoring decision to {{LOG_DIR}}/agent-routing.json:

{
  "work_item": {"repo": "owner/repo", "issue": 123, "pr": null},
  "k": 2,
  "all_scores": [
    {"agent": "code-reviewer", "score": 5, "matched_globs": ["**/*.ts"], "matched_keywords": ["security"]},
    {"agent": "documentation-writer", "score": 0, "matched_globs": [], "matched_keywords": []}
  ],
  "selected": ["code-reviewer", "qa-reviewer"]
}

This per-worker artifact is the telemetry surface required by the issue's acceptance criteria.

Unit Test Points

The scoring function is pure (no I/O beyond reading agent frontmatter and a file list), so it is unit-tested in isolation. Required cases:

1. Docs-only PR (only *.md changed, no keyword overlap) — selects documentation-writer over test-strategist at K=2.
2. Mixed TypeScript + security keyword — code-reviewer and dependency-auditor outrank structure-explorer.
3. Empty applies_to + empty keywords — agent scores 0 and is excluded.
4. K >= agent_count — all agents selected, order preserved by score.
5. Tie on score — stable alphabetical ordering.

Tests live under tests/fleet_orchestrator/test_topk_scoring.py and tests/fleet_orchestrator/test_docs_only_routing.py. As of #622 they run on every PR via .github/workflows/validate-skills.yml (python -m pytest tests/fleet_orchestrator/ -q); silent breakage in the scorer would now fail CI rather than ship to consumers of this skill.

Fallback and Compatibility

• --top-k 0 disables routing entirely; the worker spawns whatever it would have spawned before this phase existed.
• If no agent directory is found at --agents-dir, the worker logs a routing: disabled notice and proceeds without Top-K selection.
• Missing applies_to or keywords frontmatter is treated as an empty list (score contribution 0); it does not fail the worker.

Phase 3: Fan-out Worker Dispatch

Launch workers up to --max-parallel at a time. The Agent tool with run_in_background=true returns immediately, so all N calls go out in a single message:

For each repo in the first wave (size min(N, MAX_PARALLEL)):
    Agent(
        subagent_type: "general-purpose",
        model: "opus",
        run_in_background: true,
        description: "Fleet worker: <repo>",
        prompt: <render reference/worker-template.md with substitutions>
    )

Template substitutions for each worker (load reference/worker-template.md and replace these tokens before passing as prompt):

Token	Value
{{REPO}}	owner/repo
{{DIRECTIVE}}	The full directive text
{{MANIFEST_PATH}}	Absolute path to fleet-status.json
{{MAX_RETRIES}}	Retry cap for transient CI failures
{{FLEET_ID}}	For log correlation
{{TOP_K}}	--top-k value for per-worker agent routing
{{AGENTS_DIR}}	--agents-dir value for agent frontmatter lookup

Record each dispatched worker's task ID in a local map so the supervisor can call TaskOutput(block=false) on it when polling.

When a worker finishes and workers[i].status becomes completed or failed, the supervisor picks the next queued repo and dispatches a new worker (maintains --max-parallel throughput).

Phase 4: Supervisor Polling Loop

Every --poll-interval seconds (default 120):

1. Read the manifest atomically:

   flock -s "$MANIFEST.lock" -c "cat $MANIFEST" | jq .
   

2. Render a status table to the user (see the template in Status Table Rendering below).
3. For each worker task ID, call TaskOutput(block=false, timeout=2000). If the task has exited, reconcile its status in the manifest (belt-and-suspenders: the worker should have already written its terminal status).
4. Dispatch a new worker if there is a queued repo and capacity.
5. Terminate the loop when every worker is in completed or failed.

Do NOT use blocking TaskOutput(block=true) — it would freeze the supervisor and prevent monitoring peer workers.

Stuck-worker detection: if updated_at for any running worker is older than 10 × poll_interval, flag it as stuck in the rendered table but do not kill it — the worker may be legitimately running a long build. Escalate to the user only after 20 × poll_interval of silence.

Phase 5: Aggregation and Reporting

Once all workers have terminated:

1. Read the final manifest.
2. Compute the summary block:

   {
     "total": N,
     "merged": <count status=completed AND merge_status=merged>,
     "draft": <count merge_status=draft>,
     "failed": <count status=failed>,
     "skipped": <count status=skipped>,
     "duration_seconds": <ended_at - started_at>
   }
   

3. Write summary back to the manifest (flock-guarded).
4. Render the final aggregated report (see Final Report below).
5. Do NOT delete _workspace/fleet/ — keep it for the audit trail.

Phase 6: Cleanup

• Cancel any still-running background tasks (should be none, but defensive).
• Print the absolute path to _workspace/fleet/fleet-status.json so the user can inspect it later.
• If any worker ended in failed with error.class == "code-review-needed", list those repos explicitly for user attention.

Status Table Rendering

Each poll cycle renders this table (omit workers still in queued if the fleet is large, or cap at 20 rows):

## Fleet Status — fleet-20260420-071500

Elapsed: 00:12:34 | Active: 5 / 8 | Queued: 2 | Completed: 6 | Failed: 0

| # | Repo | Phase | PR | CI | Merge | Retries | Last update |
|---|------|-------|-----|-----|-------|---------|-------------|
| 1 | kcenon/repo-a | merged | #89 | success | merged | 0 | 00:11:20 ago |
| 2 | kcenon/repo-b | ci-monitoring | #90 | in_progress | — | 1 | 00:00:42 ago |
| 3 | kcenon/repo-c | implementing | — | — | — | 0 | 00:00:18 ago |
| 4 | kcenon/repo-d | failed | #91 (draft) | failure | — | 3 | 00:03:55 ago |
| 5 | kcenon/repo-e | queued | — | — | — | 0 | — |

Phase values mirror the worker's state machine (see worker template).

Final Report

## Fleet Execution Report — fleet-20260420-071500

Directive: <short form of the directive>

| Metric | Value |
|--------|-------|
| Repos dispatched | 8 |
| Merged | 6 |
| Draft (awaiting manual review) | 1 |
| Failed | 1 |
| Duration | 00:27:14 |

### Per-Repo Outcomes

| Repo | Result | PR | CI | Notes |
|------|--------|----|-----|-------|
| kcenon/repo-a | merged | https://github.com/kcenon/repo-a/pull/89 | success | — |
| kcenon/repo-b | merged | https://github.com/kcenon/repo-b/pull/90 | success | Retried once (flaky lint) |
| kcenon/repo-c | merged | https://github.com/kcenon/repo-c/pull/41 | success | — |
| kcenon/repo-d | draft | https://github.com/kcenon/repo-d/pull/91 | failure | 3 retries exhausted; user review needed |
| kcenon/repo-e | failed | — | — | Pre-excluded: no push permission |
| ... | ... | ... | ... | ... |

### Action Required

- `kcenon/repo-d` — CI failure after 3 retries. Log: `_workspace/fleet/repo-d/retry-3.log`.

Manifest: `_workspace/fleet/fleet-status.json`

Error Handling

Situation	Strategy
One worker crashes without writing terminal status	Supervisor detects exit via TaskOutput, writes {status:"failed", error:{class:"worker-crash"}} on its behalf
Manifest write races	flock around every write; readers use -s (shared) lock for consistency
Transient CI failure (test flake, runner timeout)	Worker retries up to --max-retries with exponential backoff (30s, 120s, 300s)
Genuine code failure	Worker converts PR to draft, writes error.class="code-review-needed", exits cleanly; supervisor flags in the final report
Manifest file corrupted	Fatal error — supervisor halts, prints the last good backup (worker should snapshot to fleet-status.json.N after every N writes)
Repo pre-flight fails (no push, archived)	Added to pre-excluded list in the manifest, never dispatched
Supervisor run interrupted mid-fleet	Resume is manual: re-invoke the skill with --resume <fleet-id>; it re-reads the manifest and re-launches workers in non-terminal states

Test Scenarios

Normal Flow

1. User invokes with 8 repos and a directive.
2. Phase 1 validates all 8; none are pre-excluded.
3. Phase 2 writes an 8-entry manifest, all queued.
4. Phase 3 dispatches 8 workers in parallel.
5. Phase 4 polls every 120s; workers transition queued → running → completed.
6. One worker retries a flaky CI run, succeeds on retry.
7. Phase 5 aggregates: 8 merged, 0 failed.
8. Final report printed; _workspace/fleet/ retained.

Expected outcome: manifest shows all 8 with status="completed" and merge_status="merged".

Failure Isolation Flow

1. User invokes with 8 repos.
2. Worker 3 hits a genuine compile error on retry 3.
3. Worker 3 converts its PR to draft, writes error.class="code-review-needed", exits with status failed.
4. Workers 1, 2, 4-8 continue unaffected.
5. Phase 5 reports 7 merged, 1 draft-awaiting-review.

Expected outcome: worker 3's failure does not stop peer workers; final report explicitly flags worker 3 for user action.

Pre-flight Exclusion Flow

1. User invokes with 10 repos; 2 are archived.
2. Phase 1 excludes the 2 archived repos, lists them in the manifest with pre-excluded: "archived".
3. Phase 2-5 operate on the 8 live repos.

Expected outcome: final report lists the 2 pre-excluded repos separately from the 8 dispatched.

Dependencies and Related Skills

• hooks/post-task-checkpoint.sh (issue #360 — closed): ensures worker-side checkpoints don't overwrite each other. The fleet orchestrator leans on this for cross-worker file safety inside _workspace/.
• Atomic Multi-Phase Execution rule (issue #361 — closed, in _policy.md): keeps workers from stopping mid-PR for confirmation.
• harness: the meta-skill that produced this pattern. Fleet orchestrator is an instance of Fan-out/Fan-in + Supervisor from harness/reference/agent-design-patterns.md.
• issue-work: the per-repo workflow each worker executes. The worker template is a thin wrapper that adds manifest updates around issue-work's Solo mode.
• pr-work: the retry path when a worker's initial PR fails CI.

References

• Worker prompt template: reference/worker-template.md — what each per-repo worker does step-by-step.
• Manifest schema: reference/manifest-schema.json — the authoritative JSON Schema for fleet-status.json.
• Documentation: docs/fleet-orchestrator.md — user-facing overview, examples, and the "On the Horizon" context that motivated this skill.

Output Checklist

After a fleet run, confirm:

• _workspace/fleet/fleet-status.json exists and validates against the schema.
• Every repo has a terminal status (completed, failed, or skipped).
• summary block is populated.
• Final report matches the manifest counts.
• Failed repos are called out explicitly for user action.
• _workspace/fleet/ is preserved for audit.

Reanchoring Loop Invariants

--reanchor-interval N (default 5, 0 disables) controls how often the Core invariants block from global/skills/_internal/_shared/invariants.md is emitted during the supervisor's status-polling loop.

Loop bind point: every N manifest-poll cycles. A fleet run covering 20+ repos produces enough accumulated worker outputs that the supervisor's attention drifts from the canonical rules; re-anchoring keeps the English-only, squash-merge, and CI-gate invariants in the recent context when aggregating results and deciding per-repo terminal state.

Side Effects and Loop-Safety

This skill is loop_safe: false. It spawns a fleet of worker agents that create branches, PRs, and commits across many repositories. Re-running would duplicate fleet work and re-open already-handled units. Resume an interrupted fleet via --resume <fleet-id>, never by re-invoking from the start. (The supervisor's internal status-polling loop above is bounded by --max-parallel / --reanchor-interval; that is distinct from the non-idempotent external side effects described here.)