agent-swarm

name	agent-swarm
aliases	["Parallel Agent"]
description	(Industry standard: Parallel Agent) Primary Use Case: Work that can be partitioned into independent sub-tasks running concurrently across multiple agents. Parallel multi-agent execution pattern. Use when: work can be partitioned into independent tasks that N agents can execute simultaneously across worktrees. Includes routing (sequential vs parallel), merge verification, and correction loops.

Agent Swarm

Parallel or pipelined execution across multiple agents and worktrees. The orchestrator partitions work, dispatches to agents, and verifies/merges the results.

When to Use

• Large features that can be split into independent work packages
• Bulk operations (tests, docs, migrations, RLM distillation) that benefit from parallelism
• Multi-concern work where specialists handle different aspects simultaneously

Process Flow

1. Plan & Partition -- Break work into independent tasks. Define boundaries clearly.
2. Route -- Decide execution mode:
   • Sequential Pipeline -- Tasks depend on each other (A -> B -> C)
   • Parallel Swarm -- Tasks are independent (A | B | C)
3. Dispatch -- Create a worktree per task. Assign each to an agent:
   • CLI agent (Claude, Gemini, Copilot)
   • Deterministic script
   • Human
4. Execute -- Each agent works in isolation. No cross-worktree communication.
5. Verify & Merge -- Orchestrator checks each worktree's output against acceptance criteria.
   • Pass -> Merge into main branch
   • Fail -> Generate correction packet, re-dispatch
6. Seal -- Bundle all merged artifacts
7. Retrospective -- Did the partition strategy work? Was parallelism effective?

Worker Selection

Each worktree can be assigned to a different worker type based on task complexity:

Worker	Cost	Best For
High-reasoning CLI (Opus, Ultra, GPT-5.3)	High	Complex logic, architecture
Fast CLI (Haiku, Flash 2.0)	Low	Tests, docs, routine tasks
Free Tier: Copilot gpt-5-mini	$0	Bulk summarization, zero-cost batch jobs
Free Tier: Gemini gemini-3-pro-preview	$0	Large context batch jobs
Deterministic Script	None	Formatting, linting, data transforms
Human	N/A	Judgment calls, creative decisions

Zero-Cost Batch Strategy: For bulk summarization or distillation jobs, use --engine copilot (gpt-5-mini) or --engine gemini (gemini-3-pro-preview). Both are free-tier models available via their respective CLIs. Gemini Flash 2.0 is also very cheap if more capacity is needed. Use --workers 2 for Copilot (rate-limit safe) and --workers 5 for Gemini.

Implementation: swarm_run.py

The swarm_run.py script is the universal engine for executing this pattern. It is driven by Job Files (.md with YAML frontmatter).

Key Features

• Resume Support -- Automatically saves state to .swarm_state_<job>.json. Use --resume to skip already processed items.
• Intelligent Retry -- Exponential backoff for rate limits.
• Verification Skip -- Use check_cmd in the job file to short-circuit work if a file is already processed (e.g. exists in cache).
• Dry Run -- Test your file discovery and template substitution without cost.
• Engine Flag -- --engine [claude|gemini|copilot] switches CLI backends at runtime.

Usage

# Zero-cost Copilot batch (2 workers recommended to avoid rate limits)
source ~/.zshrc   # NOTE: use source ~/.zshrc, NOT 'export COPILOT_GITHUB_TOKEN=$(gh auth token)'
                  # gh auth token generates a PAT without Copilot scope -> auth failures
python3 plugins/agent-loops/skills/agent-swarm/scripts/swarm_run.py \
    --engine copilot \
    --job plugins/rlm-factory/resources/jobs/rlm_chronicle.job.md \
    --files-from checklist.md \
    --resume --workers 2

# Gemini (free, higher parallelism)
python3 plugins/agent-loops/skills/agent-swarm/scripts/swarm_run.py \
    --engine gemini \
    --job plugins/rlm-factory/resources/jobs/rlm_chronicle.job.md \
    --files-from checklist.md \
    --resume --workers 5

# Claude (paid, highest quality)
python3 plugins/agent-loops/skills/agent-swarm/scripts/swarm_run.py \
    --job plugins/rlm-factory/resources/jobs/rlm_chronicle.job.md \
    [--dir some/dir] [--resume] [--dry-run]

Job File Schema

---
model: haiku        # haiku -> auto-upgraded to gpt-5-mini (copilot) or gemini-3-pro-preview (gemini)
workers: 2          # keep to 2 for Copilot, up to 5-10 for Gemini/Claude
timeout: 120        # seconds per worker
ext: [".md"]        # filters for --dir
# Shell template. {file} is shell-quoted automatically (handles apostrophes safely)
post_cmd: "python3 plugins/rlm-factory/skills/rlm-curator/scripts/inject_summary.py --file {file} --summary {output}"
# Optional command to check if work is already done (exit 0 => skip)
check_cmd: "python3 plugins/rlm-factory/skills/rlm-curator/scripts/check_cache.py --file {file}"
vars:
  profile: project
---
Prompt for the agent goes here.

IMPORTANT for Copilot engine: The copilot CLI ignores stdin when -p is used.
Instead, the instruction is prepended to the file content automatically by 
                

Known Engine Quirks

Copilot CLI

• No -p flag -- Copilot ignores stdin when -p is present. swarm_run.py automatically prepends the prompt to the file content instead.
• Auth token scope -- Use source ~/.zshrc to load your token. gh auth token returns a PAT without Copilot permissions, causing auth failures under concurrency.
• Rate limits -- Use --workers 2 maximum. Higher concurrency trips GitHub's anti-abuse systems and surfaces as authentication errors.
• Concurrent writes -- If using a shared JSON post-cmd output (e.g. cache), ensure the writer script uses fcntl.flock for atomic writes. See inject_summary.py.

Gemini CLI

• Accepts -p "prompt" flag normally
• Supports higher concurrency (5-10 workers)
• Model auto-upgrade: haiku -> gemini-3-pro-preview

Checkpoint Reconciliation

If a batch run is interrupted partway through and the output store (e.g. cache JSON) is partially corrupted, reconcile the checkpoint before resuming:

# Remove phantom "done" entries that aren't actually in the output store
completed = [f for f in st['completed'] if f in actual_output_keys]
st['failed'] = {}

Then rerun with --resume.

Constraints

• Each worker execution must be independent
• Post-commands must be idempotent if using resume
• Orchestrator owns the overall job state
• {file} in post_cmd is shell-quoted automatically -- filenames with apostrophes are safe

Diagram

See: plugins/agent-loops/resources/diagrams/agent_swarm.mmd