swarm-coordination

# Swarm Coordination Multi-agent orchestration for parallel task execution. The coordinator breaks work into subtasks, spawns worker agents, monitors progress, and aggregates results. ## MANDATORY: Swarm Mail **ALL coordination MUST use `swarmmail_*` tools.** This is non-negotiable. Swarm Mail is embedded (no external server needed) and provides: - File reservations to prevent conflicts - Message passing between agents - Thread-based coordination tied to cells ## When to Swarm **DO swarm when:** - Task touches 3+ files - Natural parallel boundaries exist (frontend/backend/tests) - Different specializations needed - Time-to-completion matters **DON'T swarm when:** - Task is 1-2 files - Heavy sequential dependencies - Coordination overhead > benefit - Tight feedback loop needed **Heuristic:** If you can describe the task in one sentence without "and", don't swarm. ## Worker Survival Checklist (MANDATORY) Every swarm worker MUST follow these 9 steps. No exceptions. ```typescript // 1. INITIALIZE - Register with Swarm Mail swarmmail_init({ project_path: "/abs/path/to/project", task_description: "bead-id: Task description" }); // 2. QUERY LEARNINGS - Check what past agents learned hivemind_find({ query: "task keywords domain", limit: 5 }); // 3. LOAD SKILLS - Get domain expertise skills_list(); skills_use({ name: "relevant-skill" }); // 4. RESERVE FILES - Claim exclusive ownership swarmmail_reserve({ paths: ["src/assigned/**"], reason: "bead-id: What I'm working on", ttl_seconds: 3600 }); // 5. DO WORK // ... implement changes ... // 6. REPORT PROGRESS - Every 30min or at milestones swarm_progress({ project_key: "/abs/path/to/project", agent_name: "WorkerName", bead_id: "bd-123.4", status: "in_progress", message: "Auth service 80% complete, testing remaining", progress_percent: 80 }); // 7. CHECKPOINT - Before risky operations swarm_checkpoint({ bead_id: "bd-123.4", checkpoint_name: "pre-refactor", reason: "About to refactor auth flow" }); // 8. STORE LEARNINGS - Capture what you discovered hivemind_store({ information: "OAuth refresh tokens need 5min buffer...", metadata: "auth, oauth, tokens" }); // 9. COMPLETE - Auto-releases, records outcome swarm_complete({ project_key: "/abs/path/to/project", agent_name: "WorkerName", bead_id: "bd-123.4", summary: "Auth service implemented with JWT", files_touched: ["src/auth/service.ts", "src/auth/schema.ts"] }); ``` **Why These Steps Matter:** | Step | Purpose | Consequence of Skipping | |------|---------|-------------------------| | 1. Init | Register identity, enable coordination | Can't send messages, reservations fail | | 2. Query | Learn from past mistakes | Repeat solved problems, waste time | | 3. Skills | Load domain expertise | Miss known patterns, lower quality | | 4. Reserve | Prevent edit conflicts | Merge conflicts, lost work | | 5. Work | Actually do the task | N/A | | 6. Progress | Keep coordinator informed | Coordinator assumes stuck, may reassign | | 7. Checkpoint | Safe rollback point | Can't recover from failures | | 8. Store | Help future agents | Same bugs recur, no learning | | 9. Complete | Clean release, learning signal | Reservations leak, no outcome tracking | **If your subtask prompt doesn't include these steps, something is wrong with the coordinator.** ## Task Clarity Check (BEFORE Decomposing) **Before decomposing, ask: Is this task clear enough to parallelize?** ### Vague Task Signals (ASK QUESTIONS FIRST) | Signal | Example | Problem | | ------------------------ | ------------------------------ | -------------------------------- | | No files mentioned | "improve performance" | Where? Which files? | | Vague verbs | "fix", "update", "make better" | What specifically? | | Large undefined scope | "refactor the codebase" | Which parts? What pattern? | | Missing success criteria | "add auth" | OAuth? JWT? Session? What flows? | | Ambiguous boundaries | "handle errors" | Which errors? Where? How? | ### How to Clarify ```markdown The task "<task>" needs clarification before I can decompose it. **Question:** [Specific question about scope/files/approach] Options: a) [Option A] - [trade-off] b) [Option B] - [trade-off] c) [Option C] - [trade-off] I'd recommend (a) because [reason]. Which approach? ``` **Rules:** - ONE question at a time (don't overwhelm) - Offer 2-3 concrete options when possible - Lead with your recommendation and why - Wait for answer before asking next question ### Clear Task Signals (PROCEED to decompose) | Signal | Example | Why it's clear | | ------------------ | ------------------------------ | ---------------- | | Specific files | "update src/auth/\*.ts" | Scope defined | | Concrete verbs | "migrate from X to Y" | Action defined | | Defined scope | "the payment module" | Boundaries clear | | Measurable outcome | "tests pass", "no type errors" | Success criteria | **When in doubt, ask.** A 30-second clarification beats a 30-minute wrong decomposition. ## Coordinator Workflow ### Phase 0: Socratic Planning (NEW - INTERACTIVE) **Before decomposing, engage with the user to clarify the task.** Swarm supports three interaction modes: | Mode | Flag | Behavior | |------|------|----------| | **Full Socratic** | (default) | Ask questions, offer recommendations, collaborative planning | | **Fast** | `--fast` | Skip questions, proceed with reasonable defaults | | **Auto** | `--auto` | Minimal interaction, use heuristics for all decisions | | **Confirm Only** | `--confirm-only` | Show plan, get yes/no, no discussion | **Default Flow (Full Socratic):** ```typescript // 1. Analyze task for clarity const signals = analyzeTaskClarity(task); if (signals.needsClarification) { // 2. Ask ONE question at a time const question = generateClarifyingQuestion(signals); // 3. Offer 2-3 concrete options const options = generateOptions(signals); // 4. Lead with recommendation const recommendation = selectRecommendation(options); // 5. Present to user console.log(` The task "${task}" needs clarification before I can decompose it. **Question:** ${question} Options: a) ${options[0].description} - ${options[0].tradeoff} b) ${options[1].description} - ${options[1].tradeoff} c) ${options[2].description} - ${options[2].tradeoff} I'd recommend (${recommendation.letter}) because ${recommendation.reason}. Which approach? `); // 6. Wait for answer, iterate if needed const answer = await getUserResponse(); // 7. Ask next question if needed (ONE at a time) if (needsMoreClarification(answer)) { // Repeat with next question } } // 8. Proceed to decomposition once clear ``` **Fast Mode (`--fast`):** - Skip all questions - Use reasonable defaults based on task type - Proceed directly to decomposition **Auto Mode (`--auto`):** - Zero interaction - Heuristic-based decisions for all choices - Use for batch processing or CI **Confirm Only (`--confirm-only`):** - Generate plan silently - Show final CellTree - Get yes/no only **Rules for Socratic Mode:** - **ONE question at a time** - don't overwhelm - **Offer concrete options** - not open-ended - **Lead with recommendation** - save user cognitive load - **Wait for answer** - don't proceed with assumptions ### Phase 1: Initialize Swarm Mail (FIRST) ```typescript // ALWAYS initialize first - registers you as coordinator await swarmmail_init({ project_path: "$PWD", task_description: "Swarm: <task summary>", }); ``` ### Phase 2: Knowledge Gathering (MANDATORY) Before decomposing, query ALL knowledge sources: ```typescript // 1. Past learnings from this project hivemind_find({ query: "<task keywords>", limit: 5 }); // 2. How similar tasks were solved before hivemind_find({ query: "<task description>", limit: 5, collection: "sessions" }); // 3. Design patterns and prior art pdf_brain_search({ query: "<domain concepts>", limit: 5 }); // 4. Available skills to inject into workers skills_list(); ``` Synthesize findings into `shared_context` for workers. ### Phase 3: Decomposition (DELEGATE TO SUBAGENT) > **⚠️ CRITICAL: Context Preservation Pattern** > > **NEVER do planning inline in the coordinator thread.** Decomposition work (file reading, Hivemind session searching, reasoning about task breakdown) consumes massive amounts of context and will exhaust your token budget on long swarms. > > **ALWAYS delegate planning to a `swarm-planner` subagent** and receive only the structured CellTree JSON result back. **❌ Anti-Pattern (Context-Heavy):** ```typescript // DON'T DO THIS - pollutes main thread context const plan = await swarm_plan_prompt({ task, ... }); // ... agent reasons about decomposition inline ... // ... context fills with file contents, analysis ... const validation = await swarm_validate_decomposition({ ... }); ``` **✅ Correct Pattern (Context-Lean):** ```typescript // 1. Create planning bead with full context await hive_create({ title: `Plan: ${taskTitle}`, type: "task", description: `Decompose into subtasks. Context: ${synthesizedContext}`, }); // 2. Delegate to swarm-planner subagent const planningResult = await Task({ subagent_type: "swarm-planner", description: `Decompose task: ${taskTitle}`, prompt: ` You are a swarm planner. Generate a CellTree for this task. ## Task ${taskDescription} ## Synthesized Context ${synthesizedContext} ## Instructions 1. Use swarm_plan_prompt(task="...", max_subtasks=5) 2. Reason about decomposition strategy 3. Generate CellTree JSON 4. Validate with swarm_validate_decomposition 5. Return ONLY the validated CellTree JSON (no analysis, no file contents) Output format: Valid CellTree JSON only. `, }); // 3. Parse result (subagent already validated) const beadTree = JSON.parse(planningResult); // 4. Create epic + subtasks atomically await hive_create_epic({ epic_title: beadTree.epic.title, epic_description: beadTree.epic.description, subtasks: beadTree.subtasks, }); ``` **Why This Matters:** - **Main thread context stays clean** - only receives final JSON, not reasoning - **Subagent context is disposable** - gets garbage collected after planning - **Scales to long swarms** - coordinator can manage 10+ workers without exhaustion - **Faster coordination** - less context = faster responses when monitoring workers ### Phase 4: File Ownership (CRITICAL RULE) **⚠️ COORDINATORS NEVER RESERVE FILES** This is a hard rule. Here's why: ```typescript // ❌ WRONG - Coordinator reserving files swarmmail_reserve({ paths: ["src/auth/**"], reason: "bd-123: Auth service implementation" }); // Then spawns worker... who owns the files? // ✅ CORRECT - Worker reserves their own files // Coordinator includes file list in worker prompt const prompt = swarm_spawn_subtask({ bead_id: "bd-123.4", files: ["src/auth/**"], // Files listed here // ... }); // Worker receives prompt with file list // Worker calls swarmmail_reserve themselves ``` **Why This Pattern:** | Coordinator Reserves | Worker Reserves |