multi-llm-orchestrator

name

multi-llm-orchestrator

description

Multi-LLM orchestration and consensus synthesis workflow. Use when the user asks a complex, open-ended question that would benefit from multiple AI perspectives — such as investment decisions, business strategy, career planning, product design, or risk assessment. The workflow broadcasts the question to 3 specialized LLM roles (planner, expert, evaluator), synthesizes their outputs into a consensus/divergence analysis, presents an interactive synthesis card for user feedback, then runs a second round of refinement and generates a final unified plan.

Multi-LLM Orchestrator

Overview

This skill orchestrates multiple LLM sub-agents with differentiated cognitive roles to produce higher-quality, more robust answers than any single model can achieve alone.

Core workflow:

1. Round 1 Broadcast — Spawn 3 models in parallel with distinct roles
2. Synthesize & Visualize — Extract consensus/divergence and generate a Feishu interactive card
3. User Feedback — Capture user constraints or preferences
4. Round 2 Refinement — Spawn targeted models based on feedback
5. Final Synthesis — Aggregate all outputs into a unified, actionable plan

When to Use

Trigger this skill for questions that are:

• Multi-dimensional (e.g. strategy + execution + risk)
• High-stakes (investment, career, business decisions)
• Open-ended with no single correct answer
• Benefiting from critique (user wants to avoid blind spots)

Standard Role Pool (3+1)

Role	Cognitive Style	Temperature	Purpose
系统规划者	Structured, process-oriented	0.2-0.4	Execution paths, timelines, milestones
领域专家	Deep knowledge, industry-savvy	0.4-0.6	Best practices, frameworks, tools
批判评估者	Risk-sensitive, assumption-challenger	0.3-0.5	Failure modes, boundary conditions, worst-case scenarios
创意拓展者 (optional)	Divergent, cross-domain	0.6-0.8	Non-obvious alternatives, paradigm breaks

Principle: Roles are differentiated by cognitive style, not artificially polarized. Prompts emphasize constructive contribution rather than adversarial negation.

Workflow

Step 1: Round 1 Broadcast

Spawn 3 sub-agents in parallel using sessions_spawn with runtime="subagent" and mode="run".

Assign models from the pool defined in references/model-pool-config.json.

Prompt each agent with its role template from references/prompt-templates.md, prepended to the user's question.

Step 2: Synthesize

After all 3 agents return:

1. Manually extract 3-5 high-confidence consensus points
2. Identify 2-4 dimensions of meaningful divergence
3. Summarize each agent's stance per dimension

Step 3: Generate Feishu Card

Run the card generator:

python scripts/generate_synthesis_card.py \
  --input synthesis-data.json \
  --output card.json

Input JSON schema:

{
  "round": 1,
  "question_summary": "...",
  "participants": [{"agent_id": "...", "role": "...", "model": "..."}],
  "responses": [{"agent_id": "...", "role": "...", "content": "..."}],
  "consensus_points": ["..."],
  "divergences": [{"dimension": "...", "agents": [{"agent_id": "...", "role": "...", "stance": "..."}]}]
}

Send the generated card to the user and wait for feedback.

Step 4: Capture Feedback

Accept either:

• Button clicks (feedback_focus, feedback_risk, feedback_budget, proceed_final)
• Free-text user reply

If the user chooses proceed_final, skip to Step 6.

Step 5: Round 2 Refinement

Spawn 2-3 new sub-agents with roles tailored to the feedback:

• "更聚焦" → domain specialist + strategist
• "增加风险分析" → risk controller + compliance expert
• "补充预算约束" → financial planner + execution expert

Inject the Round 1 consensus summary + user feedback into each prompt.

Step 6: Final Synthesis

Spawn a final aggregator sub-agent with the prompt template from references/prompt-templates.md (终轮聚合专家 section).

Deliver the final unified plan to the user.

Model Selection Rules

• Models can be mixed freely across rounds (e.g. Round 1 = kimi + GLM, Round 2 = Claude + GPT-4)
• Same model + different role is valid and useful for testing prompt isolation
• Dynamic enablement: any role can be disabled; adjust synthesis strategy if fewer than 3 agents participate

Bundled Resources

scripts/

• generate_synthesis_card.py — Generates Feishu interactive card JSON from synthesis data

references/

• prompt-templates.md — Standard role prompts and aggregation prompt
• model-pool-config.json — Default model pool configuration (temperatures, weights, enable flags)

assets/

• feishu-synthesis-card-template.json — Base Feishu card template with variable placeholders

Important Notes

• Do not poll sub-agents in a loop. After spawning, use sessions_yield and wait for push-based completion events.
• If a child completion arrives after your final answer, reply with NO_REPLY only.
• Consensus threshold: Only display points supported by ≥67% of participants.
• Divergence truncation: Display at most 4 divergence dimensions; truncate cell text to 15 Chinese characters in the comparison table.
• Data latency warning: Always include a disclaimer when the aggregated output involves real-time data (stock prices, news, etc.).