n8n-orchestration-patterns

name	n8n-orchestration-patterns
description	Fan-out / fan-in, parallel sub-workflow execution, and batch-orchestrator patterns in n8n. Use when a workflow processes a LIST in parallel, calls sub-workflows, needs faster wall-clock on many items, has a webhook that would otherwise block for minutes, or asks "how do I run sub-workflows in parallel / fan out / fan in".
user-invocable	false

n8n Orchestration Patterns (fan-out / fan-in / batch)

Battle-tested against the n8n docs (main) and real production workflows. The naive approach (branch-split) does NOT parallelize under the modern default — read the trap first.

---

The trap: branch-split alone does NOT parallelize

Mechanic	Behaviour
executeWorkflow mode: each	runs sequentially, one input item at a time
executeWorkflow mode: once	all items in one sub-workflow call
Workflow executionOrder: 'v1' (default ≥ 1.0)	completes one branch before starting the next
Workflow executionOrder: 'v0' (legacy)	first node of every branch, then second node of every branch — layer-by-layer, all branches interleaved
N8N_CONCURRENCY_PRODUCTION_LIMIT	does NOT apply to sub-workflow executions

So: 5 branches each [Filter → executeWorkflow] under executionOrder: 'v1' run serially — wall-clock = sum of all branches = no speedup. executeWorkflow mode: each is sequential per item.

---

Pattern A — executionOrder: 'v0' + branch-split (regular mode, no queue)

Layer-by-layer scheduling means when branch 0's executeWorkflow awaits (e.g. an LLM call), the engine schedules branch 1's, then 2,3,4 → all sub-workflow calls are in-flight at once (Node async I/O).

@workflow({
  name: 'Parallel Orchestrator',
  settings: {
    executionOrder: 'v0',     // ← CRITICAL — without this, branches run serially
    executionTimeout: 7200,
  },
})

Shape: a Code node assigns bucket: idx % N → N Filter nodes split by bucket → N executeWorkflow nodes → Merge (mode: append, numberInputs: N). Speedup ≈ N branches.

Caveat: v0 interleaving was less aggressive than expected for some (n8n issue #13620, closed "not planned"). On production-critical paths: measure, don't assume. Guaranteed N× only via queue mode.

Pattern B — Wait=OFF + DataTable fan-in ← RECOMMENDED

Cleaner than a resumeUrl callback: no webhook between workflows (org veto), no HMAC-signature pain, and a persistent audit trail. See n8n-autopilot:data-tables for the table CRUD.

1. Parent generates a batch_id per run, counts N = list length.
2. Fan-out — one executeWorkflow node, mode: each + Wait for Sub-Workflow Completion: false. Fires all sub-workflows async, no return. Pass batch_id + item_idx to each.
3. Each sub-workflow ends by writing a DataTable row: batch_id, item_idx, status, result.
4. Fan-in — Parent: a Wait node + loop polls COUNT(*) WHERE batch_id = X until >= N (or timeout), then reads the rows and merges.

Error-state (critical): without an error row the parent polls forever. Do NOT use continueOnFail: true (masks silent failures — vetoed). Instead set the risky node's "On Error" → "Continue (using error output)" and route the error output to a DataTable write (status: 'error', error text). Count still reaches N; the failure is logged and visible.

Gotchas:

• Sub-workflow retry → duplicate row: upsert on (batch_id, item_idx), not insert — see n8n-autopilot:data-tables for the exact upsert shape (filters.conditions + matchingColumns).
• Poll loop vs EXECUTIONS_TIMEOUT: add a max-iterations guard + a timeout branch that returns partial results.
• 100 items = 100 concurrent executions; N8N_CONCURRENCY_PRODUCTION_LIMIT does NOT cap sub-workflows → cap via queue-mode worker --concurrency instead.
• Count-query race: DataTable insert is atomic per row, append-only count is safe — no lock needed.

Pattern B2 — official resumeUrl fan-out/fan-in (veto grey zone)

n8n's official answer (templates 2536, 6247). executeWorkflow Wait=false + pass $execution.resumeUrl (+ unique suffix per item); parent pauses at a Wait node (resume: On Webhook Call); sub-workflow HTTP-POSTs the result to the resumeUrl → instant resume instead of polling. Cost: it IS a webhook child→parent (collides with "no webhook between workflows") and carries HMAC-signature pain. Use only if that veto is lifted for the case.

Pattern C — Queue mode (true parallelism, infra change)

EXECUTIONS_MODE=queue + ≥1 worker with --concurrency=10 (Redis + worker pods). Sub-workflow calls become queue jobs, processed in parallel. The only guaranteed N× speedup. Self-hosted/k8s: set via Helm values.

---

Synchronous batch + fast-return webhook

When a webhook triggers heavy batch work, do NOT let the webhook connection block for minutes (a real batch orchestrator once blocked 41 min). Two valid shapes:

• Fast-return: webhook immediately responds 202 { batch_id } via respondToWebhook, then the batch runs detached; the caller polls a status endpoint / DataTable by batch_id.
• Synchronous batch orchestrator: when the item count is bounded and fits inside EXECUTIONS_TIMEOUT, a single synchronous orchestrator (no queue/worker) is simpler and was chosen over an async queue/worker split in EV-Workflows once the queue's complexity outweighed its benefit. Pick the simplest shape that fits the timeout — don't add a queue you don't need.

---

Anti-patterns

• ❌ Branch-split with executionOrder: 'v1' — serial, zero speedup.
• ❌ Treating executeWorkflow mode: each as "parallel per item" — docs say sequential.
• ❌ Webhook-trigger sub-workflow (or HTTP node hitting /webhook/…) between your own workflows — org veto.
• ❌ continueOnFail: true to "keep the batch going" — masks silent failures; use the error-output branch.