benchmarking-resources

name	benchmarking-resources
description	Measures and compares resource usage between the n8n instance and the translated pure code application. Quantifies CPU, memory, response time, and cost savings. Activate when testing is complete and the user wants to benchmark, compare performance, measure resources, or calculate cost savings.

Benchmarking Resources

You are measuring the concrete benefits of migrating from n8n to pure code. The user needs hard numbers to justify the migration effort.

---

0. Methodology — 3 Layers, In Order

A naive idle-snapshot comparison is misleading: production n8n serves many users under variable load, while a fresh dev container serves zero. Always run all three layers and present them together.

Layer	What it measures	Source	Why
1. Idle baseline	RAM + steady-state CPU when no requests are flowing	cgroup cpu.stat over a multi-minute window	Honest "what does it cost to keep this running"
2. Per-interaction	Wall time + CPU per representative interaction	Real execution data (n8n) + admin-trigger calls (dev)	Honest per-request cost
3. Extrapolation	Lifetime CPU at production load	Layer-1 baseline × (production interactions / dev interactions) + layer 2 deltas	Honest "what would this cost at production scale"

Before designing the harness, STOP and confirm with the user that the fairness model below matches their definition of fair. Production usually has many more users than dev — explain how you will normalize.

Use cgroup cpu.stat, not docker stats --no-stream

docker stats --no-stream is a 1-second snapshot. It alternates wildly for busy or bursty containers (postgres routinely shows 0% then 100% in successive samples). Use cumulative cgroup CPU time as the authoritative number:

# Read cumulative CPU usage in microseconds
docker exec <container> cat /sys/fs/cgroup/cpu.stat 2>/dev/null \
  || docker exec <container> cat /sys/fs/cgroup/cpuacct/cpuacct.usage 2>/dev/null

# Container start time
docker inspect <container> --format '{{.State.StartedAt}}'

Lifetime average CPU = usage_usec / ((now - started_at) * num_cores * 1_000_000).

For shorter windows, take two cpu.stat snapshots N seconds apart and divide the delta. This is immune to the 1-second alignment artifacts of docker stats.

---

1. Baseline: n8n Resource Usage

From Discovery Phase (already collected)

Pull from migration-state/discovery/:

• server-baseline.txt — n8n container CPU/MEM, total server resources
• cgroup-baseline.json — cumulative CPU and start time per container (PREFER this)
• host-shape.json — vCPU / RAM / droplet size (for cost model)
• db-row-counts.json — production user counts (for extrapolation)
• container-tz.txt — n8n container timezone

If any of these are missing because SSH was not provided during discovery, ask the user to run the relevant commands now (see discovering-n8n-instance §3d.1 through §3d.3) — do NOT proceed with a partial baseline.

If SSH was not provided during discovery, ask now:

"To compare resource usage, I need to measure your current n8n instance. Can you provide SSH access, or can you run these commands and paste the output?"

docker stats --no-stream
free -h
top -bn1 | head -5

Additional Baseline Metrics

Ask the user to trigger a representative workflow while you measure:

# On the server — monitor n8n during a workflow execution
ssh user@ip "docker stats n8n --no-stream --format '{{.CPUPerc}}\t{{.MemUsage}}'"

Record:

• Idle CPU/memory (n8n doing nothing)
• Active CPU/memory (n8n processing a request)
• Response time (if webhook-based, measure with curl):

time curl -s -X POST https://<n8n_url>/webhook/<path> \
  -H "Content-Type: application/json" \
  -d '{"test": true}'

---

2. Measure: Pure Code Resource Usage

Start the Translated Application

cd workspace
docker compose up -d
sleep 5
docker stats --no-stream

Record idle CPU and memory.

Load Test

Send the same representative requests to the translated app:

# Single request timing
time curl -s -X POST http://localhost:3000/webhook/<path> \
  -H "Content-Type: application/json" \
  -d '{"test": true}'

# Burst test — 50 sequential requests
for i in $(seq 1 50); do
  curl -s -o /dev/null -w "%{time_total}\n" \
    -X POST http://localhost:3000/webhook/<path> \
    -H "Content-Type: application/json" \
    -d '{"test": true}'
done | tee migration-state/benchmarks/response-times.txt

# Monitor during load
docker stats --no-stream --format '{{.Name}}\t{{.CPUPerc}}\t{{.MemUsage}}'

---

3. Calculate and Compare

Generate migration-state/benchmarks/comparison.json:

{
  "n8n": {
    "idle_cpu_percent": 0.0,
    "idle_memory_mb": 0,
    "active_cpu_percent": 0.0,
    "active_memory_mb": 0,
    "avg_response_time_ms": 0,
    "p95_response_time_ms": 0
  },
  "pure_code": {
    "idle_cpu_percent": 0.0,
    "idle_memory_mb": 0,
    "active_cpu_percent": 0.0,
    "active_memory_mb": 0,
    "avg_response_time_ms": 0,
    "p95_response_time_ms": 0
  },
  "savings": {
    "memory_reduction_percent": 0.0,
    "cpu_reduction_percent": 0.0,
    "response_time_improvement_percent": 0.0,
    "estimated_monthly_cost_savings_usd": 0.0
  }
}

Cost Estimation

Current server: [size] at $[cost]/month
n8n uses: [x]% of resources

After migration, n8n is removed:
  - If current server is sufficient: \$0 additional cost, [x]% resources freed
  - If server can be downsized: potential savings of $[y]/month
  - If multiple workflows were consuming most resources:
    new app uses [z]% → could run on a $[smaller]/month server

---

4. Present Benchmark Report

Benchmark Report
================

                    n8n          Pure Code      Change
Idle Memory:        [x] MB       [y] MB         -[z]%
Active Memory:      [x] MB       [y] MB         -[z]%
Idle CPU:           [x]%         [y]%           -[z]%
Active CPU:         [x]%         [y]%           -[z]%
Avg Response:       [x] ms       [y] ms         -[z]%
P95 Response:       [x] ms       [y] ms         -[z]%

Estimated Monthly Savings: $[amount]
  Current: [server_size] @ $[cost]/mo
  Recommended: [smaller_size] @ $[cost]/mo (or keep current with [x]% headroom)

Additional Benefits:
  - Code is version-controlled and reviewable
  - Each function is independently testable
  - No n8n platform dependency or upgrade risk
  - Direct debugging with standard Python tools
  - Faster feature development (no visual editor constraints)

Then ask: "Benchmarking complete. Ready to proceed to Phase 7 (Migration Plan)?"

---

5. Fairness Caveats — Always Report Them

Any production-vs-dev comparison has confounders that cannot be eliminated. List them honestly in the report so the numbers are interpretable.

Default caveats template (delete or expand as appropriate):

Fairness caveats — read before interpreting headline numbers:

1. Production has [N] users; dev was measured with [M] simulated users.
   Per-interaction numbers are normalized; idle baselines are not.

2. Production n8n has been running [D] days and accumulated runtime state
   (caches, OS page cache, JVM/V8 warm-up). Dev was [E] hours old at
   measurement time. Long-tail RAM growth in n8n may be under-counted.

3. Sampling window: [start] to [end], local time [tz]. Production traffic
   is variable; this window may over- or under-represent typical load.

4. The dev app has [K] fewer integrations wired than production
   (e.g., mocked integrations [list]). RAM and CPU savings should be
   discounted accordingly.

5. Cost model uses [provider] [pricing date] sticker prices. Reserved or
   committed-use discounts are not factored in.

6. Network: production users hit [hostname] over the public internet;
   dev measurements were taken from [where]. Wall-clock latency comparisons
   exclude / include [X] ms of network overhead.

Always present cost savings as a range ("$5–$8/month savings depending on droplet downsizing strategy"), not a single number.

---

6. Anticipated Questions — Answer Without Asking

Question	Default answer instead of asking
"Sample now or wait for quiet hours?"	Now, with a fairness caveat noting the time window. Quiet-hours sampling biases the comparison toward n8n. Only ask if the user explicitly mentions traffic spikes.
"How many active users does production have?"	Pull from migration-state/discovery/db-row-counts.json (captured in discovery §3c.2).
"What droplet/VM is this?"	Pull from migration-state/discovery/host-shape.json (captured in discovery §3d.3).
"What pricing should I use?"	Use the provider pricing table maintained alongside this skill (e.g., DigitalOcean Basic droplet tiers). Note pricing date in the caveat.
"How long should I sample?"	Default 10 minutes for layer 1 idle, 3 trials per scenario for layer 2. Longer only if the user has reason to believe their workload is bursty.