| name | realtime-scaling |
| description | Scale a realtime backend horizontally with pub/sub fanout and presence tracking. Use when moving past a single-node WebSocket/SSE server. |
Realtime Scaling
Instructions
Realtime backends are stateful: each user's connection lives on one node. Horizontal scale is a fanout + routing problem, not a stateless load-balance problem.
1. Node Anatomy
Each realtime node holds:
- A pool of open connections (WebSocket or SSE).
- A map from
channel -> [connection] (local subscribers).
- A reverse map from
connection -> [channel] (for cleanup on disconnect).
No user state lives only in one node -- persist whatever must survive a node restart.
2. Pub/Sub Fanout
Every business-side event publishes once to a pub/sub bus; every node filters against its local subscriber map.
publisher -> bus(channel=X) -> [nodeA, nodeB, nodeC]
| | |
v v v
matching local connections dispatch
Options:
- Redis Pub/Sub: simplest, at-most-once, best for small-to-mid scale.
- Redis Streams: durable, consumer groups, good for replay.
- NATS: low-latency, subject wildcards, clustered.
- Kafka: durable, ordered per partition, best for analytics-grade fanout.
For most mobile apps under a few hundred thousand concurrent connections, Redis (Pub/Sub + Streams) is sufficient.
3. Channel Naming
Hierarchical, coarse first:
user:{user_id} -- direct messages, badges
thread:{thread_id} -- chat channel
entity:{type}:{id} -- per-object updates
broadcast:{tenant} -- fanout to all users in a tenant
Avoid per-connection channels -- they do not fan out and increase subscription churn.
4. Subscription Management
When a client subscribes, the node checks whether it is already subscribed to that bus topic. Maintain a refcount:
func (n *Node) Subscribe(conn *Conn, ch string) {
if n.refs[ch] == 0 { n.bus.Subscribe(ch) }
n.refs[ch]++
n.channels[ch] = append(n.channels[ch], conn)
}
func (n *Node) Unsubscribe(conn *Conn, ch string) {
n.refs[ch]--
if n.refs[ch] == 0 { n.bus.Unsubscribe(ch) }
}
One bus subscription per node per channel. Internal fanout is O(subscribers on node).
5. Presence
Presence = "who is online in channel X right now". Cheap versions use Redis sorted sets keyed by channel, scored by last heartbeat timestamp:
ZADD presence:thread:thr_01HX7 <now_ms> <user_id>
ZREMRANGEBYSCORE presence:thread:thr_01HX7 0 <now_ms - 30_000>
ZRANGE presence:thread:thr_01HX7 0 -1 -- currently online
The node updates the user's score on every heartbeat. A separate janitor cleans stale entries every 30 s.
For large channels, do not broadcast "X is online" to everyone; push presence diffs or count-only updates.
6. Cross-Device Fanout
A user has N devices connected to possibly N different nodes. Publishing to user:{user_id} reaches all of them via the bus. No extra logic needed if channel naming is correct.
7. Sticky Sessions and Routing
Balance connections across nodes uniformly (consistent hashing by device_id) or randomly. Sticky sessions are not required if the bus fans out properly, but they help in two cases:
- Fewer bus subscriptions when all devices of a user land on the same node (hash by
user_id).
- Reconnection warm-path -- the same node often still has recent history cached.
8. Capacity Planning
Rules of thumb per node (modern hardware, well-tuned):
- Node.js / Go / Rust: 50k–100k idle WebSocket connections per 2 vCPU / 4 GiB node.
- Memory is usually the first bottleneck (~20–40 KiB per connection including TLS state + buffers).
- Outbound bandwidth: plan for the 95th-percentile channel fanout, not the mean.
Scale by adding nodes behind the LB; the bus absorbs fanout.
9. Failure Modes
- Bus outage: connections remain open but events stop flowing. Expose a health check that fails on bus disconnect so the LB drains the node.
- Thundering herd on deploy: stagger node restarts; add a startup warm-up delay before re-subscribing clients.
- Zombie connections: heartbeat timeout + OS-level TCP keepalive must both be set, otherwise half-open connections eat capacity.
- Redis Pub/Sub is not durable: if an event must survive subscriber absence, use Redis Streams with
XADD + consumer groups, not PUBLISH.
10. Observability
Per node: open connections, messages/sec (in/out), bus lag, per-channel fanout p95, CPU %, memory. Alert on concurrent-connection drops > 10% minute-over-minute (likely bug or bus issue, not user behavior).
Checklist
