| name | networking-design |
| description | Design rules for block-junk's networking — what gets replicated and how, channel choices, bandwidth budget, and area-of-interest strategy. Use when adding any feature that crosses the client/server boundary, or when reviewing a change that adds replicated state. The transport mechanics (which lightyear API to call) live in the lightyear-028 skill; this is about *what to send and why*. |
| user-invocable | false |
Networking design rules for block-junk
This is a project-locked architecture, not a list of options. Every cross-boundary feature must obey these rules unless we explicitly revisit them. The "why" matters because it's the only way to judge edge cases.
Always-client architecture (Quake/CS pattern)
There is no "host mode" in this project. Every player runs only as a client. Solo play just means we also spin up a server thread (a headless Bevy App listening on UDP-localhost) and the client connects to it. The wire format is identical to friends-mode.
Why: the alternative (lightyear's HostClient shared-world pattern) requires special-case dedup logic, observer-timing races between server-spawn and client-snapshot, two ChunkMap resources sharing one World, and an "are we host or split" branch in every cross-side feature. The cost of always-going-through-UDP-loopback is ~1 ms latency on a localhost packet — imperceptible. The benefit is one code path for solo, friends-host, and split modes. Networking bugs surface in the dev's own play session instead of "works in solo, breaks on someone else's machine."
The three modes:
| Command | What runs |
|---|
cargo run (default = solo) | Server thread + client App on main thread, connected via UDP-localhost. The server thread binds 0.0.0.0:5050 — every hosted world is LAN-joinable, no private solo bind. |
cargo run -- server [save] [--bind ip:port] | Dedicated headless server. Loads save if it exists (default name dedicated), binds 0.0.0.0:5050 unless overridden, quit-saves on Ctrl-C. |
cargo run -- client [ip:port] | Just the client. Connects to the given address (default 127.0.0.1:5050), skipping the menu. The menu's "Connect" field is the windowed equivalent. |
Friends-host = host from the menu (pause menu shows "Friends can join at :5050") or run the dedicated server somewhere reachable. Friends connect via the menu's address field or cargo run -- client <host>:5050. Identical code path to your own local client.
Connect-time gates (both must pass before play)
- Netcode
protocol_id (network::NETCODE_PROTOCOL_ID): transport-level; bump it on incompatible wire changes (message shapes, channels, replication set). Mismatch = handshake rejected before any app code.
- Mod-set manifest (
modset.rs, sent once on connect): slot↔id tables for blocks/items/recipes + sorted NPC-kind ids + a stable FNV-1a hash over serialized defs. Client diffs against its registries; any disagreement = blocking fatal modal + disconnect. Extend ModSetManifest when a new registry's content starts crossing the wire.
Client identity
Each install has a persistent random u64 (identity.rs, ./client_id.txt, advisory-locked; a second local instance falls back to an ephemeral id so two-client smoke tests work). The id is the netcode client_id AND the key for per-player persistence (SaveFile.players). It is a claim, not a proof — TOFU pubkey binding is the planned upgrade path if spoofing ever matters; the u64 stays the key, so no migration.
This rule is load-bearing — adding a "shortcut for local play that bypasses serialisation" because it seems faster is exactly the kind of choice that sneaks divergence in. Don't.
Bandwidth budget
Target ≤ 40 kbps per player after compression. That's roughly Minecraft-vanilla territory (sources: vanilla servers report 30–50 kB/s per server with 4–8 players, so a few kB/s per player; we target a comfortable overhead for the simulation depth we want).
Anything systemic that pushes past this needs a justification, not a workaround. If a feature requires more bandwidth, that feature gets cut to AoI sooner, throttled, or quantized harder.
Replicate events, not state, for the world grid
Voxel chunks are huge — 32³ = 32,768 blocks raw, ~32 KB even at one byte each. If we let lightyear's Replicate<Chunk> push the full component on every change, a single block edit replicates the entire chunk. That's catastrophic at scale and silently encouraged by the framework.
Don't replicate Chunk directly. Server broadcasts edits as events:
- Client sends
BlockEdit { coord, pos, block } to server (request).
- Server validates against its world, applies the change.
- Server broadcasts the applied
BlockEdit to all clients in the AoI of the affected chunk.
- Each client's authoritative state = (initial snapshot received on join) + (accumulated
BlockEdits since).
Initial chunk state is sent once per chunk per joining client as a ChunkSnapshot { coord, blocks } message. The client spawns a local chunk entity and applies subsequent edits to it.
This is event-sourcing for world state. Cost per edit on the wire: ~10–20 bytes vs ~32 KB.
Replicate state, not events, for entities
Players, NPCs, projectiles — anything with a continuous Transform that all clients need to render. Per-tick snapshots are small (~30 bytes after quantization) and lossy/unreliable is fine: the next tick's snapshot supersedes a dropped one. Use lightyear's Replicate machinery for these, with SequencedUnreliable channel underneath.
The split: discrete world changes are events; continuous entity state is replicated state. Apply this rule to every new replicated thing you add.
Identify chunks by coordinate, not by Entity
Server and client live in different worlds, so server's Entity IDs don't translate. We could solve this with MapEntities on every cross-side message, or we can sidestep it: chunks have a stable global identifier — their IVec3 chunk-grid coordinate. BlockEdit and ChunkSnapshot reference that coord.
Both sides keep a ChunkMap: HashMap<IVec3, Entity> resource so they can find the local entity for a chunk coord cheaply.
This rule extends to other replicated identities later: prefer stable IDs (PlayerId, NpcId) over Bevy entity IDs in any message that crosses the wire.
Channel choices
Four lanes as implemented (protocol.rs, registered in network.rs::ProtocolPlugin — one shared registration for both sides, so client/server mismatch is impossible):
| Channel | Mode | Carries | Why this lane |
|---|
WorldChannel | OrderedReliable | client→server edit/pickup/drop/deposit requests, ActionRejected, BlockManifest, NPC detail responses, dev commands | click-latency UX; must not queue behind bulk transfers |
ChunkChannel | OrderedReliable | ChunkSnapshot, ChunkUnload, and server→client BlockEdit broadcasts | the client skips edits for unloaded chunks trusting the snapshot to carry final state — snapshot, unload, and edit MUST share one ordered stream or a small edit overtakes a fragmented snapshot and is silently lost |
StateSyncChannel | OrderedReliable | PlanEdit/PlanEditBatch/PlanFullSync, station updates/full syncs | isolates 4096-cell plan batches and connect-time full syncs from the click-critical lane; preserves full-sync-before-delta |
PeriodicSyncChannel | SequencedUnreliable | WorldClockSync | latest sample wins; a drop is superseded by the next tick |
Player/NPC transforms ride lightyear's Replicate machinery (SequencedUnreliable underneath), not a hand-rolled channel.
Decision rule: if dropping the message causes incorrect state when later messages arrive, it's reliable. If a newer message supersedes the lost one, it's unreliable.
Ordering rule: reliability alone isn't enough when one message's handler assumes another already applied — "snapshot before edit," "full sync before delta." Cross-channel ordering does not exist; anything with an apply-order dependency must share a channel (see ChunkChannel above, and the client chains receive_snapshots before receive_block_edit_broadcasts for the same-frame case).
BlockEdit is reliable because dropping one means the client never agrees the block changed — all subsequent edits assume a wrong base. Player position is unreliable because dropping tick 100's snapshot is fine when tick 101's lands a frame later.
Don't replicate procedurally-regenerable state
If a chunk has never been edited and is fully reconstructible from the world seed + chunk coords, the server doesn't send the bytes — it sends "chunk K is procedural-default" and the client regenerates. Voxel-Tools recommends this exact pattern (info.are_voxels_edited()).
Implement this once we have multi-chunk + procedural worldgen. For now (single hand-built sphere), every chunk is "edited" and gets a snapshot.
Area of interest (AoI)
Not yet — single chunk. When multi-chunk lands, each player has a chunk-radius AoI (start at 8 chunks). Server only sends snapshots and edits for chunks within that radius.
Per Lysenko's classification, three viable styles:
- Rule-based — "the player can't see this, don't send" (dimensions, fog of war).
- Static partitioning — fixed grid; player subscribes to N nearest cells. This is what we'll use.
- Geometric — prioritize by projected area / distance / visual salience.
For a Minecraft-shaped game, static partitioning at the chunk grid is the natural fit: the partition exists for free (chunks already are the grid).
When to compress
- Per-message compression (zstd/lz4): only worth it for messages > ~256 B.
BlockEdit at ~15 B doesn't need it.
- Channel-level compression (zlib partial-flush, à la Minecraft): consider for the snapshot channel later. Lightyear may already include this — verify before adding.
- RLE for chunk snapshots: trivial win since most blocks are
Empty in early chunks. Implement when ChunkSnapshot is the bandwidth bottleneck, not preemptively.
Time dilation as graceful degradation
If bandwidth or sim-cost saturates, slow the simulation tick rather than dropping packets randomly or kicking players. EVE Online's pattern, mentioned by Lysenko: dilation should vary as the inverse square of the load so the game stays playable instead of disconnecting under stress.
Lightyear has tick-rate control. Defer until we see actual saturation.
Water sim (when it lands)
Naïve cell replication will blow the bandwidth budget instantly — water cells change every tick across many positions. Strategies, in order of preference:
- Don't replicate — run the same deterministic sim on every client from the same seed. Edits desync it; periodically sync. (Risky: drift accumulates.)
- Replicate the source/sink terms only — water-level changes that introduce new water (rain, broken dams) are the events; the propagation is computed locally. Much smaller wire payload.
- Spatial AoI + delta batching — only sync cells near players, batch all changes per tick into one unreliable message.
Likely we'll do (2) primarily, with (3) as a safety net. Decision deferred until we start the water sim.
Sources
