scalpel-ipc-transport

name	scalpel-ipc-transport
description	Use when working on scalpel-ipc, scalpel-alloc, shared memory ring buffer, Unix domain sockets, string interning protocol, custom GlobalAlloc, or any unsafe memory/IPC code

IPC & Transport Expert Knowledge

Core Principle

This is the most dangerous code in the project. Bugs here are silent — they don't crash, they corrupt measurements. Every unsafe block needs a // SAFETY: comment. Every memory ordering needs justification.

When to Use

• Modifying any file in crates/scalpel-ipc/ or crates/scalpel-alloc/
• Working on the SPSC ring buffer
• Modifying string interning protocol
• Touching the custom GlobalAlloc implementation
• Working with mmap, shared memory, or atomic operations
• Debugging corrupted measurements or memory attribution

Domain Model

Two Paths — Hot and Cold

HOT PATH (<100ns budget per event):
  Probe → atomic write to ring buffer slot → Tracer reads batch of 256
  Data: 64-byte TraceEvent, fixed-size, no serialization
  Transport: POSIX shared memory via mmap (/dev/shm/scalpel-{pid})

COLD PATH (~2-5us per unique string):
  Probe → Unix socket INTERN request → lasso::ThreadedRodeo → INTERN_ACK with u32 ID
  Data: variable-length string, cached after first resolution
  Transport: Unix domain socket (SOCK_STREAM)

Ring Buffer Memory Layout

Byte offset:
[0..8]     u64 write_cursor
[8..128]   padding (120 bytes — NOT 56)
[128..136] u64 read_cursor
[136..256] padding (120 bytes)
[256..]    u8[N] data slots (N must be power of two)

128-byte separation between cursors. NOT 64. The adjacent-line prefetcher on x86 pulls the next cache line automatically, so 64-byte padding still causes false sharing. Use #[repr(C, align(128))].

Decision Trees

Memory Ordering for SPSC Ring Buffer

Writing DATA to a ring buffer slot:
  → Relaxed is sufficient (happens-before established by cursor update)

Updating WRITE CURSOR after writing data:
  → MUST be Release (publishes all preceding data writes to consumer)

Reading WRITE CURSOR to check for new data:
  → MUST be Acquire (synchronizes with producer's Release)

Reading DATA after seeing updated cursor:
  → Relaxed is sufficient (ordered after the Acquire fence)

Do you need SeqCst?
  → NO. SPSC has one producer, one consumer. No third-party observer.
  → SeqCst is for multi-observer consensus. Don't use it here.

GlobalAlloc Re-entrancy Guard

GlobalAlloc::alloc() called:
  Read thread-local IN_ALLOCATOR flag (Cell<bool>)
    → Flag is TRUE?
      → Pass through to underlying allocator. No attribution. (re-entrant call)
    → Flag is FALSE?
      → Set flag TRUE
      → Read current node ID from thread-local stack
      → Increment per-node counter (AtomicU64, Relaxed ordering)
      → Clear flag FALSE
      → Call underlying allocator

CRITICAL: Counter storage is a fixed-size AtomicU64 array.
          Sized at startup. Uses libc::malloc, NOT the tracked allocator.
          HashMap, Vec, String, format!() in allocator code = infinite recursion.

Known Traps

1. False sharing at 64-byte padding

Adjacent-line prefetcher pulls the next cache line (64 bytes beyond the one accessed). Two threads touching adjacent cache lines still thrash. Use 128-byte separation between producer and consumer metadata. #[repr(C, align(128))], not align(64).

2. Allocator re-entrancy

Any allocation inside GlobalAlloc causes infinite recursion or deadlock. This includes: String, Vec, HashMap, format!(), println!(), tracing::debug!(). All internal bookkeeping must use libc::malloc/libc::free directly or pre-allocated fixed-size arrays.

3. mmap cleanup in Drop

Drop impl MUST call munmap() then shm_unlink(). Leaking mmap'd regions leaks virtual address space permanently. Even if the process exits, /dev/shm/scalpel-{pid} files persist until explicitly unlinked.

4. TraceEvent size drift

Any field addition to TraceEvent breaks 64-byte cache-line alignment. The entire hot path assumes one event = one cache line. Compile-time guard:

const _: [(); 64 - std::mem::size_of::<TraceEvent>()] = [];

This fails at compile time if TraceEvent is not exactly 64 bytes.

5. Power-of-two buffer size

Required for bitwise AND masking: index & (capacity - 1) instead of expensive modulo index % capacity. Non-power-of-two capacity silently wraps to wrong slots, corrupting data without any error.

Verification Protocol

After ANY change to scalpel-ipc or scalpel-alloc:

1. cargo test -p scalpel-ipc -p scalpel-alloc
2. cargo miri test -p scalpel-ipc (if nightly available) — checks undefined behavior in unsafe
3. Stress test: 1M events through ring buffer, verify zero data loss
4. Verify TraceEvent is exactly 64 bytes (compile-time assertion present)
5. Verify cursor padding is 128 bytes (inspect struct layout)
6. Check ALL unsafe blocks have // SAFETY: comments

After allocator changes: Run a full trace and verify memory attribution numbers are non-zero and reasonable.

Red Flags — STOP

• Any code in GlobalAlloc that could allocate (String, Vec, format!, println!, tracing macros)
• Using Relaxed ordering on cursor updates (must be Release/Acquire)
• Adding fields to TraceEvent without checking 64-byte constraint
• Using 64-byte padding between shared atomics (need 128)
• Missing Drop impl on mmap'd regions
• Using SeqCst without justification (SPSC doesn't need it)
• Non-power-of-two buffer capacity
• Missing // SAFETY: comment on any unsafe block

Quick Reference

Operation	Rule
Cursor write (producer)	Release ordering
Cursor read (consumer)	Acquire ordering
Data slot access	Relaxed (guarded by cursor fence)
Padding between atomics	128 bytes, not 64
Buffer capacity	Must be power of two
TraceEvent	Exactly 64 bytes (compile-time assert)
Allocator internals	libc::malloc only, no Rust allocations
mmap regions	Must have Drop calling munmap + shm_unlink
New unsafe block	SAFETY comment required, no exceptions