hardware-runtime

name	hardware-runtime
description	How code actually runs on hardware and the OS, language-agnostic: cache hierarchy and data layout (cache lines, locality, false sharing, AoS vs SoA, alignment), CPU microarchitecture (branch prediction, ILP, SIMD, speculative execution), atomics and memory ordering, OS scheduling (preemption, context switches, priorities, priority inversion, QoS, heterogeneous P/E cores), and threading runtime models (thread pools, work stealing, blocking hazards). Use when diagnosing cache- or scheduling-related slowness, false sharing, lock contention, priority inversion, hangs, or when writing performance-critical or lock-free code. Methodology parent: performance-engineering (measure first).
allowed-tools	["Read","Glob","Grep","Bash"]

Hardware & Runtime (Mechanical Sympathy)

IRON LAW: THE MACHINE EXECUTES YOUR DATA LAYOUT AND YOUR SCHEDULER'S
DECISIONS — NOT YOUR SOURCE CODE'S APPEARANCE. REASON FROM CACHE LINES,
CORES, AND RUN QUEUES; CONFIRM WITH A PROFILER (see performance-engineering).

Latency Orders of Magnitude (carry this table in your head)

Operation	~Cost	vs L1
L1 cache hit	~1 ns	1x
L2 hit	~4 ns	~4x
L3 hit	~10–40 ns	~10–40x
Main memory (DRAM)	~60–120 ns	~100x
Atomic RMW (contended)	~10–100+ ns	varies with sharing
Context switch	~1–10 µs	~10⁴x
NVMe read	~10–100 µs	~10⁵x
Same-DC network RTT	~100–500 µs	~10⁵–10⁶x

Implication: one cache miss costs ~100 arithmetic ops; one context switch costs thousands. Most "CPU-bound" slowness is memory-bound.

Memory Hierarchy & Data Layout

• Cache lines are the unit of transfer: 64 bytes on x86-64 and most ARM; 128 bytes on Apple Silicon. Touching 1 byte loads the whole line.
• Spatial locality: iterate contiguously; prefer arrays over pointer-rich graphs for hot loops. Linked lists are cache-miss generators.
• AoS vs SoA: if a hot loop touches one field of many records, structure of arrays packs useful data per line (this is also why ECS and columnar databases are fast).
• Alignment & padding: order struct fields by decreasing alignment to minimize padding; measure with the language's layout introspection.
• False sharing: two threads writing different variables on the SAME cache line serialize on coherence traffic. Symptom: parallel speedup far below core count with low lock contention. Fix: pad/align hot per-thread data to a cache line.
• Prefetching rewards predictable access patterns (linear, strided); random access defeats it. Sort indices before gathering when possible.
• TLB: massive working sets with random access pay page-walk costs; locality helps here too.

CPU Microarchitecture (what the core rewards)

• Branch prediction: predictable branches are nearly free; mispredictions cost ~10–20 cycles. Sort data to make branches predictable, or go branchless only when the profiler shows mispredicts.
• ILP / out-of-order: independent operations overlap; long dependency chains (e.g., pointer chasing, serial reductions) stall. Break reductions into parallel accumulators.
• SIMD: data-parallel loops over contiguous data vectorize (auto or intrinsics); branches and gathers break vectorization.
• Speculative execution leaks via cache side channels (Spectre-class): relevant when sandboxing untrusted code or handling secrets — constant-time comparison for secrets, no secret-dependent branches/indices (see secure-coding).

Atomics & Memory Ordering

• A data race (unsynchronized conflicting access, at least one write) is undefined behavior in C, C++, and Swift — not "just stale reads".
• Ordering levels: relaxed (atomicity only) < acquire/release (pairwise happens-before: release-store publishes to acquire-load) < seq_cst (single total order). Default to seq_cst/locks; weaken only with a stated invariant and a proven need.
• Contended atomics ping-pong cache lines (coherence cost) — sharded counters/per-thread accumulation beat one hot atomic.
• Lock-free ≠ fast: it trades throughput under contention for progress guarantees and complexity (ABA, memory reclamation). Prefer a well-tested mutex until measurement says otherwise.

OS Scheduling & Threads

• Preemptive scheduling: the OS time-slices runnable threads onto cores; more runnable threads than cores = queueing + context-switch overhead.
• Thread pools: size ≈ core count for CPU-bound work; oversubscription ("thread explosion") wastes memory (~MBs of stack each) and thrashes caches. I/O-bound work belongs on async I/O, not extra threads.
• Priorities & QoS: schedulers run higher-priority work first; on Apple platforms QoS classes (userInteractive → background) also steer heterogeneous cores — background QoS may run on efficiency (E) cores; user-interactive favors performance (P) cores. Tag work honestly.
• Priority inversion: low-priority holder blocks a high-priority waiter. Mutexes with priority inheritance (e.g., os_unfair_lock-based locks) bound it; semaphores and condition-variable waits convey no ownership, so the kernel cannot boost the holder — a classic hang source when bridging async code with semaphores.
• Context-switch hygiene: batch work per wakeup; coalesce timers; avoid ping-pong handoffs between threads for tiny work items.
• Work stealing runtimes (async executors, fork-join pools) keep cores busy with cheap task handoff — but blocking a pool thread (sync I/O, semaphore wait, long lock hold) starves the pool. Blocking calls get a dedicated thread or an async equivalent.

Diagnosis Cues (symptom → suspect)

Symptom	Suspect	Confirm with
Parallel speedup ≪ cores, locks "fine"	False sharing	Per-line counters (perf c2c, Instruments)
Slowness scales with data size, flat CPU%	Cache/memory bound	Miss counters, bandwidth profile
Random latency spikes under load	Oversubscription, priority issues	Scheduler/System Trace
UI hangs while CPU idle	Priority inversion / blocked pool	Wait-state trace, lock holder stacks
Fast single-thread, slow multi	Contended atomic/lock line ping-pong	Contention profilers

Related Skills

Need	Skill
Measurement methodology FIRST	performance-engineering
Algorithmic complexity before micro-tuning	algorithms
Swift specifics (layout, ARC, cooperative pool, QoS)	swift (memory-layout-performance, scheduling-qos)
GPU equivalent of this knowledge	metal, graphics
Memory corruption rather than performance	memory-safety