subsystem-summary-of-util

# Util Subsystem — Technical Summary The `src/util/` directory provides foundational infrastructure used across all of stellar-core. It covers timing, scheduling, logging, numeric safety, filesystem operations, serialization, data structures, threading, and various small helpers. --- ## 1. Virtual Clock & Timers (`Timer.h`, `Timer.cpp`) ### `VirtualClock` Central timing facility with two modes: `REAL_TIME` (wall clock) and `VIRTUAL_TIME` (simulated, for tests). Owns an `asio::io_context` for IO dispatch and a `Scheduler` for main-thread action scheduling. - **Key types**: `time_point` (steady clock), `system_time_point` (wall/calendar time), `duration`. - **`crank(bool block)`**: The main event-loop step. Dispatches pending timers, polls IO (with exponential priority biasing under overload), runs scheduled actions, and transfers items from the thread-safe pending queue to the scheduler. In `VIRTUAL_TIME` mode, advances time to the next event if idle. - **`postAction()`**: Thread-safe submission point for deferred work callbacks. Uses a mutex-protected pending queue; wakes the main thread via `asio::post` when enqueueing into an empty queue. - **`setCurrentVirtualTime()`**: Advance virtual time forward (monotonic; asserts forward progress). - **`sleep_for()`**: Real sleep in `REAL_TIME`, virtual time advance in `VIRTUAL_TIME`. - **`shouldYield()`**: In `REAL_TIME`, returns true after 500ms has elapsed (time-slice limit). Always returns true in `VIRTUAL_TIME`. - **Time conversion helpers**: `to_time_t`, `from_time_t`, `systemPointToTm`, `tmToSystemPoint`, `isoStringToTm`, `tmToISOString`, `systemPointToISOString`. ### `VirtualTimer` Timer coupled to a `VirtualClock` that uses virtual time. Supports `expires_at()`, `expires_from_now()`, `async_wait()`, `cancel()`. Typical usage for all delayed / periodic operations in core. ### `VirtualClockEvent` Internal event representation stored in the clock's priority queue. Carries a callback, trigger time, sequence number, and cancelled state. --- ## 2. Scheduler (`Scheduler.h`, `Scheduler.cpp`) A multi-queue fair scheduler implementing a variant of the LAS (Least Attained Service) / FB (Foreground-Background) algorithm. ### Design - Multiple named `ActionQueue`s each track cumulative runtime (`mTotalService`). - On each `runOne()`, the queue with the **lowest** accumulated service time runs its next action. - A latency window (default 5s) serves three roles: (1) credit cap to prevent bursty monopolization, (2) overload detection threshold, (3) idle queue expiration. - Droppable actions are shed under overload (queue time exceeds latency window). ### `ActionQueue` (private inner class) - Stores a deque of `Element{Action, enqueueTime}`. - Tracks `mTotalService`, `mLastService`, idle list membership. - `tryTrim()`: drops droppable actions that exceed the latency window. - `runNext()`: runs the front action and updates total service (floored at `minTotalService`). ### `Scheduler` public interface - `enqueue(name, action, type)`: Creates or reactivates a queue and appends the action. - `runOne()`: Dequeues from the lowest-service queue, trims stale idle queues. - `getOverloadedDuration()`: Reports how long any queue has been overloaded. - `currentActionType()`: Returns the `ActionType` of the currently executing action. - `shutdown()`: Clears all queues. --- ## 3. Logging (`Logging.h`, `Logging.cpp`, `SpdlogTweaks.h`, `LogPartitions.def`) ### `Logging` class Static utility managing spdlog-based logging with partitioned loggers. - **Partitions**: Defined in `LogPartitions.def` via X-macro (`LOG_PARTITION(name)`), yielding ~15 named partitions (e.g., Bucket, Herder, Ledger, Overlay, Tx, etc.). Each partition has its own spdlog logger with independent log level. - **`init()`**: Creates sinks (stderr + optional file), registers all partition loggers, configures flush behavior (every 1s, immediate on error). Also initializes Rust-side logging. - **`setLogLevel(level, partition)`**: Per-partition or global level setting. - **`setLoggingToFile(filename)`**: Supports `{datetime}`-formatted filenames. Opens in append mode (`O_APPEND`) for durability with external log rotation. - **`setFmt(peerID)`**: Sets the log pattern including a peer identifier. - **`rotate()`**: Log rotation support. ### Log macros - `CLOG_TRACE/DEBUG/INFO/WARNING/ERROR/FATAL(partition, fmt, ...)`: Partition-aware logging with compile-time format string checking via `FMT_STRING`. - `LOG_TRACE/DEBUG/INFO/WARNING/ERROR/FATAL(logger, fmt, ...)`: Logger-handle-based logging. - `LOG_CHECK`: Guards log calls with `should_log()` check to avoid formatting overhead. ### `CoutLogger` Fallback logger when spdlog is not available. Writes to `std::cout`. ### `LogLevel` enum `LVL_FATAL(0)` through `LVL_TRACE(5)`. ### `format_as` overloads Provides `fmt`-compatible formatting for XDR enum types and `std::filesystem::path`. --- ## 4. Numeric Utilities (`numeric.h`, `numeric.cpp`, `numeric128.h`) Safe arithmetic for financial calculations where overflow matters. ### 64-bit operations (`numeric.h`) - **`bigDivide(result, A, B, C, rounding)`**: Computes `A*B/C` safely when `A*B` would overflow `int64_t`. Returns success/failure bool. - **`bigDivideOrThrow(A, B, C, rounding)`**: Throwing variant. - **`bigDivideUnsigned()`**: Unsigned variant. - **`bigSquareRoot(a, b)`**: Integer square root of `a*b`, `ROUND_DOWN` only. - **`saturatingMultiply(a, b)`**: Caps at `INT64_MAX` on overflow (non-negative inputs). - **`saturatingAdd(a, b)`**: Unsigned, caps at type max. - **`isRepresentableAsInt64(double)`**: Checks safe double→int64 conversion. - **`doubleToClampedUint32(double)`**: Clamps double to uint32 range. - **`Rounding` enum**: `ROUND_DOWN`, `ROUND_UP`. ### 128-bit operations (`numeric128.h`) - **`bigMultiply(a, b)` / `bigMultiplyUnsigned(a, b)`**: Returns `uint128_t`. - **`bigDivide128()` / `bigDivideUnsigned128()`**: Divide 128-bit by 64-bit. - **`hugeDivide(result, a, B, C, rounding)`**: Computes `a*B/C` when `C < INT32_MAX * INT64_MAX`. --- ## 5. Filesystem Utilities (`Fs.h`, `Fs.cpp`, `FileSystemException.h`, `TmpDir.h`) ### `fs` namespace - **File locking**: `lockFile()` / `unlockFile()` using `flock` (POSIX) or `CreateFile` (Win32). - **Durable I/O**: `flushFileChanges()` (fsync), `durableRename()` (rename + dir fsync), `openFileToWrite()`. - **Directory ops**: `exists()`, `deltree()`, `mkdir()`, `mkpath()` (recursive), `findfiles()`. - **Path construction**: `hexStr()`, `hexDir()`, `baseName()`, `remoteName()`, `remoteDir()` — used for history archive paths. - **Handle counting**: `getMaxHandles()`, `getOpenHandleCount()` — Linux reads `/proc/self/fd`. - **`bufsz()`**: Returns 256KB (AWS EBS IOP alignment). ### `FileSystemException` Extends `std::runtime_error` with `failWith()` and `failWithErrno()` static helpers that log before throwing. ### `TmpDir` / `TmpDirManager` RAII temporary directory management. `TmpDirManager` creates and cleans a root directory; `TmpDir` creates prefixed subdirectories that are deleted on destruction. --- ## 6. XDR Stream I/O (`XDRStream.h`) ### `XDRInputFileStream` Reads XDR-framed objects from a file one at a time. - `readOne(T& out, SHA256* hasher)`: Reads 4-byte big-endian size header, then that many bytes, unmarshals into `out`. Optional hash accumulation. - `readPage(T& out, key, pageSize)`: Reads a page-sized chunk and scans for a specific `LedgerKey`. Used for indexed bucket lookups. - Supports `seek()`, `pos()`, size limits. ### `OutputFileStream` Low-level durable write stream using ASIO buffered writes (POSIX) or `FILE*` (Win32). - `writeBytes()`, `flush()`, `close()` with optional fsync on close. - Cross-platform via `fs::stream_t` / `fs::native_handle_t`. ### `XDROutputFileStream` Extends `OutputFileStream` with XDR framing. - `writeOne(T, hasher, bytesPut)`: Encodes 4-byte size header + XDR payload. Optional hash and byte counting. - `durableWriteOne()`: `writeOne` + flush + fsync for crash safety. --- ## 7. Data Structures ### `RandomEvictionCache<K, V>` (`RandomEvictionCache.h`) Fixed-size cache with random-2-choices eviction (not LRU). More robust under pathological access patterns. - Stores entries in `unordered_map` with stable pointers in a `vector` for random access. - Tracks generation counter for last-access ordering. - `put()`, `get()`, `maybeGet()`, `exists()`, `erase_if()`, `clear()`. - Maintains hit/miss/insert/update/evict counters. ### `BitSet` (`BitSet.h`) Value-semantic C++ wrapper around a C bitset (`cbitset.h`). Uses inline storage for small sets (≤64 bits) to avoid heap allocation. - Full set operations: union (`|`), intersection (`&`), difference (`-`), symmetric difference. - Counting variants: `unionCount()`, `intersectionCount()`, etc. - Iteration: `nextSet(i)` for scanning set bits. - Hash function support via `BitSet::HashFunction`. ### `UnorderedMap<K, V>` / `UnorderedSet<K>` (`UnorderedMap.h`, `UnorderedSet.h`) Type aliases for `std::unordered_map` / `std::unordered_set` using `RandHasher` to prevent hash-flooding attacks. ### `BinaryFuseFilter<T>` (`BinaryFuseFilter.h`, `BinaryFuseFilter.cpp`) Probabilistic membership-test filter (like Bloom filter but more space-efficient). Wraps `binary_fuse_t` library. Supports 8/16/32-bit widths with corresponding false-positive rates (1/256, 1/65536, 1/4B). Serializable via cereal to `SerializedBinaryFuseFilter` XDR type. Used for efficient key-membership checks in bucket indexes. ### `TarjanSCCCalculator` (`TarjanSCCCalculator.h`) Tarjan's algorithm for computing strongly connected components of a directed graph. Uses `BitSet` for SCC representation. Used in quorum analysis (SCP). --- ## 8. Threading Utilities ### `ThreadAnnotations.h` Comprehensive Clang thread-safety annotation macros (`GUARDED_BY`, `REQUIRES`, `ACQUIRE`, `RELEASE`, `TRY_ACQUIRE`, etc.). Provides annotated mutex wrappers: - **`Mutex`**: Wraps `std::mutex` with annotations. - **`SharedMutex`**: Wraps `std::shared_mutex` with `Lock()`/`LockShared()`. - **`RecursiveMutex`**: Wraps `std::recursive_mutex`. - **`MutexLocker` / `RecursiveMutexLocker` / `SharedLockExclusive` / `SharedLockShared`**: RAII lock guards with annotations. ### `GlobalChecks.h` / `GlobalChecks.cpp` - **`threadIsMain()`**: Checks if current thread is the main thread (captured at static init). - **`releaseAssert(e)`**: Assert that is NOT compiled out by `NDEBUG`. Prints backtrace and aborts. - **`releaseAssertOrThrow(e)`**: Like `releaseAssert` but throws `runtime_error` instead of aborting. - **`dbgAssert(e)`**: Debug-only assert. - **`LockGuard` / `RecursiveLockGuard`**: Aliases that adapt between Tracy-instrumented and plain lock guards. ### `Thread.h` / `Thread.cpp` - `runCurrentThreadWithLowPriority()` / `runCurrentThreadWithMediumPriority()`: OS-level thread priority adjustment. - `futureIsReady(std::future|std::shared_future)`: Non-blocking readiness check. ### `JitterInjection.h` / `JitterInjection.cpp` Testing framework for probabilistic thread-delay injection (enabled via `BUILD_THREAD_JITTER`). - `JitterInjector::injectDelay(probability, minUsec, maxUsec)`: Probabilistically sleeps to expose race conditions. - `JITTER_INJECT_DELAY()`, `JITTER_YIELD()` macros: Compile to no-ops in production. - Reproducible via seeded RNG tied to test seed. ### `NonCopyable.h` Base structs: `NonCopyable`, `NonMovable`, `NonMovableOrCopyable` — disable copy/move semantics via deleted special members. --- ## 9. Metrics & Performance Monitoring ### `MetricsRegistry` (`MetricsRegistry.h`, `MetricsRegistry.cpp`)