subsystem-summary-of-rust

# Rust Subsystem (Non-Soroban) — Technical Summary ## Overview The rust subsystem provides a Rust static library (`rust_stellar_core`) that is linked into the stellar-core C++ binary. It uses the `cxx` crate (v1.0.97) to define a bidirectional FFI bridge between C++ and Rust. The subsystem's primary responsibilities are: 1. **Soroban host function invocation** — dispatching to the correct protocol-versioned soroban host. 2. **Fee computation** — transaction resource fees, rent fees, and rent write fees. 3. **Module caching** — pre-compiled WASM module cache for Soroban contracts. 4. **128-bit integer arithmetic** — exposing Rust's native `i128` to C++. 5. **Base64 encoding/decoding** — used for XDR serialization interop. 6. **Ed25519 signature verification** — using `ed25519-dalek` for faster verification. 7. **Logging bridge** — routing Rust `log` crate output to the C++ spdlog system. 8. **Quorum intersection checking** — using the `stellar-quorum-analyzer` SAT solver. 9. **Utility functions** — rustc version, executable path, backtrace capture, XDR version checks. The crate is built as `crate-type = ["staticlib"]` (edition 2021, rust-version 1.82.0). Optional features include `tracy` (profiling), `next` (pre-release protocol), `testutils` (test-only code), and `unified` (IDE-friendly single cargo build). ## File Layout | File | Role | |------|------| | `Cargo.toml` | Crate metadata, multi-host soroban dependencies, feature flags | | `src/lib.rs` | Crate root; declares modules, re-exports bridge symbols, defines `tracy_span!` macro | | `src/bridge.rs` | `#[cxx::bridge]` module — all FFI type/function declarations | | `src/common.rs` | `RustBuf`/`CxxBuf`/`BridgeError` impls; `get_rustc_version`, `current_exe`, `capture_cxx_backtrace`, `check_xdr_version_identities` | | `src/b64.rs` | `to_base64` / `from_base64` | | `src/ed25519_verify.rs` | `verify_ed25519_signature_dalek` (unsafe raw-pointer FFI) | | `src/i128.rs` | `i128_add`, `i128_sub`, overflow/underflow checks, conversion | | `src/log.rs` | `StellarLogger` implementing `log::Log`, routes to C++ spdlog | | `src/quorum_checker.rs` | `network_enjoys_quorum_intersection` wrapping `stellar-quorum-analyzer` | | `src/soroban_invoke.rs` | `invoke_host_function`, fee computation, transaction parsing dispatchers | | `src/soroban_module_cache.rs` | `SorobanModuleCache` struct; per-protocol caches | | `src/soroban_proto_all.rs` | Protocol-versioned host modules (p21–p26), dispatch table, adaptors | | `src/soroban_proto_any.rs` | Protocol-agnostic host invocation code, mounted inside each pN module | | `CppShims.h` | Thin C++ shim functions (`shim_isLogLevelAtLeast`, `shim_logAtPartitionAndLevel`) | | `RustBridge.h` | cxx-generated C++ header with all bridge types and function declarations | | `RustBridge.cpp` | cxx-generated C++ implementation (extern "C" thunks, Vec/Box specializations) | | `RustVecXdrMarshal.h` | Declares `rust::Vec<uint8_t>` as valid xdrpp byte buffer type | ## The CXX Bridge Mechanism ### How it works The bridge is defined in `src/bridge.rs` inside a `#[cxx::bridge]` attribute macro on `mod rust_bridge`. This module contains three sections: 1. **Shared types** — structs and enums visible to both sides, defined once: - `CxxBuf` (C++→Rust data: wraps `UniquePtr<CxxVector<u8>>`) - `RustBuf` (Rust→C++ data: wraps `Vec<u8>`) - `XDRFileHash`, `InvokeHostFunctionOutput`, `CxxLedgerInfo`, `CxxTransactionResources`, `CxxFeeConfiguration`, `CxxLedgerEntryRentChange`, `CxxRentFeeConfiguration`, `CxxRentWriteFeeConfiguration`, `CxxI128`, `FeePair`, `SorobanVersionInfo` - Enums: `LogLevel` (shared with `stellar::LogLevel`), `BridgeError`, `QuorumCheckerStatus` - `QuorumSplit`, `QuorumCheckerResource` 2. **`extern "Rust"` block** (`#[namespace = "stellar::rust_bridge"]`) — Rust functions callable from C++: - All functions listed in the "Key Functions" section below. - The opaque type `SorobanModuleCache` with its methods. 3. **`extern "C++"` block** (`#[namespace = "stellar"]`) — C++ functions callable from Rust: - `shim_isLogLevelAtLeast(partition: &CxxString, level: LogLevel) -> Result<bool>` - `shim_logAtPartitionAndLevel(partition: &CxxString, level: LogLevel, msg: &CxxString) -> Result<()>` ### Data passing convention - **C++ → Rust**: Data is passed as `CxxBuf` containing `UniquePtr<CxxVector<u8>>` (a C++-allocated `std::vector<uint8_t>`). The Rust side reads from it via `data.as_slice()`. - **Rust → C++**: Data is returned as `RustBuf` containing `Vec<u8>` (Rust-allocated). The C++ side reads from `data` (a `rust::Vec<uint8_t>`). - XDR serialization/deserialization is done with `ReadXdr`/`WriteXdr` using `non_metered_xdr_from_cxx_buf` and `non_metered_xdr_to_rust_buf` helper functions with a depth limit of 1000 and length limit matching the buffer size. - `RustVecXdrMarshal.h` allows xdrpp to directly unmarshal from `rust::Vec<uint8_t>`. ### Generated files `RustBridge.h` and `RustBridge.cpp` are generated by the `cxxbridge` tool. They contain: - Full implementations of `rust::String`, `rust::Slice<T>`, `rust::Box<T>`, `rust::Vec<T>`, `rust::Opaque`, `rust::Error`. - C struct definitions mirroring the shared types. - `static_assert` checks ensuring `LogLevel` enum values match between C++ and Rust. - `extern "C"` function declarations for the mangled bridge symbols. - C++ wrapper functions in `namespace stellar::rust_bridge` that call through extern "C" thunks and translate Rust errors to C++ exceptions (`rust::Error`). - Template specializations for `rust::Vec<RustBuf>`, `rust::Vec<XDRFileHash>`, `rust::Vec<CxxBuf>`, etc. - `rust::Box<SorobanModuleCache>` alloc/dealloc/drop specializations. ### CppShims.h Provides simple inline wrapper functions that cxx.rs can call, bridging to C++ APIs that are too complex for cxx to handle directly (e.g., static member functions): - `shim_isLogLevelAtLeast` → `Logging::isLogLevelAtLeast` - `shim_logAtPartitionAndLevel` → `Logging::logAtPartitionAndLevel` ## Key Data Structures ### `CxxBuf` / `RustBuf` Directional byte-buffer wrappers for passing XDR-serialized data across the FFI boundary. `CxxBuf` owns a `std::unique_ptr<std::vector<uint8_t>>` (C++ allocated). `RustBuf` owns a `Vec<u8>` (Rust allocated). Both implement `AsRef<[u8]>`. ### `CxxI128` Split representation of 128-bit integer: `{ hi: i64, lo: u64 }`. Used because C++ lacks native `i128` on all platforms. Converted to/from Rust `i128` via `int128_helpers::{i128_from_pieces, i128_hi, i128_lo}`. ### `InvokeHostFunctionOutput` Return value of `invoke_host_function`. Contains: - `success: bool`, `is_internal_error: bool` - `diagnostic_events: Vec<RustBuf>` (XDR-encoded `DiagnosticEvent`) - `cpu_insns`, `mem_bytes`, `time_nsecs` (and excluding-VM-instantiation variants) - `result_value: RustBuf`, `contract_events: Vec<RustBuf>`, `modified_ledger_entries: Vec<RustBuf>`, `rent_fee: i64` ### `SorobanModuleCache` An opaque Rust type exposed to C++ via `rust::Box<SorobanModuleCache>`. Holds per-protocol `ProtocolSpecificModuleCache` instances (p23, p24, p25, and optionally p26 with `next` feature). Each `ProtocolSpecificModuleCache` contains a `ModuleCache` (from soroban-env-host, threadsafe via internal locking) and an `AtomicU64` tracking memory consumption. Methods: - `compile(&mut self, ledger_protocol: u32, wasm: &[u8])` — parse and cache a WASM module for the given protocol. - `shallow_clone(&self) -> Box<SorobanModuleCache>` — clone shared ownership handles for multithreaded compilation. - `evict_contract_code(&mut self, key: &[u8])` — remove a module from all protocol caches by 32-byte hash. - `clear(&mut self)` — clear all protocol caches. - `contains_module(&self, protocol: u32, key: &[u8]) -> bool` - `get_mem_bytes_consumed(&self, protocol: u32) -> u64` ### `HostModule` A dispatch table struct (not crossing FFI) containing function pointers for a specific protocol version's soroban host. Fields include `max_proto`, `invoke_host_function`, `compute_transaction_resource_fee`, `compute_rent_fee`, `compute_rent_write_fee_per_1kb`, `contract_code_memory_size_for_rent`, `can_parse_transaction`, and `get_soroban_version_info`. The static array `HOST_MODULES` holds one entry per protocol version (p21–p25/p26), populated via the `proto_versioned_functions_for_module!` macro. ### `ProtocolSpecificModuleCache` Per-protocol cache wrapper (defined in `soroban_proto_any.rs`). Wraps a `ModuleCache` from the protocol's soroban-env-host and a `CoreCompilationContext` (unlimited budget for compilation). Supports `compile`, `evict`, `clear`, `contains_module`, `get_mem_bytes_consumed`, and `shallow_clone`. ### `CoreCompilationContext` Implements `CompilationContext` (= `ErrorHandler + AsBudget`) with an unlimited budget, used for compiling WASM modules outside of transaction execution. ## Key Functions (Exported Rust → C++) ### Soroban Host Invocation - `invoke_host_function(config_max_protocol: u32, enable_diagnostics: bool, instruction_limit: u32, hf_buf: &CxxBuf, resources: CxxBuf, restored_rw_entry_indices: &Vec<u32>, source_account: &CxxBuf, auth_entries: &Vec<CxxBuf>, ledger_info: CxxLedgerInfo, ledger_entries: &Vec<CxxBuf>, ttl_entries: &Vec<CxxBuf>, base_prng_seed: &CxxBuf, rent_fee_configuration: CxxRentFeeConfiguration, module_cache: &SorobanModuleCache) -> Result<InvokeHostFunctionOutput>` — Dispatches to the correct protocol-versioned host via `get_host_module_for_protocol`. Wraps the call in `panic::catch_unwind`. ### Fee Computation - `compute_transaction_resource_fee(config_max_protocol: u32, protocol_version: u32, tx_resources: CxxTransactionResources, fee_config: CxxFeeConfiguration) -> Result<FeePair>` — Returns `(non_refundable_fee, refundable_fee)`. - `compute_rent_fee(config_max_protocol: u32, protocol_version: u32, changed_entries: &Vec<CxxLedgerEntryRentChange>, fee_config: CxxRentFeeConfiguration, current_ledger_seq: u32) -> Result<i64>` - `compute_rent_write_fee_per_1kb(config_max_protocol: u32, protocol_version: u32, bucket_list_size: i64, fee_config: CxxRentWriteFeeConfiguration) -> Result<i64>` - `contract_code_memory_size_for_rent(config_max_protocol: u32, protocol_version: u32, contract_code_entry: &CxxBuf, cpu_cost_params: &CxxBuf, mem_cost_params: &CxxBuf) -> Result<u32>` — Only valid for protocol ≥ 23. ### Transaction Parsing - `can_parse_transaction(config_max_protocol: u32, protocol_version: u32, xdr: &CxxBuf, depth_limit: u32) -> Result<bool>` — Checks if a `TransactionEnvelope` XDR can be deserialized in the given protocol. ### 128-bit Integer Arithmetic - `i128_add(lhs: &CxxI128, rhs: &CxxI128) -> Result<CxxI128>` - `i128_sub(lhs: &CxxI128, rhs: &CxxI128) -> Result<CxxI128>` - `i128_add_will_overflow(lhs: &CxxI128, rhs: &CxxI128) -> Result<bool>` - `i128_sub_will_underflow(lhs: &CxxI128, rhs: &CxxI128) -> Result<bool>` - `i128_from_i64(val: i64) -> Result<CxxI128>` - `i128_is_negative(val: &CxxI128) -> Result<bool>` - `i128_i64_eq(lhs: &CxxI128, rhs: i64) -> Result<bool>` ### Ed25519 Verification - `verify_ed25519_signature_dalek(public_key_ptr: *const u8, signature_ptr: *const u8, message_ptr: *const u8, message_len: usize) -> bool` — Unsafe raw-pointer interface. Uses `ed25519-dalek`'s `verify_strict` (rejects small-order points, matching libsodium). Never panics; returns false for invalid input. ### Base64 - `to_base64(b: &CxxVector<u8>, s: Pin<&mut CxxString>)` — Encode bytes to base64. - `from_base64(s: &CxxString, b: Pin<&mut CxxVector<u8>>)` — Decode base64 with error-tolerant stripping of invalid characters. ### Logging - `init_logging(maxLevel: LogLevel) -> Result<()>` — Initializes the `StellarLogger` as the global Rust logger, routing to C++ spdlog. Uses `AtomicBool` for one-time initialization. Log partitions (e.g., `TX`, `Ledger`, `SCP`) are defined in `log::partition` and must match `util/LogPartitions.def` on the C++ side. ### Quorum Checker - `network_enjoys_quorum_intersection(nodes: &Vec<CxxBuf>, quorum_set: &Vec<CxxBuf>, potential_split: &mut QuorumSplit, resource_limit: &QuorumCheckerResource, resource_usage: &mut QuorumCheckerResource) -> Result<QuorumCheckerStatus>` — Returns `UNSAT` (quorum intersection holds), `SAT` (split found, populates `potential_split`), or `UNKNOWN`. Time limit enforced internally; memory limit is a hard abort via global allocator. ### Module Cache