slv-rpc-gateway

name	slv-rpc-gateway
description	A small Rust JSON-RPC + WebSocket gateway that fronts yellowstone-faithful (of1) and routes `jet*` analytics methods to ClickHouse (jetstreamer data). Standard Solana RPC methods pass straight through; new analytic methods are answered from precomputed jetstreamer aggregates.

SLV RPC Gateway Skill

A unified JSON-RPC + WebSocket entry point that combines the per-call latency of of1 (yellowstone-faithful) with the bulk-analytics speed of jetstreamer's ClickHouse tables. Standard methods go to the underlying RPC; new methods are answered from a custom indexer.

client ── POST / ─► rpc-gateway ──► standard RPC (getTransaction, …)  ── of1
                              └──► jet* (analytics)                  ── ClickHouse
                              └──► getTransactionsForAddress         ── ClickHouse + of1
                              └──► getTransfersByAddress             ── ClickHouse

client ── /ws  ──► rpc-gateway ──► transactionSubscribe              ── Yellowstone gRPC
                              └──► slotSubscribe (multi-source)      ── multiplexed pubsub
                              └──► standard pubsub                   ── upstream pubsub WS

Why

Layer	Component	Responsibility
Per-call (read 1 tx)	yellowstone-faithful (of1)	getTransaction, getBlock, getBlockTime, getSignaturesForAddress
Cross-epoch analytics	jetstreamer + ClickHouse	aggregations over millions of txs
Per-address index	slv_gtfa_plugin + slv_transfers_plugin	per-address tx / transfer lookups
Unified surface	rpc-gateway (this skill)	dispatch by method name, return JSON-RPC

Without the gateway, clients have to know two endpoints and two protocols (JSON-RPC vs ClickHouse SQL). With it, everything is one JSON-RPC endpoint.

HTTP JSON-RPC methods

Method	Backend	Notes
Standard Solana RPC	of1	Forwarded transparently
jetTopPrograms	CH	Top-N programs by invocation count over a time window
jetProgramStats	CH	Time series for a single program (calls, errors, CU)
jetSlotStats	CH	Per-slot tx counts (single slot or range up to 100k)
jetTpsTimeseries	CH	Total + non-vote TPS in time buckets
jetEpochSummary	CH	Aggregate stats for one epoch
getTransactionsForAddress	CH (gtfa_tx_mentions) + of1 fan-out for full mode	Per-address transaction index, signatures or full mode
getTransfersByAddress	CH (token_transfers + by_to MV)	Per-address SPL token transfer index, direction-aware

All jet* methods return JSON arrays of records ({column: value, …}) following the JSON-RPC result envelope.

WebSocket methods (/ws)

The gateway also serves an enhanced WebSocket at /ws (and at / when the client sends an Upgrade: websocket header). Clients connect once and get both standard Solana pubsub and the enhanced transactionSubscribe method through the same socket.

Method	Backend	Notes
transactionSubscribe / transactionUnsubscribe	upstream Yellowstone gRPC	Filter-based transaction subscription. Filters: vote, failed, signature, accountInclude, accountExclude, accountRequired. Options: commitment, encoding, transactionDetails, showRewards, maxSupportedTransactionVersion.
slotSubscribe / slotUnsubscribe	multi-source cascade (see below)	Lowest-latency path picked based on env vars
accountSubscribe / Unsubscribe	upstream pubsub WS	Forwarded
logsSubscribe / Unsubscribe	upstream pubsub WS	Forwarded
programSubscribe / Unsubscribe	upstream pubsub WS	Forwarded
signatureSubscribe / Unsubscribe	upstream pubsub WS	Forwarded
slotsUpdatesSubscribe / Unsubscribe	upstream pubsub WS	Forwarded
blockSubscribe / Unsubscribe	upstream pubsub WS	Forwarded
voteSubscribe / Unsubscribe	upstream pubsub WS	Forwarded
rootSubscribe / Unsubscribe	upstream pubsub WS	Forwarded

Subscription IDs ≥ 1_000_000_000 are local (gateway-served); below that they belong to the upstream pubsub. Unsubscribe routes to the right side based on this range.

slotSubscribe priority cascade

Picked by env-var presence, highest first:

SLOT_FIRST_SHRED_MULTIPLEX_URLS  → N upstreams × slotsUpdatesSubscribe × firstShredReceived filter
  (+ optional SLOT_GRPC_URL       → jito-shredstream-proxy SubscribeEntries, joins the same dedup window
                                    and bypasses the validator's TVU processing step)
  (+ optional SLOT_UDP_BIND       → raw shred UDP listener, reads slot from shred header directly
                                    (skips the gRPC proxy's decode round-trip, ~150-450 µs per slot))
  → SLOT_FIRST_SHRED_URL          → single URL  × firstShredReceived filter
  → SLOT_MULTIPLEX_URLS           → N upstreams × standard slotSubscribe (dedup only)
  → SLOT_PUBSUB_URL               → per-client PubsubForward to a slot-only upstream
  → PUBSUB_WS_URL                 → same upstream as every other standard pubsub

The first-shred variants emit slotNotification at first-shred-received time (= before the bank for that slot is frozen), trading consistency for latency. Clients that need bank state after the tick should fetch via getAccountInfo.

Source

slv-plugins/rpc-gateway/ in the slv repo (= Cargo workspace member, sibling of the jetstreamer plugins that emit the backing ClickHouse rows):

slv-plugins/rpc-gateway/
├── Cargo.toml
├── README.md
└── src/
    ├── main.rs                         # axum bind + env load
    ├── lib.rs                          # roadmap doc
    ├── jsonrpc.rs                      # Request/Response types
    ├── clickhouse.rs                   # HTTP client
    ├── of1.rs                          # upstream JSON-RPC client
    ├── dispatch.rs                     # Gateway struct + method routing
    ├── handlers/
    │   ├── jet.rs                      # jetTopPrograms, jetSlotStats, …
    │   ├── gtfa.rs                     # getTransactionsForAddress
    │   └── transfers.rs                # getTransfersByAddress
    └── ws/
        ├── mod.rs                      # WS entry + slot priority cascade
        ├── pubsub_forward.rs           # per-client upstream WS bridge
        ├── slot_source.rs              # 3 slot multiplex variants
        └── yellowstone_bridge.rs       # transactionSubscribe via gRPC

Build: cargo build --release -p slv-rpc-gateway (inside slv-plugins/).

Deployment

template/<VERSION>/ansible/cmn/install_rpc_gateway.yml provisions end-to-end on any node that has of1 + ClickHouse reachable:

1. Installs build deps (clang, libclang-dev, build-essential, …) and rustup (idempotent for the solv user).
2. Clones the slv repo (depth=1) to /opt/slv-build/slv.
3. cargo build --release -p slv-rpc-gateway and installs the binary to /opt/rpc-gateway-rust/slv-rpc-gateway.
4. Optionally builds a persistent SSH tunnel for ClickHouse when the CH host binds loopback only (rpc_gateway_ch_tunnel_via inventory var).
5. Installs rpc-gateway.service systemd unit and enables/starts it.
6. Waits for /health to return 200.

Inventory variables: rpc_gateway_port, rpc_gateway_of1_url, rpc_gateway_ch_url, rpc_gateway_ch_db, rpc_gateway_ch_user, rpc_gateway_ch_pass, rpc_gateway_yellowstone_grpc (default localhost:10000), rpc_gateway_pubsub_ws (default ws://localhost:7111), and the slot-source env vars listed in the cascade table above.

Companion: local shred relay (= jito UDP fast path)

To feed the gateway's SLOT_UDP_BIND listener with shreds from a jito block-engine, a separate jito-shredstream-proxy instance needs to run on the same host:

[ jito block-engine ] ──UDP─► [ shredstream-local :<src_port> ]
                                       └─UDP─► [ gateway :<udp_bind_port> ]

template/<VERSION>/ansible/cmn/install_local_shred_relay.yml provisions this relay end-to-end:

1. Downloads the upstream jito-shredstream-proxy release binary.
2. Copies the operator-supplied keypair (controller → host, 600 perms, never logged).
3. Renders shredstream-local.service with the chosen src bind port and dest (defaults to forwarding 127.0.0.1:20000, which matches the gateway's recommended SLOT_UDP_BIND=0.0.0.0:20000).
4. Enables + starts the systemd unit, waits for the jito heartbeat handshake, fails the play if the unit is not active.

Operator-side prerequisites (NOT done by the play):

• nftables / firewall must accept inbound UDP on the src bind port from jito's known shred-source IPs (currently observed: 64.130.40.0/24 — confirm per region).
• The jito tier on the keypair must allow as many concurrent region heartbeats as local_shred_relay_desired_regions lists.

Required inventory vars: local_shred_relay_block_engine_url, local_shred_relay_desired_regions, local_shred_relay_auth_keypair_src (path on controller).

Topology recommendations

• Co-locate the gateway with each of1 RPC node (cheap, low latency to of1).
• Point CLICKHOUSE_URL at the central jetstreamer node, or at a regional ClickHouse read replica if cross-region latency matters. When CH binds loopback only, use the tunnel mode above.
• Public-facing nodes should additionally:
  • Add a read-only ClickHouse user and bind it via CLICKHOUSE_USER/PASS.
  • Front the gateway with HTTPS termination (Caddy / nginx / pingora) and rate-limit jet* separately from standard methods (jet* are heavier).

Health probes

• GET /health — synchronous, doesn't touch upstreams (cheap; OK to poll often).

Adding a new jet* method

1. Add a handler function in slv-plugins/rpc-gateway/src/handlers/jet.rs.
2. Add the dispatch arm in src/dispatch.rs's Gateway::dispatch.
3. Update the per-phase roadmap table in src/lib.rs.
4. Add a unit test in the same file's #[cfg(test)] module.
5. Smoke test: curl -X POST http://localhost:8889 -d '{"jsonrpc":"2.0",…}'.

The handler should:

• Validate params with the local helpers in handlers/mod.rs (as_int, as_string, as_bool_or, as_date_string, param_obj).
• Use quote_string() from clickhouse.rs for any literal that ends up in SQL.
• Return Result<serde_json::Value, String>; the dispatcher wraps it into a JSON-RPC envelope via ok / err.