| name | http-server |
| description | Run or point an agent at a sparq SPARQL 1.1 Protocol HTTP endpoint (sparq-server) — /sparql query+update over GET/POST (plus the query-only HTTP QUERY method, w3c/sparql-protocol#40, for Oxigraph interop), content negotiation (SELECT/ASK JSON/XML/CSV/TSV; CONSTRUCT/DESCRIBE + Graph Store N-Triples/prefix-Turtle/RDF-XML/JSON-LD — JSON-LD via the default-on jsonld feature), Graph Store read AND write (PUT/POST/DELETE/PATCH on graph resources, RDF/XML + default-on JSON-LD bodies accepted, atomic SPARQL-Update + opt-in Solid N3-Patch on PATCH), EXPLAIN, Prometheus /metrics, WebSocket + SSE subscriptions, opt-in grouped facet counts and prefix completion, and generation-pinned snapshot reads (a Sparq-Generation header + ?generation=N pin in the DEFAULT build, bounded to the ring's concurrency-retention window; opt-in time-travel widens it). Use when starting the server, querying/updating a running endpoint, choosing Accept/Content-Type, or embedding the axum router. |
sparq-http-server
sparq-server is a W3C-conformant HTTP server (axum/tokio) that exposes the sparq
query engine over a sparq_core::Graph — in-memory by default, or durable on disk
with --persist DIR (updates WAL-fsync'd, survive a restart with no rebuild; see the
"Durability" gotcha). It implements the SPARQL 1.1 Protocol (query + update at
/sparql) and the Graph Store HTTP Protocol (read + write), with Accept-driven
content negotiation, hardening guards, Prometheus /metrics, WebSocket + SSE subscriptions,
and opt-in grouped facet counts, prefix completion, time-travel, and GeoSPARQL.
Build with the default-off response-compression feature to negotiate gzip or zstd for outbound
result bodies from Accept-Encoding. Compression is transport-transparent and streaming; SSE
subscription responses and bodies that already have a Content-Encoding are left uncompressed:
cargo run -p sparq-server --features response-compression
curl --compressed -G http://127.0.0.1:3030/sparql \
--data-urlencode 'query=SELECT * WHERE { ?s ?p ?o }'
Quickstart
Run the binary (server stack is the default-on server feature):
cargo run -p sparq-server -- --format turtle data.ttl
cargo run -p sparq-server
cargo run -p sparq-server -- --addr 0.0.0.0:8080 --allow-remote --format ntriples data.nt
Opt-in HTTP/2 switches the TCP listener from its default HTTP/1.1-only builder to the
hyper-util h1+h2 auto builder. With no certificate flags it accepts HTTP/1.1 and cleartext h2c;
supplying both PEM paths enables TLS 1.3 with ALPN h2,http/1.1:
cargo run -p sparq-server --features http2 -- data.ttl \
--tls-cert ./cert.pem --tls-key ./key.pem
The PEM flags are a pair: supplying only one fails startup. Omitting both preserves plain HTTP.
The HTTP/1 slow-loris header deadline, body idle deadline, peer ConnectInfo, graceful drain, and
WebSocket upgrade remain on the same bespoke serve path; HTTP/2 uses the same router and request
middleware. [GPT-5.6] sq-oprna.6
Opt-in HTTP/3 adds an encrypted QUIC/UDP listener beside the unchanged plain-HTTP TCP
listener. The two listeners dispatch through the same router; the UDP address defaults to
--addr and can be overridden independently:
cargo run -p sparq-server --features http3 -- data.ttl \
--http3 --http3-addr 127.0.0.1:3443 \
--tls-cert ./cert.pem --tls-key ./key.pem
Both PEM paths are required when --http3 is set, and malformed or mismatched material
fails startup. QUIC is mandatorily encrypted. With http3 alone the TCP listener remains plain
HTTP for proxy/backward compatibility; with --features http2,http3 the same PEM pair also
secures TCP and negotiates h2/h1 through ALPN. The pre-1.0 h3 stack is contained behind the
default-off feature. /subscriptions WebSockets remain HTTP/1.1-only; clients must fall back because
HTTP/3 extended CONNECT is not implemented. Once the UDP endpoint has bound, every plain-HTTP TCP
response advertises its live port as Alt-Svc: h3=":<port>"; ma=86400; no header is emitted when
--http3 is absent or startup cannot bind the QUIC listener. [GPT-5.6] sq-oprna.4
Security: optional Bearer-token write gate; loopback by default. With no token
configured, every endpoint is unauthenticated (the back-compat default). Set
--auth-token <TOKEN> (env SPARQ_AUTH_TOKEN) to require Authorization: Bearer <TOKEN>
on every write (a SPARQL Update on /sparql — application/sparql-update, or a
query/update body that parses as an update — and the GSP PUT/POST/DELETE
methods); otherwise 401 with WWW-Authenticate: Bearer (constant-time compared; mirrors
QLever's -a <token>). Add --auth-token-read (env SPARQ_AUTH_TOKEN_READ=1) to ALSO
gate reads. The subscription transports — the /subscriptions WebSocket and the
/subscriptions/sse SSE stream (both a read surface) — are gated by --auth-token-read
too (bead sq-cxk5, closing the prior read-auth bypass): the SSE GET takes the
Authorization: Bearer header; the WS upgrade accepts that header OR (for browsers, which
cannot set headers on a WS handshake) a Sec-WebSocket-Protocol: bearer.<token> subprotocol.
The Prometheus GET /metrics endpoint is gated by --auth-token-read as well (bead
sq-9jrx): its gauges leak the live triple count and active-subscription count, so it is a
read; /health stays ungated for liveness probes.
With no read gate, both are open (back-compatible). The server binds loopback by default and refuses a non-loopback
bind (e.g. 0.0.0.0) unless you set --allow-remote (env SPARQ_ALLOW_REMOTE=1) OR the
whole surface is authenticated (--auth-token AND --auth-token-read) — a write-token
alone still leaves reads open. Deliver the token over TLS (terminate it at a proxy). For
real per-user authz, front it with a reverse proxy / API gateway (or sparq-solid). SPARQL
SERVICE federation is OFF
in the default build (the service cargo feature is off); a SERVICE clause then
errors at execution. Build with --features service to enable it — and even then the
server is default-DENY-all SERVICE: a SERVICE <iri> reaches nothing unless its
host is on the egress allowlist (--service-allow / --service-allow-file /
SPARQ_SERVICE_ALLOW; bead sq-4w18). This is an SSRF guard: a SERVICE clause turns
attacker-controlled query text into an outbound request from the server host (worst case
the 169.254.169.254 cloud-metadata IP). The allowlist is enforced before any socket is
opened, on the resolved IP (DNS-rebinding-safe). See "SERVICE federation (egress
allowlist)" below.
Security response headers (always on, ASVS V14.4 / ASVS-G1; beads sq-cmvh, sq-2bhm).
Every response — success, streamed, error and auth-gated (401) alike — carries a hardening
header set, stamped by a map_response layer in harden():
X-Content-Type-Options: nosniff,
Content-Security-Policy: default-src 'none'; frame-ancestors 'none',
X-Frame-Options: DENY, and Referrer-Policy: no-referrer. These suit a SPARQL data API
(no HTML is rendered): the tightest CSP says the body loads/runs nothing, and frame-ancestors
X-Frame-Options: DENY say it is never meant to be framed. Each header is only added when
absent, so a custom handler can override one. Deliberately omitted: Strict-Transport- Security (the origin serves plain HTTP — HSTS belongs on the fronting TLS proxy);
X-XSS-Protection (deprecated, superseded by CSP); Cross-Origin-* / Permissions-Policy
(browser-app document policies, meaningless for a data API). CORS is OFF by default (no
Access-Control-* header — a cross-origin browser read is blocked, the safe posture) but is an
opt-in first-party origin allowlist (sq-o7o0; see "CORS" below). No blanket
Cache-Control: no-store is forced: results are uncached by
default (no ETag/Cache-Control: public is ever set), so there is nothing to tighten, and a
blanket value would wrongly override /health / /metrics — but the sensitive auth-refusal
(401 from unauthorized()) does carry Cache-Control: no-store so a shared cache never
retains it (sq-2bhm).
Error bodies carry a generic class, never internals (ASVS V7 / ASVS-G3; beads sq-cz89,
sq-j9zs, [OPUS-4.8] sq-kfel). Every error is the structured {"error":"<msg>"} envelope
where <msg> is a STABLE generic category (malformed-query / auth / not-found / server-error) —
never the caller's input, a loaded-RDF fragment, a server filesystem path, a secret, or a
Debug of an internal type. The full detail goes to the server log under
target: "sparq_server" (gated behind --verbose / RUST_LOG), not the response body. All
sensitive error paths funnel through one sanitized_error helper; regression-guarded by
tests/hardening.rs (no_echo_* + FORBIDDEN_INTERNALS) and tests/tpf.rs.
Point a client at it (the endpoint is /sparql):
curl -G http://127.0.0.1:3030/sparql --data-urlencode 'query=SELECT * WHERE { ?s ?p ?o } LIMIT 5'
curl http://127.0.0.1:3030/sparql -H 'Content-Type: application/sparql-query' \
--data 'ASK { ?s ?p ?o }'
curl -X QUERY http://127.0.0.1:3030/sparql -H 'Content-Type: application/sparql-query' \
-H 'Accept: application/sparql-results+json' --data 'SELECT * WHERE { ?s ?p ?o } LIMIT 5'
curl http://127.0.0.1:3030/sparql -H 'Accept: text/csv' \
--data-urlencode 'query=SELECT * WHERE { ?s ?p ?o }'
curl -i http://127.0.0.1:3030/sparql -H 'Content-Type: application/sparql-update' \
--data 'INSERT DATA { <http://ex/a> <http://ex/p> <http://ex/b> }'
Embed the router in your own tokio app (library use):
use sparq_core::Graph;
use sparq_server::{router, AppState};
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let graph = Graph::load_str(include_str!("data.ttl"), "turtle")?;
let app = router(AppState::new(graph));
let listener = tokio::net::TcpListener::bind("127.0.0.1:3030").await?;
axum::serve(listener, app).await?;
Ok(())
}
Key APIs
Library surface re-exported from sparq_server (behind the default server feature):
fn router(state: AppState) -> axum::Router — builds the hardened endpoint router
(/sparql, /sparql/graph, /graphs/*path, /subscriptions, /subscriptions/sse,
/health, /metrics, plus feature-gated routes such as /facets and /complete). [OPUS-4.8] sq-bxog:
/subscriptions/sse is the SSE transport. [GPT-5.6] sq-lsp7k.5.2: /facets needs the
facets build feature and the runtime ServerConfig::facets flag. [GPT-5.6] sq-lsp7k.9.3:
/complete similarly needs the complete feature and ServerConfig::complete.
AppState::new(graph: Graph) -> AppState — default ServerConfig.
AppState::with_config(graph: Graph, config: ServerConfig) -> AppState.
AppState::current(&self) -> PinnedGen — lock-free pin of the current immutable
generation snapshot (PinnedGen = Arc<sparq_serve::Generation<Graph>>; call
.snapshot() -> &Graph, .number() -> u64, .published_at(), .epochs()).
AppState::apply_update(&self, sparql: &str) -> Result<u64, String> — submit a SPARQL
Update through the sequenced writer; blocks until the containing generation is
published; returns that generation number (read-your-writes token). Call off the async
workers (spawn_blocking). With adaptive group-commit (default; adaptive_commit),
a serial interactive client commits in engine-time (µs) — the writer drains only the
already-queued backlog and commits, instead of paying a fixed group-commit window;
concurrent load still batches. FIFO order, per-update atomicity and durability are
unchanged (sq-p7kk5).
AppState::at(&self, number: u64) -> Option<PinnedGen> — pin a retained generation
(time-travel feature only; the HTTP ?generation=N pin does NOT need this — it
resolves against the ring's concurrency-retention window directly, so it works in the
default build via sq-ci2d6).
struct ServerConfig { query_timeout: Option<Duration>, update_where_timeout: Option<Duration>, adaptive_commit: bool, max_body_bytes: usize, max_concurrent: usize, header_read_timeout: Option<Duration>, body_read_timeout: Option<Duration>, max_results: Option<usize>, max_query_rows: Option<usize>, max_query_bytes: Option<usize>, max_decompress_ratio: usize, max_subscriptions: usize, max_subscriptions_per_conn: usize, verbose: bool, redact_logs: bool, allow_remote: bool, auth_token: Option<String>, auth_token_read: bool, service_allow: ServiceAllowlist, /* + time_travel_* / facets / complete under their features, + audit_log under audit-log feature */ } with
ServerConfig::default() and ServerConfig::from_env().
(update_where_timeout = separate, typically-SHORTER writer-side WHERE deadline for SPARQL
UPDATE that bounds writer-queue head-of-line blocking from a slow update — None =
use query_timeout, sq-nulp;
adaptive_commit = adaptive group-commit (default true): a serial interactive client
commits in engine-time (µs) rather than paying a fixed group-commit window; concurrent load
still batches. Pure latency change — FIFO/atomicity/durability unchanged. false = always
windowed. --no-adaptive-commit / SPARQ_ADAPTIVE_COMMIT=0 — sq-p7kk5;
max_query_rows = coarse memory cap; max_query_bytes =
byte-accounted memory cap that also prices row WIDTH + computed-literal size — sq-s5is;
max_decompress_ratio = zip-bomb guard — sq-ebii;
header_read_timeout = slow-loris guard: max time a connection may take to send its
complete request-header block, enforced at hyper's HTTP/1 connection layer by
sparq_server::serve — None disables; default 15s — sq-2gqr;
body_read_timeout = slow-body guard (the complement to header_read_timeout): an idle
deadline between consecutive request-body reads, applied by sparq_server::serve via a
tower_http RequestBodyTimeoutLayer/TimeoutBody (the timer resets after each chunk, so an
honest large upload is not penalised — only a stall is) — closes the slow-body dribble a
complete-header client could otherwise use; None disables; default 30s — sq-lodb.)
auth_token (set: gates the write surface with a Bearer token, constant-time compared;
None: no write auth) and auth_token_read (gate reads too) are honoured by the library
router itself — embedders get the gate for free (sq-zcby).
fn bind_posture(addr: &SocketAddr, allow_remote: bool, auth: AuthPosture) -> BindPosture
— the bind gate the binary applies: Loopback (proceed), RemoteAllowed { warning }
(proceed + log), or RemoteRefused { message } (refuse). AuthPosture::{None, WriteOnly, ReadAndWrite} (via AuthPosture::from_config(&config)) folds the token + read-gate into
the decision: a non-loopback bind is allowed when --allow-remote is set OR the surface is
fully authenticated (ReadAndWrite); a write-token alone (WriteOnly) still requires
--allow-remote because reads stay open. This is a bind-time posture gate; per-request
auth is the auth_token fields above (enforced by router/harden-wrapped handlers).
fn harden(routes: axum::Router, config: &ServerConfig) -> axum::Router — wrap any
router in the production middleware (panic→500, concurrency-limit→429, body-limit→413,
JSON error bodies, optional trace).
async fn serve(listener: tokio::net::TcpListener, app: axum::Router, header_read_timeout: Option<Duration>, body_read_timeout: Option<Duration>, shutdown: impl Future<Output=()>) -> std::io::Result<()> — the accept + graceful-drain loop the binary runs instead of
axum::serve (sq-2gqr / sq-lodb). It is a faithful port of axum's own loop (per-connection
task, watch-channel drain, with_upgrades() so the /subscriptions WebSocket still works) with
two additions: (1) it installs a hyper_util TokioTimer and hyper's HTTP/1
header_read_timeout (the slow-loris HEADER guard); (2) it optionally wraps the request body in a
tower_http RequestBodyTimeoutLayer keyed on body_read_timeout (the slow-BODY guard — applied
via tower::util::option_layer, so None is a true no-op Identity layer). Why: axum::serve
never installs a timer, so hyper's header-read deadline is inert there and a slow-loris client
holds a connection (and a concurrency_limit slot) open indefinitely; and even with that fixed, a
complete-header client can still dribble the BODY to hold the slot — body_read_timeout closes
that complementary hole. Pass None on either to opt back out of that guard. With the
default-off http2 feature this same function uses hyper-util's h1+h2 auto builder (h2c on a
plain listener) and retains the timer on the builder's HTTP/1 configuration.
async fn serve_tls(listener: tokio::net::TcpListener, app: axum::Router, tls_config: Arc<rustls::ServerConfig>, header_read_timeout: Option<Duration>, body_read_timeout: Option<Duration>, shutdown: impl Future<Output=()>) -> std::io::Result<()> (http2 feature
only) — TLS counterpart to serve; the binary supplies a TLS-1.3 config advertising
h2,http/1.1 via ALPN. It shares the auto-builder connection body, middleware, timeout hooks,
peer ConnectInfo, and graceful drain with the cleartext feature-on path. [GPT-5.6] sq-oprna.6
- Re-exports for cache layers/tests:
PinnedGen, GLOBAL_POD: &str
("urn:sparq:pod:global"), and sparq_serve::{Epoch, PodEpochs, PodId}.
- Response-bytes result cache (opt-in,
sparq-serve's result-cache feature,
OFF by default — [OPUS-4.8] sq-jluc). A serving-layer cache from a request
identity to the pre-serialized response body, distinct from sparq-engine's
in-engine algebra-keyed result-cache. Public surface (feature-gated):
sparq_serve::{ResultCache, CacheConfig, ScopeKey, ReadFootprint, LeaseOutcome, CacheStats}. Key = canonical-query × visibility-scope × per-pod epoch-vector
(ResultCache::lease/get): the scope is the identity of the accessible
graph set (ScopeKey::of_graphs(AuthIndex::accessible(session, mode))), never
the WebID — bytes cached for one access scope can never be served to another (an
access-control correctness invariant, not a privacy guarantee). Invalidation is
per-pod epoch bumps (ReadFootprint::Pods) or the global generation
(ReadFootprint::Unbounded); single-flight leases dedup a stampede; a byte-budget
LRU + admission bound keeps it from caching oversize/streaming bodies. Wiring it
into the axum endpoint and the canonical perf-validation are follow-ups (the perf
targets need a canonical host).
- Serializer/negotiation helpers (always compiled, no
server feature): module
sparq_server::negotiate — fn negotiate(accept: Option<&str>) -> Format (lenient, always
yields a format), fn negotiate_or_406(accept: Option<&str>) -> Result<Format, NotAcceptable>
(Oxigraph-parity strict — the query path uses this; a present-but-unsatisfiable Accept is
Err(NotAcceptable) → 406), fn negotiate_graph(accept: Option<&str>) -> GraphFormat and its
strict sibling fn negotiate_graph_or_406(...) -> Result<GraphFormat, NotAcceptable>; module
sparq_server::exec — fn prepare(&str) -> Result<Prepared, PrepareError> and
fn prepare_with_dataset(&str, &DatasetOverride) -> Result<Prepared, PrepareError> (applies the
SPARQL-Protocol default-graph-uri/named-graph-uri override, sq-z33x),
fn apply_update_dataset(&str, &UsingOverride) -> Result<String, UpdateDatasetError> (the
update-side using-* override), enum QueryForm { Select, Ask, Construct, Describe }; module
sparq_server::results.
In-process embedding seam — sparq_serve::embed ([OPUS-4.8] sq-xa15c, #1248)
When the consumer is another Rust process (e.g. solid-server-rs) and wants to
drop the HTTP hop entirely, embed the engine in-process via the sparq_serve::embed
facade instead of running sparq-server and talking to it over HTTP. It is the
documented embedding seam: thin wrappers over the engine entry points plus a
re-export of the runtime-agnostic concurrency wrapper. No axum/tokio, no HTTP.
- Data path (over one
&Graph / &mut Graph): embed::query_json(&Graph, sparql) -> Result<String, _> (SPARQL-JSON), embed::query(&Graph, sparql) -> Result<QueryResult, _>, embed::ask(&Graph, sparql) -> Result<bool, _>,
embed::update_in_place(&mut Graph, sparql) (+ ..._atomic for all-or-nothing on
one graph), embed::apply_delta_nquads(&mut Graph, inserts, deletes) (quad-level,
per-graph; blank nodes by label), and the probes embed::exists(&Graph) -> bool,
embed::named_graph_exists(&Graph, &Term) -> bool, embed::metadata(&Graph) -> Metadata { triples, named_graphs }. query_json_with_budget takes a QueryBudget.
- Concurrency wrapper (re-exported from the crate root):
GenerationRing /
Generation / GraphApplier / Writer (+ RingConfig, WriterConfig, PodId,
TimeTravelConfig) — the SAME fork → update → publish + generation-pinning model
sparq-server wraps behind its endpoint. A reader ring.current() pins an
immutable snapshot; the writer publishes new generations without blocking readers.
- Stability — API tier-1 (proposed-stable): the proposed semver-stable embedding
surface in the API stability & deprecation policy, but
NOT yet frozen — the formal freeze is the maintainer's to ratify on #1248 / #1346
(pre-
1.0 minor releases MAY still change it). Pin to sparq_serve::embed rather
than reaching into sparq-core / sparq-engine directly so the freeze, when
ratified, has one well-defined shape.
Common recipes
1. Query forms and result negotiation. Default result media is SPARQL-JSON. Set
Accept to choose (q-value aware, defaults to JSON for SELECT/ASK, N-Triples for
CONSTRUCT/DESCRIBE). An absent / empty / */* Accept gets
that default; a present Accept that names only unsupported media types and no wildcard is
406 Not Acceptable (Oxigraph parity, w3c/sparql-protocol#40) — sparq no longer silently
falls back to JSON in that case (the EXPLAIN Accept: text/x-sparq-explain short-circuits this
and the Graph-Store-Protocol read path keeps its lenient default):
| Query form | Accept | Content-Type returned |
|---|
| SELECT | application/sparql-results+json (default) / +xml / text/csv / text/tab-separated-values | matching results media |
| ASK | json (default) / xml | application/sparql-results+json / +xml |
| CONSTRUCT / DESCRIBE | application/n-triples (default) / text/turtle / application/rdf+xml / application/ld+json (the jsonld feature — default-on) | matching RDF media; N-Triples, prefix-compacting Turtle, RDF/XML, or flattened JSON-LD |
Per the W3C SPARQL Results TSV format, the TSV serialiser abbreviates an
xsd:integer / xsd:decimal / xsd:double / xsd:boolean literal whose lexical form is
a valid Turtle token to its bare token (no quotes, no ^^datatype) — e.g. 30, 2.2,
1.0E6, true; everything else (incl. integer/decimal subtypes like xsd:negativeInteger
and custom datatypes) stays quoted + typed, and xsd:string keeps the implicit-datatype short
form. The bare token is the literal's own lexical form, so a data-sourced literal
round-trips its original spelling ("1.0E6"^^xsd:double → 1.0E6, not canonicalised);
computed numerics arrive already in the engine's canonical form, so they serialise
canonically. CSV writes each value's bare lexical string (datatype/lang dropped — lossy by
spec); JSON/XML carry the full term (value + datatype) unchanged.
SELECT-JSON streams (TTFB). The application/sparql-results+json SELECT path streams its
body: the engine serialises on a blocking worker feeding a bounded channel, so the results
header + early solutions are written to the socket before the whole result is serialised
(first byte before full materialisation, rather than after it). The streamed body is
byte-identical to the single materialised JSON string (same solution order, same escaping) —
query_json_stream_with_budget shares one emit core with the buffered serialiser. Two response
shapes: a single-chunk (small, ≤ one 64 KiB chunk) result is returned buffered with a
Content-Length; a multi-chunk result streams under chunked transfer-encoding with no
Content-Length (the total is unknown until the last row). Note the streaming scope: a
DISTINCT / join / ORDER-BY SELECT must fully evaluate before any solution exists, so its first
byte still lands after the join materialises (the header is flushed the instant that finishes,
before serialisation) — the earliest-first-byte win is largest on scan-shaped and
below-parallel-threshold results; XML/CSV/TSV stay buffered. Error mid-stream (honest
contract): the status is chosen from the first chunk, so a failure detected before any byte
(parse error, or a row/byte cap / deadline the engine confirms before the header) still returns
the correct 400 / 413 / 503. But once the header has been flushed for a genuinely
multi-chunk result, the HTTP status is committed: a later cap/deadline trip is surfaced by
truncating the chunked body (the stream ends without its terminating zero-length chunk, so
the client sees a broken/incomplete response) — a streamed 200 cannot retroactively become a
413/503.
Single parse per request (HTTP floor). The read path parses each request query with
spargebra exactly ONCE — the server parses to classify the form + apply any protocol dataset
override, then hands the resulting algebra straight to the engine's *_prepared entry points
(query_json_stream_prepared_with_budget for JSON SELECT, ask_prepared_with_budget for ASK),
so the engine does not re-parse the query string. This shaves one full parse off the per-request
floor for ARBITRARY novel queries (no query cache — parsing is still paid, just not twice);
negligible on a trivial ASK{} (sub-µs), and larger on realistic multi-pattern queries.
curl -G http://127.0.0.1:3030/sparql -H 'Accept: application/sparql-results+xml' \
--data-urlencode 'query=SELECT ?s WHERE { ?s ?p ?o }'
curl -G http://127.0.0.1:3030/sparql -H 'Accept: text/turtle' \
--data-urlencode 'query=CONSTRUCT { ?s ?p ?o } WHERE { ?s ?p ?o }'
curl -G http://127.0.0.1:3030/sparql -H 'Accept: application/rdf+xml' \
--data-urlencode 'query=CONSTRUCT { ?s ?p ?o } WHERE { ?s ?p ?o }'
curl -G http://127.0.0.1:3030/sparql -H 'Accept: application/ld+json' \
--data-urlencode 'query=CONSTRUCT { ?s ?p ?o } WHERE { ?s ?p ?o }'
HTTP QUERY method (sq-b3df9, w3c/sparql-protocol#40, epic sq-my8wd) — query-only,
Oxigraph-interop. /sparql accepts the registered HTTP QUERY verb in addition to
GET/POST. It is a new INPUT path that feeds the SAME query execution + Accept negotiation as
POST, so all the result-media rows above apply unchanged. Mirroring Oxigraph
(("/sparql", "POST" | "QUERY"), let is_query = method == "QUERY"):
- Send the query as the raw
application/sparql-query body (preferred), OR as the
query= value of an application/x-www-form-urlencoded body. No other Content-Type is
accepted (else 415); missing Content-Type is 415.
- Dataset graphs (
default-graph-uri / named-graph-uri / union-default-graph) ride
the URL query string, never the body — even when the query is the raw body.
- Query-only: an
update= form field is a 400 and an application/sparql-update
body is a 415 (it is not a valid operation under QUERY). Use POST for an update.
- It enforces the SAME auth / egress gates as GET/POST (a QUERY is a read — gated by
--auth-token-read). It does not emit Cache-Control / Content-Location headers
(Oxigraph does not either). [OPUS-4.8]
curl -X QUERY 'http://127.0.0.1:3030/sparql?default-graph-uri=http://ex/g1' \
-H 'Content-Type: application/sparql-query' \
-H 'Accept: text/csv' \
--data 'SELECT * WHERE { ?s ?p ?o }'
SPARQL 1.1 Protocol conformance lane (sq-jaj38, epic sq-my8wd) — what is ratcheted. The
protocol surface above is now covered by a dedicated W3C SPARQL 1.1 Protocol (HTTP)
conformance suite in sparq-conformance (opt-in http-protocol feature, OFF by default; it
reuses the in-process loopback server — sparq_server::serve on an ephemeral 127.0.0.1:0
port — and drives RAW HTTP at the bound port, NOT the federated SERVICE transport, so it does
not touch the engine's egress allowlist). It ratchets a MEASURED PASS floor over: query via
GET / POST-urlencoded / POST-direct, the QUERY method, update via POST, the
default-graph-uri / named-graph-uri overrides, SELECT/ASK negotiation (SRJ / SRX / CSV /
TSV), a present-but-unsatisfiable Accept → 406 Not Acceptable (Oxigraph parity,
w3c/sparql-protocol#40 — formerly a divergence, now a genuine
PASS that raised the floor 20→21), and the 200 / 400 / 405 (with Allow) / 406 / 415
status codes. Honest boundary: two behaviours remain DOCUMENTED DIVERGENCES (reported
separately, NOT summed into the floor, so they never inflate the conformance number): an
absent / */* Accept defaults to SPARQL-results JSON (a W3C-permitted default
representation — only a present-but-unsatisfiable Accept is a 406), and an ASK with
Accept: text/csv falls back to a JSON boolean (CSV/TSV have no boolean serialisation). Run
it with
cargo test -p sparq-conformance --features http-protocol --test http_protocol_suite; the row
is in the central scoreboard (W3C SPARQL 1.1 Protocol (HTTP)). [OPUS-4.8]
Service Description + Graph Store Protocol conformance lane (sq-1uuxz, epic sq-my8wd) — what
is ratcheted. The federation-descriptor + GSP write surfaces (see "Federation discovery" and
the Graph-Store-Protocol section below) are covered by a dedicated SD + GSP conformance suite
in sparq-conformance (opt-in federation-descriptors feature, OFF by default; it reuses the
in-process loopback server — sparq_server::serve on an ephemeral 127.0.0.1:0 port, stood up
with the server's federation-descriptors runtime flag ON — and drives RAW HTTP at the bound
port, so it does not touch the engine's egress allowlist). It ratchets a MEASURED PASS floor
over (A) the Service Description (GET /sparql with no query): the sd:Service advertises
exactly the result/input formats (SRJ/SRX/CSV/TSV + Turtle/N-Triples/RDF-XML), the query/update
languages, the SPARQL versions (sd:supportedVersion 1.0/1.1/1.2) and sd:BasicFederatedQuery
that the server GENUINELY implements — no over-advertising (each advertised result format is
cross-checked against a real SELECT request, and JSON-LD — not served in this build — must NOT
appear) — and (B) the Graph Store Protocol: a GET/PUT/POST/DELETE round-trip on a named graph
(indirect ?graph=<iri> + direct /graphs/<path>) and the default graph (?default), VERIFYING
store state after every op (PUT→GET-back-equal; PUT replaces; POST merges; DELETE removes;
200/201/204/400/404/405 (with Allow)/415). Honest boundary: a GSP read of an ABSENT
named graph is 200 + empty graph, NOT 404 (GSP-permitted — sparq treats an empty graph as
existing); this is a DOCUMENTED DIVERGENCE, reported separately and NOT summed into the floor. Run
it with cargo test -p sparq-conformance --features federation-descriptors --test sd_gsp_suite;
the row is in the central scoreboard (SPARQL 1.1 Service Description + Graph Store Protocol).
[OPUS-4.8]
JSON-LD content negotiation (jsonld feature — default-on, [OPUS-4.8] sq-oy1f.4). The
server speaks application/ld+json out of the box (the jsonld feature is in the default set —
a maintainer-directed exception to opt-in-by-default). application/ld+json joins the
q-value-aware RDF negotiation in BOTH directions: a CONSTRUCT/DESCRIBE or a Graph-Store-Protocol
read with Accept: application/ld+json is served as the engine's flattened JSON-LD (a
{"@graph": […]} node-object document that toRdf-round-trips to the same graph), and a GSP
PUT/POST with Content-Type: application/ld+json is parsed into the store via the engine's
JSON-LD parser. Toggleable off via --no-default-features --features server: then
application/ld+json is unrecognised — an Accept for it falls back to a supported graph format
(never a 406 — the endpoint always has a representation), and a write body in it is a plain 415
— byte-identical to a JSON-LD-disabled build. What is default-on now: JSON-LD parse + serialise
(flattened) + content-negotiation; full conneg-conformance ratcheting is on the sq-oy1f roadmap.
Default-on algebra rewrite (algebra-rewrite feature — [FABLE-5] sq-7d3dj.30.13). The
server's default set also lights sparq-engine's pre-execution algebra rewrite pass (#1735): a
result-equivalent FILTER(?v = <iri>) IRI-constant folding + FILTER(!bound) anti-join applied
at parse time, so the shipped server executes the same plans the CLI and the canonical benchmarks
measure. IRI constants only (a literal equality is never rewritten — the sq-lr2ii avoidance
contract); zero new dependencies. Drop it with --no-default-features --features server,jsonld
for an explicitly rewrite-dark build; the sparq-engine LIBRARY default remains OFF for lean
library consumers.
2. EXPLAIN a query plan (no execution) or analyze (execute + per-operator trace).
text/plain response. Use explain / explain=plan (or Accept: text/x-sparq-explain)
for the dry run, explain=analyze to run + trace (SELECT/ASK only):
curl -G 'http://127.0.0.1:3030/sparql?explain=true' \
--data-urlencode 'query=SELECT * WHERE { ?a <http://ex/knows> ?b . ?b <http://ex/age> ?age }'
Structured plan tree (explain-json feature, default-on for the server binary —
sq-ixc3.19). Accept: application/x-sparq-explain+json answers the SAME explain /
explain=analyze surface with the typed plan tree (explain_json::PlanNode) as camelCase
JSON — per operator {"operator", "estimated", "actual", "nanos", "qError", "children"}
(the sq-jbqh4 schema contract the GUI plan explorer renders; qError =
max(est/actual, actual/est)). Like the text CT, the JSON Accept alone requests a
plan-only explain. A --no-default-features --features server,… build answers it 406
(the text explain still works) so clients degrade explicitly, never mis-parse text as JSON:
curl -G -H 'Accept: application/x-sparq-explain+json' \
'http://127.0.0.1:3030/sparql?explain=analyze' \
--data-urlencode 'query=SELECT * WHERE { ?a <http://ex/knows> ?b . ?b <http://ex/age> ?age }'
3. Generation-pinned snapshot reads (DEFAULT build — sq-ci2d6). Every /sparql response
carries the Sparq-Generation header (the generation the response was produced against; an
update's 204 carries the generation containing the write — the read-your-writes token).
Capture it and pin a later read with ?generation=N (URL param or url-encoded body — body wins)
to get an immutable snapshot of the store as of that generation. This is available with no
feature enabled: the pin is bounded to the generation ring's concurrency-retention window —
the last K = 4 generations older than current are always kept (sparq_serve::ring::DEFAULT_RETAIN;
RingConfig::retain). That is exactly the window that makes multi-request LIMIT/OFFSET
pagination snapshot-consistent: pin one generation, page it, and the pages union to the
single-shot result at that generation even if writes interleave (with a deterministic order —
add ORDER BY, or rely on the stable order over the immutable snapshot):
G=$(curl -si http://127.0.0.1:3030/sparql -H 'content-type: application/sparql-update' \
--data 'INSERT DATA { <http://ex/a> <http://ex/p> <http://ex/b> }' \
| grep -i sparq-generation | tr -d '\r' | awk '{print $2}')
curl -G http://127.0.0.1:3030/sparql --data-urlencode "generation=$G" \
--data-urlencode 'query=SELECT ?child WHERE { <c> <member> ?child } ORDER BY ?child LIMIT 1000 OFFSET 0'
curl -G http://127.0.0.1:3030/sparql --data-urlencode "generation=$G" \
--data-urlencode 'query=SELECT ?child WHERE { <c> <member> ?child } ORDER BY ?child LIMIT 1000 OFFSET 1000'
Honest boundary (the retention contract). The default window is bounded by the ring's
concurrency retention (K = 4), not a durable history: a pin more than K generations behind
current has aged out → 410 Gone (the snapshot is gone; re-read the current generation and
restart pagination — the server never silently substitutes a different generation). If a
pagination sweep must survive more than K interleaved writes, build with the opt-in time-travel
feature, which EXTENDS retention (--time-travel-generations N, --time-travel-max-age SECS) so
?generation=N reaches far older snapshots — same header/pin mechanics, wider window:
cargo run -p sparq-server --features time-travel -- data.ttl --time-travel-generations 32
Status: aged-out generation → 410 Gone; never-published / unparsable / pinning an
update (an update always applies to current) → 400. All identical in both feature states —
time-travel only changes how far back a pin can reach.
4. WebSocket subscriptions (SEPA-style live SELECT). Connect to
ws://127.0.0.1:3030/subscriptions, send subscribe, get subscribed + an initial
notification (sequence 0 = full result as addedResults), then added/removed bindings
diffs after each committed update that changes the result:
client: {"subscribe": {"query": "SELECT ?s WHERE { ?s <http://ex/age> ?o }", "alias": "ages"}}
server: {"subscribed": {"id": 1, "alias": "ages"}}
server: {"notification": {"id": 1, "sequence": 0, "addedResults": {…full result…},
"removedResults": {"head":{"vars":["s"]},"results":{"bindings":[]}}}}
…POST /sparql update commits…
server: {"notification": {"id": 1, "sequence": 1, "addedResults": {…}, "removedResults": {…}}}
client: {"unsubscribe": {"id": 1}}
server: {"unsubscribed": {"id": 1}}
addedResults/removedResults are each full SPARQL-JSON results objects. Refusals and
failed re-evaluations come back as {"error": {"message": …, "id"?: n}}.
[OPUS-4.8] sq-cxk5: when --auth-token-read is set, the upgrade is gated behind the read
token (401 before upgrade otherwise). A non-browser client sends Authorization: Bearer <TOKEN> on the handshake; a browser (which cannot set WS handshake headers) passes it as
a subprotocol: new WebSocket("ws://host/subscriptions", ["bearer." + token]). The server
takes the substring after the bearer. prefix as the token, validates it (constant-time),
and echoes the subprotocol back per RFC 6455. With no read token configured, the upgrade is
open (back-compatible).
4b. SSE subscriptions (text/event-stream). [OPUS-4.8] sq-bxog: the SAME subscription
engine over Server-Sent Events, for clients that prefer a plain HTTP GET stream to a
WebSocket. GET /subscriptions/sse?query=<SELECT>[&alias=<x>] opens one subscription per
stream (the query is in the query string — SSE is one-way, so there is no subscribe/
unsubscribe frame; close the stream to unsubscribe). The events carry the SAME JSON as
the WS path — only the framing differs (event: / data: / id: lines, blank-line
terminated, : ping keep-alive comments hold idle connections open). The SSE id: mirrors
the per-subscription sequence.
curl -N 'http://127.0.0.1:3030/subscriptions/sse?query=SELECT%20?s%20WHERE%20{%20?s%20%3Chttp://ex/age%3E%20?o%20}&alias=ages'
event: subscribed
data: {"subscribed":{"id":1,"alias":"ages"}}
event: notification
id: 0
data: {"notification":{"id":1,"sequence":0,"alias":"ages","addedResults":{…full result…},"removedResults":{…empty…}}}
…POST /sparql update commits…
event: notification
id: 1
data: {"notification":{"id":1,"sequence":1,"alias":"ages","addedResults":{…},"removedResults":{…}}}
[OPUS-4.8] sq-cxk5: like the WS path, when --auth-token-read is set this GET is gated
behind the read token via the Authorization: Bearer <TOKEN> header (it is a plain GET, so
the header is the only auth channel — no WS subprotocol) — 401 before the stream opens
otherwise. A registration refusal (missing/non-SELECT/malformed query → 400; capacity/budget →
503) is returned as a normal {"error":"…"} JSON HTTP response BEFORE the stream opens —
SSE cannot set a status once the stream is flowing. A later re-evaluation failure ends the
stream with a final event: error frame. Both transports share one registry + change
source, so the per-conn / server-wide subscription caps and the sparq_active_subscriptions
gauge count SSE streams and WS subscriptions together.
5. Graph Store read + write + operational endpoints.
curl http://127.0.0.1:3030/sparql/graph?default
curl 'http://127.0.0.1:3030/sparql/graph?graph=http://ex/g'
curl http://127.0.0.1:3030/graphs/whatever
curl -H 'Accept: application/rdf+xml' http://127.0.0.1:3030/sparql/graph?default
curl -H 'Accept: application/ld+json' http://127.0.0.1:3030/sparql/graph?default
curl -X PUT 'http://127.0.0.1:3030/sparql/graph?graph=http://ex/g' \
-H 'content-type: text/turtle' --data '<http://ex/s> <http://ex/p> <http://ex/o> .'
curl -X POST 'http://127.0.0.1:3030/sparql/graph?graph=http://ex/g' \
-H 'content-type: application/n-triples' --data '<http://ex/s2> <http://ex/p> <http://ex/o2> .'
curl -X DELETE 'http://127.0.0.1:3030/sparql/graph?graph=http://ex/g'
curl -X PATCH 'http://127.0.0.1:3030/sparql/graph?graph=http://ex/g' \
-H 'content-type: application/sparql-update' \
--data 'DELETE DATA { GRAPH <http://ex/g> { <http://ex/s> <http://ex/p> <http://ex/o> } } ;
INSERT DATA { GRAPH <http://ex/g> { <http://ex/s> <http://ex/p> <http://ex/o2> } }'
curl -X PATCH 'http://127.0.0.1:3030/sparql/graph?default' \
-H 'content-type: text/n3' --data '@prefix solid: <http://www.w3.org/ns/solid/terms#>.
@prefix ex: <http://ex/>.
_:p a solid:InsertDeletePatch;
solid:where { ?s ex:name "Alice" . };
solid:deletes { ?s ex:age 30 . };
solid:inserts { ?s ex:age 31 . }.'
curl http://127.0.0.1:3030/health
curl http://127.0.0.1:3030/metrics
curl -X POST -H 'Authorization: Bearer <TOKEN>' http://127.0.0.1:3030/admin/compact
curl -X POST -H 'Authorization: Bearer <TOKEN>' http://127.0.0.1:3030/admin/backup -o snapshot.spqb
curl -X POST -H 'Authorization: Bearer <TOKEN>' --data-binary @snapshot.spqb http://127.0.0.1:3030/admin/restore
curl -X POST -H 'Authorization: Bearer <TOKEN>' --data-binary @snapshot.spqb 'http://127.0.0.1:3030/admin/restore?persist=true'
curl -X POST -H 'Authorization: Bearer <TOKEN>' 'http://127.0.0.1:3030/admin/backup/delta?from=0' -o d0.spqd
sparq-server --restore snapshot.spqb --restore-delta d0.spqd --restore-delta d1.spqd
/metrics is hand-rolled Prometheus text exposition (no metrics dependency); the
middleware wraps the whole hardening stack, so shed (429), body-limit (413) and panic
(500) responses are counted with the status the client saw:
| Metric | Type | What |
|---|
sparq_http_requests_total{endpoint,status} | counter | requests by endpoint + response status |
sparq_query_duration_seconds | histogram | wall time of /sparql (query + update); buckets 1 ms … 10 s |
sparq_active_subscriptions | gauge | active WebSocket subscriptions (scrape time) |
sparq_graph_triples | gauge | triples in the published graph (scrape time) |
sparq_updates_total | counter | successfully applied SPARQL updates |
POST /admin/compact — WAL compaction / vacuum (erasure-completeness, sq-x32t). A logical
DELETE / DROP GRAPH retracts data from the live view but leaves the superseded bytes in earlier
--persist WAL segments (and the dictionary) until a compaction folds the live state into a fresh
base. This admin op does that on demand: it physically rewrites the on-disk store to only the
current live triples, with a re-interned (purged) dictionary, then atomically swaps the
directory (rollback-safe two-rename + WAL truncate; an interrupted swap is healed on the next
open). So a deleted triple's value — including an orphaned literal value (e.g. personal data) —
is gone from disk, not just hidden. It runs on the single writer thread strictly between
batches (no race with a concurrent write), preserves the live triple set exactly (no
generation published; reads keep flowing throughout). POST-only, gated by the write token
(--auth-token), like an UPDATE. Responses: 200 ok; 409 if the server is in-memory (no
--persist — there is no on-disk history to purge, so a no-op success would mislead); 503 on a
transient durable-write error (retryable, writer stays alive). Offline equivalent: stop the
server and run sparq-cli compact <persist-dir> (see the cli skill). Honest scope: this
scrubs the engine's own on-disk segments + dictionary; it cannot reach bytes already copied off-box
(filesystem snapshots, COW history, external backups) — those are the operator's responsibility
(see compliance/privacy/retention-erasure-runbook.md §7a/§7b).
POST /admin/backup + POST /admin/restore — online consistent snapshot backup/restore
(feature backup, default OFF; sq-o5bi). An ONLINE point-in-time backup of the live serving
store, distinct from the offline sparq-cli save (stop-the-world, index rebuild) and from the
--persist per-graph WAL. /admin/backup pins the CURRENT generation lock-free and serialises
it off the immutable Arc — so it runs while serving: readers never block the writer and the
writer never blocks readers throughout. The response is one self-describing artifact (Stardog
backup-ID model, "Option A"): a small textual header recording the generation number (= the single
writer's seq), the per-pod epoch vectors, the triple count and a body integrity digest, then the
full dataset (default + named graphs) as N-Quads. /admin/restore imports such an artifact and
atomically installs a freshly-built ring+writer rehydrated from it — readers in flight keep
serving from the old store until they release their pin; every read/update after the swap sees the
restored store. Fail-closed: a corrupt / truncated / version-mismatched / non-artifact body is
rejected (400) and the live store is left untouched (the new core is built fully before the
swap). Both routes are POST-only and gated by the write token (the admin gate). Responses:
backup 200 (application/octet-stream); restore 200 on success, 400 on a rejected artifact.
Single-flight (sq-fy8ci): a restore posted while another is still in flight is rejected
409 (a restore is already in progress) instead of being silently serialized behind it by
the writer thread (last-writer-wins) — retry after the first completes. Embedders driving
AppState::restore_from directly can claim the same permit via AppState::try_begin_restore()
(an RAII RestoreGuard; dropping it — even on a panic — releases the permit).
Restore into a live durable store (sq-ft7u). On a --persist server, a plain
/admin/restore (no opt-in) is 409 — an in-memory-only swap on a durable server would be
silently lost on the next restart (a footgun). Add ?persist=true to write the restore
THROUGH to the durable directory, so it survives a restart (the on-disk base becomes the
restored image). The durable swap runs on the single writer thread (sequenced with updates — no
lock, no race with a concurrent durable commit) and is crash-safe: the imported image is written
to a sibling and swapped in with a two-rename directory swap (parent dir fsync'd between renames)
that an interrupted crash heals deterministically (Graph::restore_into_durable reuses the exact WAL
compaction swap + recover_compaction healer). Fail-closed throughout: the artifact is imported
- validated before any on-disk change, so a corrupt artifact is
400 with the durable store
untouched and no swap-sibling leftovers; a swap I/O error leaves the OLD store intact and the
writer alive. ?persist=true on an in-memory server is 409 (no durable dir). Restore-on-start:
--restore <FILE> / SPARQ_RESTORE seeds the in-memory store before binding (bootstrap for
horizontal-scaling stage-2 + PITR base); with --persist DIR it is refused unless you add
--restore-persist / SPARQ_RESTORE_PERSIST=1, which writes the artifact through to DIR
crash-safely on start. Honest scope: at-rest encryption of the artifact is out of scope — the
integrity digest detects accidental corruption, not tampering, and is not a confidentiality or
authenticity guarantee; protect the artifact at the storage tier.
POST /admin/backup/delta?from=N + --restore-delta — incremental change-stream / PITR
(same backup feature; sq-bu1a). The Option-A backup above is the BASE half of a
point-in-time-recovery story; this is the incremental companion. /admin/backup/delta streams
a single self-describing delta artifact (distinct magic SPARQ-BACKUP-DELTA) keyed off the
generation/writer-seq: a header recording the from-generation N → the current to-generation
plus the per-pod epoch vector at to, then the quad-set change (inserted + deleted quads, each
as N-Quads) between those two generations. from must still be retained by the ring — without
the time-travel feature only the last few generations (the concurrency window) are retained, so a
from older than that is 410 Gone (widen retention with time-travel); a missing/invalid from
is 400. To recover to a chosen point, restore the base then replay the delta chain forward:
sparq-server --restore base.spqb --restore-delta d0.spqd --restore-delta d1.spqd … (oldest
first; or SPARQ_RESTORE_DELTA). Fail-closed: a corrupt / version-mismatched / out-of-order /
gapped delta aborts the whole recovery and the live store is untouched (import + full replay happen
before any swap). Honest scope: deltas are same-lineage only — the chain must be the writer
history of that base (blank-node labels are stable within a lineage, which is what the quad-set diff
relies on); cross-lineage diffing is unsupported. At-rest encryption is out of scope, same as the
base.
Durable, replayable change-data-capture stream (sparq-serve feature change-stream,
default OFF; sq-b4fns, gh-906). Where the delta above is the PITR backup companion, this is
a continuous CDC stream in the Amazon-Neptune-Streams shape: sparq_serve::change_stream::ChangeLog
records every commit as one ordered, monotonically-sequenced change record —
(seq, generation, timestamp, +inserts / −deletes as N-Quads) — to a segmented, fsync'd
append-only log on disk. A consumer poll(from_seq)s from any offset and replays after a
process restart (ChangeLog::open re-reads the segments, recovers a torn tail, and resumes at the
next seq) — a raw durable cross-service feed (replication / live downstream index / triggers),
distinct from the ephemeral in-process WebSocket/SSE subscriptions (a disconnect misses every
change). Same N-Quads same-lineage quad-set diff + fail-closed FNV-1a digest as the backup family
(no new dependency; no HTTP/async in the library). At-rest encryption + authenticity are out of
scope, same as the backup family. Retention/truncation (sq-n9s4d): ChangeLog::apply_retention
(and the deterministic apply_retention_at) drops whole OLD segments — oldest-first, never the
active segment, never a partial segment — under a RetentionPolicy composing a consumer-ack
watermark (a HARD safety bound: any unacked record keeps its segment), max_age, and
max_total_bytes pressure; the default policy is a no-op and nothing is ever dropped implicitly.
A poll from a trimmed-away offset fails closed (never a silent skip) — consumers resume from
ChangeLog::first_seq. A ChangeSink trait for an external broker (Kafka/NATS) is
tracked as a separate later opt-in (deferred follow-up bead sq-l6zks).
Recording seam (sq-bdaw5): install ChangeLog::into_commit_hook(on_error) via
Writer::spawn_with_commit_hook and every commit-publish is recorded on the writer thread —
one append+fsync per PUBLISHED GENERATION, gapless by construction (the writer is the sole
publisher), with no caller-side lock and no serialisation of submitters; acks happen-after the
record is durable. Restore publishes never fire the hook (not same-lineage — re-baseline
explicitly). A recording failure is dropped + reported to on_error, never failing the
already-published write; the log's discontinuity check then fail-closes later records rather than
silently gapping. Resync (sq-r2cu1): ChangeLog::rebase_to(generation) is that explicit
operator re-base — it appends an honest gap record (ChangeRecord::rebase = true, no changes)
marking the uncaptured span and re-arms recording from generation (strictly forward-only,
fail-closed otherwise) without wiping the log; a consumer replaying past the gap re-bootstraps
(e.g. from a backup at/after that generation) instead of trusting a silently incomplete diff stream.
On a RUNNING recorder, ChangeLog::into_commit_hook_with_control yields a ChangeStreamControl
that runs the resync on the LIVE log (serialized with the hook — no stop/re-open), and
rebase_to_new_lineage is the post-restore variant for a REPLACED lineage whose generation
numbering restarted (gh-2436).
POST /admin/change-stream/rebase — operator resync of the running server's tracked log
(sparq-server feature change-stream; gh-2436). WRITE/admin-gated, POST-only, same
double-opt-in 404 as /streams. Empty body = re-baseline to the writer's CURRENT generation
(the dropped-record resync); JSON fields generation (explicit target) and newLineage: true
(AFTER /admin/restore — the restored ring restarts at generation 0, so the same-lineage
forward-only check would reject it forever). 200 returns the gap record (seq/generation/
rebase + nextSequenceNumber); a non-forward same-lineage target is a 409; an unknown body
field is a fail-closed 400. GET /streams renders the gap as ONE explicit "op": "REBASE"
marker entry (no data) — never silently flattened away.
GET /streams — CDC poll endpoint (sparq-server feature change-stream, default OFF;
sq-2999l, gh-906). The HTTP poll surface over that durable log, in the Amazon-Neptune-Streams
GetRecords shape. Configure a log directory (--change-stream DIR / SPARQ_CHANGE_STREAM,
ServerConfig::change_stream_dir) and the server (1) RECORDS every published group-commit
generation as one ordered change record on the writer thread (possibly containing several
concurrent SPARQL Updates; [GPT-5.6] sq-kqofk), and (2) serves GET /streams over it (the route
is 404 unless the directory is set — the same double-opt-in as /tpf). Parameters: iteratorType
(TRIM_HORIZON = replay all, the default; AT_SEQUENCE_NUMBER + at=N; AFTER_SEQUENCE_NUMBER +
after=N, the resume case; LATEST = tail only) and limit (max commits per page, default 100,
clamped to 10000). The JSON response flattens each commit's quad-level changes to one stream record
per (op, quad) — { eventId: { commitNum: <seq>, opNum: <1-based> }, op: ADD|REMOVE, generation, commitTimestampNanos, data: { stmt: "<n-quads line>" } } — plus a nextSequenceNumber continuation
token (pass as at=/after=), lastSequenceNumber, totalRecords (commit count) and
hasMoreRecords. A poll is a READ (gated by the read auth). A sequence-anchored iteratorType with
no anchor is a fail-closed 400 (never a silent replay-all). With the feature off the route + the
recording hook are #[cfg]-stripped (byte-identical); with it on, recording rides the sequenced
writer's group commit and does not serialise submitters. The ChangeSink broker trait stays a
separate later opt-in (sq-l6zks).
GET /queries + DELETE /queries/{id} — running-query registry (sparq-server feature
query-registry, default OFF; sq-qsm5z, [SONNET-4.6]). An opt-in in-memory registry of
currently executing SPARQL queries, providing GraphDB query-monitoring and kill parity.
GET /queries — READ-gated (fail-closed). Returns {"queries":[…]} where each entry
carries id (opaque hex), start (Unix epoch ms), fingerprint (FNV-1a hex — a
NON-REVERSIBLE hash of the trimmed query text, never raw text per the #241 / sq-m9prn
audit-log posture), and elapsed_ms. Sorted by id (registration order).
DELETE /queries/{id} — WRITE-gated (fail-closed). Cooperatively cancels the named
query by flipping the sq-kq9ia Arc<AtomicBool> cancel flag wired into its QueryBudget.
The engine observes it at the next poll site and aborts. Returns 204 No Content on success;
404 if the id is not found (already finished or bad id).
- RAII lifetime: each executing SELECT/ASK/CONSTRUCT/DESCRIBE/EXPLAIN (plan + analyze)
registers on start and deregisters on completion, error, or panic — the entry is always cleaned
up. (EXPLAIN ANALYZE wiring added in sq-t1isr, [SONNET-4.6].)
- Fingerprint-only: the
GET /queries listing NEVER exposes raw query text; only the
non-reversible fingerprint is visible, satisfying the audit-log discipline.
- Zero cost when feature is OFF: no
AppState fields, no routes; byte-identical to before.
curl -H 'Authorization: Bearer <TOKEN>' http://127.0.0.1:3030/queries
curl -X DELETE -H 'Authorization: Bearer <TOKEN>' http://127.0.0.1:3030/queries/0000000000000001
GSP writes translate into a server-minted SPARQL Update (DROP/CLEAR + INSERT DATA) and submit through the SAME sequenced group-commit writer the
application/sparql-update operation uses — so they share its atomicity, snapshot
consistency, blocking-on-commit semantics, the Sparq-Generation header (default build,
sq-ci2d6), AND its auth gate (a GSP write is as powerful as an UPDATE, so PUT/POST/
DELETE/PATCH are gated by --auth-token exactly like an UPDATE; GET/HEAD are reads). A
malformed body → 400; an unsupported body Content-Type → 415.
PATCH — atomic, graph-scoped in-place modify (sq-hj4n, gh-916). A PATCH on a graph
resource applies an atomic modify to the addressed graph, with two body dialects:
application/sparql-update (ALWAYS-ON, no feature): the body IS a SPARQL Update, applied
atomically through the shared writer. It is scoped to the addressed graph by defaulting the
update's WHERE dataset to that graph (the SPARQL 1.1 Protocol §2.2 using-graph-uri mechanism).
Honest scope: this scopes what the WHERE reads; an operation that explicitly names a
different graph (INSERT DATA { GRAPH <other> { … } }, or an in-body WITH/USING) still
targets where it says — and supplying the override alongside an in-body USING/WITH on a
named-graph PATCH is the §2.2 conflict 400. For ?default the scoping is a no-op (the default
graph is already the WHERE default). Success → 204.
text/n3 (OPT-IN — the n3-patch build feature AND --n3-patch / SPARQ_N3_PATCH=1): a
Solid-style N3-Patch (solid:InsertDeletePatch with solid:where / solid:deletes /
solid:inserts formulas), translated into ONE atomic graph-scoped SPARQL Update (every block
wrapped in GRAPH <g> { … } for a named graph). With a solid:where it is a pattern-based
DELETE/INSERT … WHERE; without one it is ground DELETE DATA/INSERT DATA — either way one
writer submission. Validation: exactly one patch resource, each formula property at most once, at
least one of deletes/inserts, and no blank node in deletes/where (only inserts may mint
one) → otherwise 400. Double opt-in (build feature + runtime flag) mirrors tpf/shacl:
with the feature off the N3-Patch parser + dispatch are #[cfg]-stripped (a text/n3 body is
then a plain 415, byte-identical to before, and no new dependency / no sparq-core /
sparq-engine / wasm impact); with the feature on but the flag off a text/n3 body is
also 415. The N3 parsing rides oxttl's N3Parser — already a server dependency — so even
feature-on adds NO new crate. Success → 204. PATCH is a WRITE, gated by --auth-token like an
UPDATE.
5b. Container (ghcr.io). Published on every vX.Y.Z release tag as a distroless OCI image
index at ghcr.io/jeswr/sparq-server, with linux/amd64 and linux/arm64 runtime images.
[GPT-5.6] sq-fvzi6: each release publishes :X.Y.Z (pin this for reproducible deployments),
:X.Y (tracks the newest patch in that minor line), and :latest (tracks the newest release);
an omitted tag selects :latest. The image sets SPARQ_ALLOW_REMOTE=1 so the 0.0.0.0 bind it
needs (loopback is unreachable through Docker's port map) boots out of the box — running the
container is the operator's explicit choice to publish a surface. This default has NO auth.
Because every SPARQ_* var is read from the environment, secure it with -e (no flag wiring):
docker run --rm -p 3030:3030 ghcr.io/jeswr/sparq-server
docker run --rm -p 3030:3030 -v "$PWD/data:/data:ro" \
ghcr.io/jeswr/sparq-server --format turtle /data/dataset.ttl
docker run --rm -p 3030:3030 \
-e SPARQ_AUTH_TOKEN="$TOK" -e SPARQ_AUTH_TOKEN_READ=1 \
ghcr.io/jeswr/sparq-server
Deliver the token over TLS (terminate at a proxy). See crates/sparq-server/README.md →
"Running the container image".
The image self-declares a HEALTHCHECK (CIS Docker §4.6, sq-toze.36). Since distroless has
no shell/curl/wget, the check is the server binary probing itself: sparq-server --health-probe opens a TCP connection to the loopback /health and exits 0 (healthy)/non-zero
(unhealthy) — exec-form in the Dockerfile. Override the probed address with --health-probe-addr HOST:PORT or SPARQ_HEALTH_PROBE_ADDR. docker ps then shows (healthy)/(unhealthy); k8s
usually runs its own /health probe and ignores the image healthcheck.
5c. Federation discovery — VoID + Service Description (OPT-IN, sq-d3d8). A server can
advertise itself as a discoverable federation node by serving two read-only RDF descriptors:
-
GET /.well-known/void — a W3C VoID dataset description
(void:triples / void:entities / void:classes / void:properties + one
void:classPartition per class and void:propertyPartition per predicate), plus the
characteristic-set source statistics (sq-mr32, federation A3/Z2): sparq already mines
per-entity-type predicate co-occurrence + multiplicity (Neumann & Moerkotte characteristic
sets), so the served VoID also emits them under a documented sparq extension vocab
scs: (<http://sparq.dev/ns/cs#>) — scs:characteristicSet linking the dataset to one
scs:CharacteristicSet node per retained set (scs:subjects = count(C), plus one
per-predicate scs:predicateStat reusing void:property/void:triples and adding
scs:avgMultiplicity), with the EXACT distinct-set count on scs:distinctCharacteristicSets.
The CS stats are a strict superset of the standard VoID (a VoID-only client ignores the
scs: triples; a CostFed/Odyssey-class source-selector gets sharp star/multi-join
cardinalities). Generated by sparq-introspect's Introspection::to_void_with_cs
(Introspection::to_void remains the CS-free base).
-
GET /sparql with no query parameter — a SPARQL 1.1 Service
Description generated from the server's
actual capabilities (sq-qfcb), never a hard-coded fiction:
sd:Service + sd:endpoint (the request Host's /sparql);
sd:supportedLanguage — SPARQL11Query always, plus the SPARQL-1.2-SD
version-agnostic SPARQLQuery (sq-2msb); SPARQL11Update + SPARQLUpdate only when an
anonymous client can run one (suppressed when a --auth-token write gate is configured,
because then an unauthenticated SD reader cannot use Update);
sd:supportedVersion (sq-2msb, gh-917) — the SPARQL language versions this build
conformance-verifies, as sparql:version-* IRIs (http://www.w3.org/ns/sparql#). SPARQL
1.2 SD moves version negotiation off sd:Language onto sd:supportedVersion, so a
1.2-aware federation client can discover triple-term / dir-lang support without probing.
sparq advertises version-1.0, version-1.1 and the full version-1.2 (not the
version-1.2-basic profile) because the engine passes the complete W3C SPARQL 1.0/1.1/1.2
suites at 100% (conformance-report.md). HONESTY GATE: there is no sparql12/rdf12
cargo feature — SPARQL 1.2 is compiled into the base engine — so this is keyed off the
DOCUMENTED conformance state (descriptors::CONFORMANCE_VERIFIED_VERSIONS), never a cfg!
or an aspiration; were any 1.2 group to regress to a partial pass, the honest edit is to
drop to version-1.2-basic (or omit 1.2) in that one constant;
sd:resultFormat — the four SPARQL-results serialisations (JSON/XML/CSV/TSV) plus the RDF
graph serialisations the CONSTRUCT/DESCRIBE/GSP-read path emits (Turtle/N-Triples/RDF-XML),
and sd:inputFormat — the RDF serialisations the GSP write path parses (Turtle/N-Triples/
RDF-XML). These mirror exactly what crate::negotiate produces/accepts; an integration test
drives a real SELECT per advertised result format and asserts the returned Content-Type
matches, so the advertisement cannot over-promise;
sd:feature sd:BasicFederatedQuery — only when the service cargo feature is compiled
in (the SERVICE clause is then evaluable); omitted otherwise;
sd:feature <http://sparq.dev/ns/prov#lineage> (sq-yyy3) — the sparq W3C-PROV-O
data-lineage extension, advertised only when this node genuinely serves PROV-O lineage
for the data it derives (Capabilities::provenance). sparq-server exposes no lineage
endpoint today, so it keeps the flag false and the feature is omitted — advertising it
would over-promise. The descriptor + the sparq-fedclient discovery parser
(Capability::provenance_lineage) support the round-trip so a lineage-serving node can flip
it honestly without a vocabulary change;
sd:extensionFunction — one per function the engine has actually registered: the
geof: GeoSPARQL set with the geo feature (read back through
FunctionRegistry::iris(), so it can never drift from what runs), none without it;
- the default dataset — its default graph linked to the VoID document via
dcterms:source,
plus an sd:namedGraph enumeration of every IRI-named graph in the served dataset
(sq-optl): each is an sd:NamedGraph with its sd:name (the FROM NAMED-referenceable IRI)
and an sd:graph sd:Graph carrying that graph's void:triples count. The names come off
the same pinned snapshot the VoID descriptor reads, sorted for determinism, and IRI-only
— a blank-node-named graph is skipped because it is not FROM NAMED-referenceable, so
advertising it would be a fiction. A default-only dataset emits no sd:namedGraph.
Note sd:UnionDefaultGraph is deliberately not advertised — the engine's default graph
holds only default-graph triples (named graphs are not folded in), so claiming it would be
dishonest.
Both are double opt-in and OFF by default: compiled only with the
federation-descriptors cargo feature and served only when
--federation-descriptors / SPARQ_FEDERATION_DESCRIPTORS=1 is also set. Without the
feature there is zero cost (no sparq-introspect dependency, no routes); with the feature
but not the flag, /.well-known/void is 404 and a no-query GET /sparql is the historical
400 missing 'query'. Both content-negotiate the RDF syntax (Accept: text/turtle default,
application/n-triples, application/rdf+xml); reads are gated by --auth-token-read like
any GET. The dataset/endpoint IRIs self-describe the Host the client used.
cargo run -p sparq-server --features federation-descriptors -- data.ttl --federation-descriptors
curl http://127.0.0.1:3030/.well-known/void
curl -H 'Accept: application/n-triples' http://127.0.0.1:3030/sparql
5c-bis. Grouped facet counts (OPT-IN, [GPT-5.6] sq-lsp7k.5.2; feature facets).
POST /facets evaluates one sparq_introspect::FacetRequest against a pinned store snapshot and
returns its FacetResponse JSON: candidate-subject count plus ranked type, predicate, and requested
object-value distributions. class is an optional rdf:type class IRI; every constraints pair is
[predicate IRI, object N-Triples term] and the pairs are AND-combined; facet_predicates: null
requests values for every candidate predicate; top_k bounds every retained distribution.
cargo run -p sparq-server --features facets -- data.ttl --facets
curl -X POST -H 'Content-Type: application/json' http://127.0.0.1:3030/facets -d '{
"class":"http://ex/Person",
"constraints":[["http://ex/status","<http://ex/active>"]],
"facet_predicates":["http://ex/tag"],
"top_k":10
}'
The response is 200 application/json; malformed JSON is a sanitized 400. The scan runs on the
blocking pool and holds one immutable generation pin for the whole evaluation. The endpoint is a
READ, so --auth-token-read gates it. It is double opt-in and OFF by default: the facets cargo
feature compiles the route + sparq-introspect dependency, and --facets / SPARQ_FACETS=1
serves it. Feature on but flag off gives 404; feature off compiles no route or dependency.
5c-ter. IRI/label prefix completion (OPT-IN, [GPT-5.6] sq-lsp7k.9.3; feature
complete). GET /complete?q=<prefix>&limit=<k> returns case-insensitive prefix matches from
complete IRI strings, IRI local names, and literal rdfs:label / skos:prefLabel values. q is
required (missing → 400); limit defaults to 20 and is capped at 100. The response is a JSON
array in deterministic rank/key/id/source order:
[
{"iri":"http://ex/alice","key":"alice","kind":"localName","score":0.0},
{"iri":"http://ex/alice","key":"alice","kind":"label","score":0.0}
]
kind is iri, localName, or label. One entity may appear more than once when multiple keys
or sources match. v1 passes no external rank scores, so score is 0.0; PageRank-backed ranking
is a separate composition step. The server pins one immutable generation, lazily builds its
CompletionIndex on the blocking pool, caches it in AppState, and rebuilds when a later update
publishes a different generation. Thus a successful update is visible to the next completion
request without serving stale candidates.
cargo run -p sparq-server --features complete -- data.ttl --complete
curl -G http://127.0.0.1:3030/complete --data-urlencode 'q=ali' --data-urlencode 'limit=10'
The endpoint is a READ, so --auth-token-read gates it. It is double opt-in and OFF by default:
the complete cargo feature compiles the route and the pure-index sparq-text dependency
(default-features = false), while --complete / SPARQ_COMPLETE=1 serves it. Feature on but
flag off gives 404; feature off compiles no route, cache, or text dependency.
5d. Triple Pattern Fragments / LDF source endpoint (OPT-IN, sq-bzh1). A server can expose
itself as a low-cost Linked Data Fragments /
Triple Pattern Fragments
source that a TPF client (Comunica / the LDF client) drives a join against — far cheaper per
request than a full SPARQL endpoint:
GET /tpf?subject=&predicate=&object= — a paged RDF fragment of the triples matching one
triple pattern. Each of subject / predicate / object is an N-Triples term
(<iri>, "lit", "lit"@en, "lit"^^<dt>); an absent / empty parameter is a variable
(unbound). page=N (0-based) selects the page; the page size is bounded (default 100).
- The fragment carries Hydra controls:
hydra:totalItems / void:triples (the matched-triple
count, reusing the engine's cheap cardinality estimate — NOT a full scan),
hydra:itemsPerPage, the full PartialCollectionView paging vocabulary —
hydra:next / hydra:previous (present only when a next / previous page exists) plus
hydra:first / hydra:last (emitted on EVERY page so a client can jump to either end of the
view from anywhere; first is always page 0, last is the page holding the final match —
derived from the same estimate as totalItems/next) — and a hydra:search / hydra:template
/ hydra:mapping control describing the {subject,predicate,object} URI template so a generic
client can request any other pattern.
5d-bis. brTPF — bind-restricted Triple Pattern Fragments (OPT-IN, sq-dxhb; feature brtpf,
implies tpf). brTPF (Hartig & Buil-Aranda, ODBASE 2016)
extends the SAME /tpf endpoint so a client can attach a set of solution mappings and the
server returns only the page of pattern matches COMPATIBLE WITH AT LEAST ONE supplied binding —
pushing a bind-join's semi-join down to the source, so far less data crosses the wire than
re-fetching the whole pattern once per binding.
- The binding set rides the
values query parameter (GET) or — preferred for a large set —
the request body of a POST /tpf. The wire format is one mapping per line; within a line,
whitespace-separated position=term pairs where position is subject/predicate/object
(or short s/p/o) and term is the SAME N-Triples-term grammar as the pattern parameters
(e.g. s=<http://ex/alice>). A blank line / empty payload is the no-restriction (plain-TPF)
case; a malformed payload is a sanitized 400 (the offending input is NOT echoed).
- The fragment is the deduplicated union of each mapping's specialised-pattern matches.
hydra:totalItems and the paging window reflect the bindings-RESTRICTED result, not the
unrestricted pattern, and the hydra:search control advertises an extra hydra:mapping for the
values variable so a client discovers the dataset accepts a restriction.
- A
tpf-only build is byte-identical to before: the values parsing + the POST route are