sparq

name	sparq
description	How to USE the sparq RDF triplestore + SPARQL 1.1/1.2 engine (Rust crates, CLI, HTTP server, Python, JS/WASM) and its capabilities (reasoning, SHACL, full-text, vector, GeoSPARQL, streaming RSP-QL, ZK query proofs). Router skill — read this first, then the per-surface skill that matches the task.
license	MIT
metadata	{"version":"0.1.0","homepage":"https://github.com/jeswr/sparq"}

sparq — usage skills (router)

sparq is a high-performance RDF + SPARQL engine (Rust workspace; also shipped as a CLI, a W3C SPARQL HTTP server, a Python package — PyPI sparq-rdf, import sparq — and a WASM/JS package). These skills teach an agent how to use sparq. (Skills under .claude/skills/ are different — those are for developers working on this repo.)

Pick the surface that matches the task:

• sparql-query — Run SPARQL 1.1/1.2 queries (SELECT/ASK/CONSTRUCT/DESCRIBE) and UPDATE against the sparq RDF engine in Rust — load RDF into a sparq_core::Graph, then use sparq_engine::{query, ask, query_json, count, construct, describe, update}; covers property paths, RDF 1.2 triple terms, aggregates/subqueries, custom extension functions (query_with_functions / FunctionRegistry), prepared queries, query budgets/timeouts, named-graph dataset views, and EXPLAIN. Use when an agent or developer needs to embed/execute SPARQL over sparq. (status: Verified against sparq-engine 0.1.0 source (crates/sparq-engine/src/{lib.rs,construct.rs,update.rs,explain.rs}) and integration tests on branch main (2026-06-13). Supported query surface (per code + README): SELECT/ASK natively; CONSTRUCT/DESCRIBE via dedicated functions; FILTER with the full builtin set (REGEX/hash with default features); OPTIONAL/UNION/MINUS/BIND/VALUES; aggregation + GROUP BY/HAVING; ORDER BY; DISTINCT/REDUCED/LIMIT/OFFSET; sub-SELECT; all 8 property-path operators; named graphs (GRAPH, FROM/FROM NAMED via load_dataset); RDF 1.2 triple-term patterns (incl. variables inside the triple term, when oxrdf/oxttl rdf-12 is enabled — default in this workspace). SPARQL Update is the full operation set over default+named graphs. Caveats verified in source: DESCRIBE returns concise-bounded-description (not configurable); CONSTRUCT templates do not support RDF 1.2 triple terms in SUBJECT position (object triple-terms are instantiated); errors are plain String; QueryBudget enforcement is cooperative/approximate (coarse check sites). On wasm (sparq-wasm disables default features) regex/hash builtins and QueryBudget.deadline are unavailable; UUID()/STRUUID() native-only. The non-default zk feature only adds a witness-recording trace seam; it does not change query results. Code carries [OPUS-4.8] review markers (pending re-review).)
• data-formats — Parse and load RDF into a sparq Graph (Turtle/N-Triples/N-Quads/TriG via sparq-core, and HDT incl. compressed .hdt.gz/.hdt.zst/.hdt.bz2 via sparq-hdt), do streaming/parallel/external-memory ingest of compressed dumps, and take cheap immutable copy-on-write Graph snapshots. Use when ingesting RDF files, choosing a loader, wiring HDT, or snapshotting a graph for serving. (status: Verified against the source on 2026-06-13 (sparq-core + sparq-hdt at workspace v0.1.0). Caveats: (1) sparq-core ships NO RDF text serializer — only parsing/loading. To emit RDF, iterate Graph::iter_ids() + Dict::term and drive an oxttl serializer yourself (recipe 6 shows the shape; the oxrdf Triple reconstruction is left as match-on-term-kind because there is no helper). (2) HDT is read-only — writing .hdt archives is unsupported (blocked upstream in the wrapped hdt crate). (3) HDT (sparq-hdt) is native-only with MSRV 1.87 and opt-in in the CLI via --features hdt; it never enters the wasm build. (4) The parallel parse fast paths require the parallel feature (default native, off for wasm); load_reader_parallel's pipeline is N-Triples-only and silently falls back to serial for other formats. (5) build_external/open/save require the mmap feature; dict-spill (RSS-bounded build) further requires that feature + the SPARQ_DICT_SPILL env var. (6) parse_to_triples/load_str route any UNKNOWN format string to the Turtle parser (the _ => arm) — pass the exact accepted format string.)
• jsonld — Parse, expand, flatten, compact, and frame W3C JSON-LD 1.1 with sparq: the native, dependency-free document-level pipeline in the sparq-jsonld crate (expand/flatten/compact/frame/from_rdf over a JSON AST, deny-by-default DocumentLoader), plus how each surface exposes the four output forms (CLI dump … jsonld[-compact] --context, the HTTP server's application/ld+json flattened content-negotiation, the wasm serializeCompact). Use when converting JSON-LD document forms, choosing a surface, or reasoning about the honest conformance / remote-loading posture. (status: [GPT-5.6] sq-dzgdh, verified against crates/sparq-jsonld/src, examples/jsonld_roundtrip.rs, and research/jsonld-interop-matrix.md on 2026-07-13. HONEST: sparq-jsonld is publish = false with zero mandatory deps and no cargo feature of its own — the jsonld opt-in gating lives in the consumer crates (CLI/server default-on, wasm opt-in). expand/flatten/compact/frame/from_rdf are implemented; to_rdf (native JSON-LD→RDF) is a documented stub. The surfaces currently serialise from a Graph via the engine's RDF-first writer, not yet the native pipeline (wiring is bead sq-oy1f.41); only the server's flattened form is a verified wire surface, and CLI/wasm framing are planned (sq-oy1f.42/.44). The six conformance floors — toRdf 413, fromRdf 52, expand 276, flatten 53, compact 228, frame 92 — are MEASURED rise-only minima at the pinned suite revision, NOT a blanket conformance claim.)
• rdf-wrapper — Traverse a sparq_core::Graph as Rust-native focus objects with the opt-in sparq-wrapper crate: borrowed/owned store lifetimes, .out()/.r#in() iterators, raw-term .values(), strict string/integer/boolean/typed-literal accessors, and owned-store mutation. Includes default-OFF implementations of rdfjs/wrapper's still-unlanded distinct-result, typed-cardinality-error, and graph-scope proposals. Use instead of handling raw triples or dictionary IDs in application code. (status: M1 [GPT-5.6], sq-1rg2q. The base wrapper traverses the default graph; proposed-graph-scope adds explicit read-many/write-one named-graph views [GPT-5.6] sq-1rg2q.6. SHACL-to-Rust object generation is decomposed separately and must reuse sparq-shacl::ShapesModel.)
• rdf-canon — Canonicalize an RDF dataset with RDFC-1.0 (the W3C RDF Dataset Canonicalization, the URDNA2015 successor) via the opt-in sparq-canon crate: oxrdf quads (or one graph's triples) → deterministic, blank-node-relabelled canonical N-Quads string, or just the canonical blank-node issuer map. Use to hash/sign/diff/deduplicate/content-address a dataset, or test two graphs for RDF-isomorphism. (status: Verified against sparq-canon 0.1.0 (2026-06-15). W3C rdf-canon test-suite validated (eval + issued-map + negative, SHA-256 and SHA-384) through the public API. Opt-in: publish = false, non-default workspace member — sparq-core's default build and the wasm artifact are untouched unless pulled in. The RDFC-1.0 algorithm is the rdf-canon 0.15.3 crate; sparq-canon owns the single oxrdf-0.3↔0.2 bridge. RDF-1.2 quoted-triple terms are out of the data model (rejected); poison graphs fail closed at the HNDQ call limit.)
• cli — Use when you need to drive the sparq RDF/SPARQL engine from the command line: load a Turtle/N-Triples/N-Quads/TriG (or HDT) file and run a SPARQL query, build/query memory-mapped on-disk indexes for datasets larger than RAM, materialize RDFS/OWL-RL/N3 reasoning closures, stream-ingest huge gzip/bzip2/zstd dumps, or benchmark query suites. Covers the actual sparq-cli subcommands, positional argument order, and cargo feature flags. (status: Verified against crates/sparq-cli/src/main.rs (915 lines, single-file binary) and the hdt_cli integration test at HEAD (commit f7f8383). The CLI is a hand-rolled positional-arg parser — NOT clap: there is no --help, the only GNU-style flags are --reason <profile> and --proof (N3). query/query-mmap print only the solution COUNT to stdout, never the bindings — confirmed by the HDT test asserting '328 solutions'. The ingest, scaling, probe-compress, compare-compress, bench-remap subcommands are measurement/experiment harnesses (Wikidata-vs-QLever/RDFox comparisons, browser-storage sizing) rather than general user tooling; documented but flagged as instrumentation. HDT support is opt-in (--features hdt) and read-only, and raises the binary's MSRV from 1.85 to 1.87. Version 0.1.0; APIs may still shift.)
• http-server — Run or point an agent at a sparq SPARQL 1.1 Protocol HTTP endpoint (sparq-server): /sparql query+update over GET/POST, content negotiation (SELECT/ASK JSON/XML/CSV/TSV; CONSTRUCT/DESCRIBE + Graph Store N-Triples/Turtle/RDF-XML), Graph Store read AND write (PUT/POST/DELETE), EXPLAIN, Prometheus /metrics, WebSocket + SSE subscriptions, and opt-in time-travel ?generation pinning. Use when starting the server, querying/updating a running endpoint, choosing Accept/Content-Type, or embedding the axum router. (status: Server stack is production-shaped and integration-tested over real HTTP (tests/protocol.rs, hardening.rs, updates.rs, subscriptions.rs, time_travel.rs, named_graphs.rs); re-verified against current source 2026-06-19 [OPUS-4.8]. Current scope (named-graph claims previously here were STALE and are fixed): (1) Named graphs are REAL — the engine stores the full dataset (default + named), so GRAPH /GRAPH ?g patterns evaluate and GSP graph resources address genuine named graphs; FROM/FROM NAMED and the protocol's default-graph-uri/named-graph-uri (and update using-graph-uri/using-named-graph-uri) dataset-override params are APPLIED, not no-ops (sq-z33x). (2) Graph Store write verbs PUT/POST/DELETE are supported (RDF/XML, Turtle, N-Triples bodies). (3) SPARQL Update handles the full operation set — INSERT DATA / DELETE DATA / CLEAR / DROP / CREATE (DEFAULT|named|ALL) / LOAD / DELETE-INSERT…WHERE — over the default AND named graphs; a multi-operation ';'-separated body is taken as ONE request and applied atomically (one commit, all-or-nothing); a failing op is a 400 with no partial effect. (4) Durability is opt-in: default in-memory (updates lost on restart), or --persist DIR for WAL-fsynced durability that survives restart with no rebuild. (5) time-travel and geo are opt-in cargo features (default off); each retained time-travel generation is a full Graph (large memory), and time-travel pinning is /sparql queries only. CONSTRUCT/DESCRIBE + GSP serialise to N-Triples/Turtle/RDF-XML; DESCRIBE returns CBD.)
• helm-deploy — [GPT-5.6] Deploy sparq-server or native sparq-lws-core to Kubernetes with the in-repo Helm chart or plain manifest: existing-Secret auth, ingress-nginx TLS, selector-dependent images/ports/probes, non-root/read-only hardening, persistence, and the single-replica data-safety boundary. Use when installing or validating deploy/helm.
• http3-server — Embed the internal, default-off sparq-http3 transport bridge in an axum server: convert an aws-lc-rs-backed rustls 0.23 configuration into Quinn, serve h3 on a UDP endpoint, dispatch streaming requests into a cloned Router, inject peer ConnectInfo<SocketAddr>, and shut down the endpoint. Use when wiring the shared HTTP/3 listener into sparq-server or sparq-lws-core; the end-user server flags and Alt-Svc advertisement are separate surfaces. ([GPT-5.6] sq-oprna.1 added the helper; sq-oprna.2 wires the default-off sparq-lws-core/http3 listener when TLS is configured. h3/h3-quinn remain pre-1.0 and contained in the helper; Alt-Svc is a separate phase.)
• python — Use the sparq RDF + SPARQL engine from Python (PyPI distribution sparq-rdf; import sparq). Reach for this when an agent or developer needs to load RDF (Turtle/N-Triples/N-Quads/TriG), run SPARQL 1.1 SELECT/ASK/CONSTRUCT/DESCRIBE, apply SPARQL Update, do opt-in RDFS/OWL-RL/Notation3 reasoning + OWL inconsistency checks, run BM25 full-text search (text: magic predicates), or persist/memory-map a triplestore — all via the pyo3 sparq.Graph class. (status: Alpha (pyproject classifier 'Development Status :: 3 - Alpha'). The PyPI distribution name is sparq-rdf (pip install sparq-rdf) but the import name is sparq (import sparq) — they differ because the bare sparq distribution name is taken on PyPI; the crate (crates/sparq-py, publish = false) is built/distributed via maturin, not crates.io — at the time of writing there may be no published wheel, so expect to build from source with maturin develop. API verified directly against crates/sparq-py/src/lib.rs and the passing pytest suites (tests/test_basic.py, tests/test_text.py) at repo HEAD. abi3-py39 → CPython >= 3.9, one wheel per platform. The wheel bundles reasoning (sparq-reason) and full-text (sparq-text); the ZK/Noir estate is NOT exposed in Python. Full-text + the TextIndex lifecycle are noted as a follow-up in the crate's open beads but are implemented and tested. Term.kind 'triple' (RDF 1.2 triple terms) is rendered N-Triples-style and is not the common path.)
• javascript-wasm — Use the sparq RDF+SPARQL engine from JavaScript/TypeScript (Node >=18 or the browser) via its WebAssembly build and the @jeswr/sparq RDF-JS wrapper: load Turtle/N-Triples/N-Quads/TriG into an in-memory store, run SPARQL 1.1 SELECT/ASK, stream large results, count without materialising, apply SPARQL Update / quad deltas, do RDF-JS match()/countQuads(), and ingest gzip/zstd-compressed RDF. Reach for this when wiring sparq into a Node service, browser tab, or RDF-JS pipeline — not when editing the Rust crates. (status: Verified against the current source (js/src/*.ts, crates/sparq-wasm/src/lib.rs, js/README.md, js/package.json) as of 2026-06-13. Honest limitations: (1) The high-level SparqStore wrapper exposes SELECT/ASK only — CONSTRUCT/DESCRIBE require dropping to the raw wasm Store.queryQuads (which returns N-Triples), and SERVICE/federation is unsupported. (2) REGEX/REPLACE are compiled out of the default wasm bundle (non-default regex cargo feature). (3) dataset and compressed options are mutually exclusive (no compressed dataset loader yet). (4) Mutations are overlay-based: append-only dictionary, delete tombstones until reload. (5) Browsers truncate multi-member gzip to the first member; zstd dictionary frames need an external dict-capable decoder (fzstd cannot). (6) The package is unpublished-style local (@jeswr/sparq v0.1.0, ESM, build via npm run build = wasm-pack --target web + tsc); the @jeswr/sparq/wasm/... import path for the raw Store reflects the package files layout and may differ in a source checkout (where it's ../wasm/sparq_wasm.js). The bench harness (bench/vs-oxigraph.mjs) and dictionary protocol are present but secondary.)
• javascript-wasm, Solid host — [GPT-5.6] Run the separate @jeswr/solid-server loopback Node package with npx, or import startSolidServer({ port, baseUrl, ownerWebid, oidc }). It owns one in-memory Solid/LDP + WAC wasm pod. The v1 listener defaults to unauthenticated fixed-owner mode and offers opt-in Node-side Solid-OIDC/DPoP verification; it remains local-development software and must not be exposed as a production server.
• inference — Use when you need RDFS, OWL 2 RL, or Notation3/EYE-rule entailment over a sparq RDF graph — materialize the deductive closure (forward-chaining), query the entailed triples, maintain the closure incrementally under inserts/deletes, get derivation proof-trees (why()), or check OWL inconsistency; backed by the sparq-reason crate. (status: Verified against sparq-reason v0.1.0 source on 2026-06-13 (crates/sparq-reason). Maturity by profile: RDFS batch+incremental is the most complete (the non-explosive subset rdfs2,3,5,7,9,11 — no axiomatic/reflexive triples). OWL 2 RL covers a useful subset (sameAs/equality, inverseOf, Symmetric/Transitive/Functional/InverseFunctional, equivalentClass/Property, XSD numeric tower) and includes RDFS; incremental OWL silently drops to a full-rematerialize Fallback mode for sameAs/Functional/chains/restrictions/cardinality/hasKey/oneOf/intersection/union and on any TBox mutation (check .mode()/.full_rebuilds()). N3 is forward-chaining with EYE/cwm parity (~98.8% of the reasoner manifest run) incl. goal-directed <= and a large math:/string:/list:/time:/log: builtin set; the incremental MaterializedN3Graph only runs its exact-counting fast path for a restricted monotone, input-stratified rule fragment (whitelisted builtins log:uri/equalTo/notEqualTo, string:concatenation/scrape/encodeForUri; store-scoped notIncludes guards) and otherwise re-runs the batch engine every mutation (check .mode()/.fallback_reason()). The explain/why() feature is explicitly opt-in and returns ONE derivation (a witness), not all derivations. Native-only crate (pulls regex + rayon) — never compiled into the wasm dependency graph; for wasm/single-thread build with default-features=false. Incremental TBox maintenance and live integration with the on-disk store overlay are documented follow-ups (every TBox edit triggers a full rebuild today).)
• prov-lineage — Capture W3C PROV-O data lineage for derived RDF, or explain a missing target binding for one BGP through the opt-in sparq-prov crate. Use for CONSTRUCT/DESCRIBE, UPDATE, and reasoner lineage; enable the non-default why-not feature to report exactly which target-substituted triple patterns are absent and serialize them as deterministic RDF 1.2. (status: The CONSTRUCT/DESCRIBE derivation path, SPARQL UPDATE write path, and non-default reason proof-tree bridge are covered. [GPT-5.6] sq-lsp7k adds bounded missing-answer explanation under why-not: one BGP only, exact absent-conjunct set in pattern order plus PROV-flavoured N-Triples/Turtle; other algebra and invalid/incomplete target bindings fail closed. Opt-in standalone crate (publish=false): nothing in sparq's default build or wasm artifact depends on it; zero core overhead when absent.)
• arrow-columnar — Move SPARQL SELECT QueryResult values through the opt-in sparq-arrow crate as a faithful Apache Arrow RecordBatch or in-memory Parquet bytes. Use for dataframe, analytics, or storage interop that must preserve RDF term kinds, literal metadata, RDF 1.2 triple terms, empty literals, and unbound cells. (status: Arrow import/export plus [GPT-5.6] Parquet round-trip under separate default-OFF arrow / parquet features; neither dependency enters the lean engine or WebAssembly build.)
• shacl-validation — Validate RDF data against SHACL shapes with the sparq engine: SHACL Core constraints (class, datatype, cardinality, ranges, paths, logical, node/property, qualified, closed, in/hasValue), SHACL-SPARQL sh:sparql constraints (§5.2), and custom SPARQL-based constraint components (sh:ConstraintComponent, §6) — then read the conformance/violations validation report (W3C report vocabulary as Turtle or human text). Use when an agent needs to check whether a sparq_core::Graph conforms to shapes, run shape validation, or produce a SHACL validation report in Rust. (status: Stable and well-tested. SHACL Core: 98/98 of the W3C data-shapes core sht:Validate suite passes (pinned commit b6e73695). SHACL-SPARQL (sh:sparql, §5.2) and custom SPARQL-based constraint components (sh:ConstraintComponent, §6) are BOTH implemented and pass local + W3C sparql/node + sparql/property sub-suites (the crate's own tests/sparql_constraints.rs and tests/sparql_components.rs all pass); the crate README documents §6 components as implemented (no stale "deferred" caveat to discount — model.rs discover_components / Component::CustomSparql + the sh:validator validators). Known scope limits (per the crate's open beads, mirrored in the README's "Scope and non-goals"): the full sparql/pre-binding semantics (rejecting variable re-binding, $shapesGraph) are not implemented; the W3C sparql/component/* suite isn't runnable offline because it owl:imports external dash vocabulary, so §6 is exercised via inline-declared components. Validation results are intentionally NOT deduplicated, ill-formed shapes/queries are silently skipped rather than erroring, and recursion is treated as conforming. Native-only crate (no [features] of its own); never in the wasm graph. Verified against the source on 2026-06-19 [OPUS-4.8]; cargo test -p sparq-shacl passes (80 in-crate unit tests plus the integration + W3C-suite test files under tests/).)
• shacl-compact-syntax — Parse and faithfully write SHACL Compact Syntax with the opt-in sparq-shaclc crate: strict W3C Community Group syntax plus its RDF 1.2 layer, or the explicit shaclc-js extended profile, into oxrdf::Triple shapes consumable by sparq-shacl. Covers typed source errors, base/prefix outcomes, the generated callback/push parsers, and the all-or-nothing residual-consumption writer. Use when ingesting or emitting SHACL-C/SCS rather than validating an already-built shapes graph. ([GPT-5.6] New public surface in PR #2136; separate opt-in crate, no build-time code generation.)
• shacl-forms — Derive a DASH-compatible, renderer-agnostic form description from SHACL shapes with the opt-in sparq-forms crate, then turn editable value changes into one SPARQL 1.1 DELETE/INSERT request with FormDiff / to_sparql_update. The serde-JSON FormDescription includes an applicable-shape switcher, property-group/field layout, DASH widget selection, cardinality, constraints, and nested sub-forms. Use when an agent or GUI needs a headless shape-directed data-entry/edit/view form and a pure update builder (no GUI deps, builds for wasm32). (status: F1 derivation plus the pure F4 diff kernel; applying/guarding commits, in-form validation/suggestions, and the GUI renderer remain follow-on work.)
• full-text-search — Full-text search over RDF string literals in sparq via the sparq-text crate: build a BM25 inverted index (TextIndex) and run text: magic predicates (text:matches / text:matchesAny / text:phrase / text:near / text:slop / text:score) inside plain SPARQL. Use when an agent needs keyword/prefix/phrase/proximity search, BM25 or proximity relevance ranking, or autocomplete over literals in a sparq Graph. (status: APIs verified against current source (crates/sparq-text/src/{lib,index,rewrite,tokenize}.rs) and the integration tests (tests/e2e.rs, tests/phrase.rs) on branch main, 2026-06-19 [OPUS-4.8]. The crate is at version 0.1.0 and marked OPUS-4.8 / opt-in. text:phrase (adjacency) and text:near (proximity/slop, relevance-ranked) are both wired into the SPARQL magic predicates and documented in the crate README — there is no longer any stale "future work" caveat to discount; trust the code. Positions are opt-in (build_with_positions / with_positions); the default build stores none, and text:phrase / text:near against a positionless index is a hard query error. Tokenizer is deliberately minimal (no stemming/stopwords/diacritic folding) and CJK is whole-run match. BM25 constants k1=1.2, b=0.75. Benchmark numbers are non-canonical — run the crate's benches rather than baking them here.)
• vector-search — Semantic / ANN vector search over a sparq RDF graph: build a memory-mapped per-term-id embedding store (.spqv), then run cosine top-k with an in-RAM HNSW, a persistent on-disk DiskANN/Vamana graph (.spqg), or an exact brute-force baseline; verbalize entities (label+type+description) for embedding, scalar/product quantize (SQ/PQ) for large stores, and fuse with another ranked signal (RRF / score blend) for hybrid retrieval. Use when adding embedding/semantic-search/nearest-neighbour over a sparq Graph in the sparq-vectors crate. (status: Stable & tested, but read these caveats: (1) The crate is OPT-IN and nothing in the workspace depends on it — the default engine build does not compile it, and there is NO SPARQL-level integration (no SERVICE / function / GRAPH binding to vectors); it is a standalone Rust library over sparq-core's public read API. (2) Embeddings are produced OUT-OF-PROCESS: the only built-in Embedder, HashEmbedder, is deterministic lexical n-gram hashing with NO semantics ("car" ≈ "automobile" is false) — it exists to exercise the store/ANN/pipeline in tests; supply a real Embedder for retrieval. (3) The live provider (OpenAI-compatible /v1/embeddings) lives behind the non-default provider cargo feature and never opens a socket — you must supply a Transport impl (reqwest/ureq/cassette). (4) The HNSW VectorIndex is rebuilt from the mmap'd store per process (the build cost is real — run the crate's bench for the non-canonical number); use DiskAnnIndex for a persistent index that survives restart with no rebuild. (5) HONEST DiskANN scope: DiskAnnIndex is a Vamana on-disk graph that, by default, searches full-precision vectors straight from the mmap; the PQ-compressed in-RAM candidate cache that full DiskANN ranks on (quant.rs) exists, is tested, and IS now wired into search_slots (sq-qamd / #620): build with DiskAnnIndex::build_with_pq (then has_pq_cache() is true) and search_slots dispatches to search_slots_pq — rank the beam on the RAM-only PQ codes, re-rank the survivors off the mmap; the manual PQ filter + full-precision re-rank loop (see recipe) is the same loop the index now drives internally. Without a PQ cache, search stays full-precision-from-mmap. (6) Format is little-endian only (.spqv/.spqg reject big-endian targets); all-zero vectors are rejected at put and a zero query returns no results. (7) Vamana build is single-threaded (slower than the rayon HNSW build) — only the open is cheap. Measured recall@10: HNSW ~0.998, DiskANN ~0.966, PQ-alone ~0.60 / PQ+re-rank ~0.98 at 8×.)
• geosparql — Use when adding GeoSPARQL spatial support to the sparq RDF/SPARQL engine: parsing geo:wktLiteral geometries, calling geof: functions (distance, sf*/eh*/rcc8* DE-9IM relations, envelope/boundary/convexHull/buffer, intersection/union/difference/symDifference, getSRID) inside SPARQL FILTER/BIND/SELECT via the sparq-engine extension-function registry, or building an R-tree GeoIndex over a Graph for within_distance / nearest / intersects queries. Crate: sparq-geo. (status: GeoSPARQL 1.0/1.1 CORE only, all current and verified against crates/sparq-geo source on the main branch (sparq v0.1.0; geo 0.33 / wkt 0.14 / rstar 0.12). Honest limitations to convey to users: (1) Relations are PLANAR DE-9IM in coordinate/degree space, not geodesic. (2) geof:distance metric units use exact haversine when either operand is a point, but a LOCAL EQUIRECTANGULAR APPROXIMATION between two extended geometries (and geof:buffer in metres is likewise local-equirectangular) — accurate at local scale, distorted at continental scale / near poles. (3) Set ops are exact for polygon-vs-polygon (geo BooleanOps) and the well-defined line/point cases; 1-D set SUBTRACTION (line-line / line-polygon difference & symDifference) is an honest GeoError::Unsupported (per-row SPARQL expression error), NOT a wrong answer. (4) GeoIndex only indexes GEOGRAPHIC-CRS (CRS84/EPSG:4326), non-empty wktLiterals reachable via geo:asWKT (optionally owned through geo:hasGeometry / geo:hasDefaultGeometry); other CRSs, empties, non-wktLiteral objects, and unparseable WKT are counted in skipped() and excluded. (5) No GML literals, no RIF/geor: query rewriting, no serialization to .hdt. (6) reproject covers only a small curated EPSG table (27700, 3857, 2154, 25832/25833, UTM 326xx/327xx). geof_registry holds 35 functions and is cheap-clone + Send+Sync (build once, reuse across threads).)
• streaming-rsp — Use when running continuous/standing SPARQL over a live RDF triple stream with the sparq engine — sliding/tumbling time windows (RANGE/STEP), count (ROWS) windows, RSTREAM/ISTREAM/DSTREAM output, RSP-QL surface syntax (REGISTER STREAM, FROM NAMED WINDOW ... ON ... RANGE/STEP), and multi-window joins (WINDOW {} JOIN WINDOW {}). Covers the sparq-rsp crate's ContinuousQuery / ContinuousConstruct / ContinuousAsk / ContinuousMultiQuery / RspqlQuery / WindowSpec. (status: Stable core (33 integration tests + 3 doctests pin window-boundary inclusivity, sliding overlap, t0 origin, empty-window reporting, lateness/late-drop, ROWS/SLIDE, ISTREAM/DSTREAM multiset diffs, three-mode EvalMode equivalence, PersistentDict compaction, CONSTRUCT/ASK, register-time validation). Crate is unpublished (workspace version 0.1.0); depend by path. Honest limits: the RSP-QL surface parser is a deliberately small subset (no WINDOW variables, no ROWS in surface grammar, no NOW-relative bounds, ISO-8601 durations are seconds-resolution and reject Y/M/W). ContinuousMultiQuery supports time windows only and RSTREAM only — multi-window ISTREAM/DSTREAM is a documented follow-up in the crate's open beads. EvalMode::Delta is kept but never wins the crate's own benchmark. All streaming code is marked [OPUS-4.8] pending re-review.)
• zk-query-proofs — Prove and verify a SPARQL query result over committed RDF Verifiable Credentials in zero knowledge with sparq-zk + sparq-zk-compose — per-graph Poseidon2 commitments, BGP scan + integer FILTER Noir proofs, issuer Schnorr attestation (incl. hidden-key set membership), status-list revocation (clear- or hidden-index), verifier nonces / single-use replay defence, and the ProofManifest/circuit family. Use when building or driving the zk-query-proofs surface (proving a query answer, verifying a manifest, attesting an issuer, checking revocation). Requires the Noir toolchain (nargo + bb). (status: EXPERIMENTAL / research-stage (v1), authored by Opus 4.8 while Fable unavailable — flagged for ZK re-review. Honest scope, verified against source: (1) Only EntailmentRegime::Simple is proved (no in-circuit RDFS/OWL reasoning; Rdfs/Owl are schema placeholders). (2) Proved query fragment = BGP triple-pattern SCAN proofs (commitment recompute + scan completeness, the differentiator over soundness-only prior art) and hidden-operand numeric FILTER over xsd:integer (filter_int); negative xsd:integer (filter_signed_int) and fixed-point xsd:decimal (filter_decimal) are also manifest-composable, and the integer-valued xsd:double fragment composes (filter_f64) — only the general fractional/scientific xsd:double form (in-circuit decimal→IEEE printing) remains deferred. (3) GENERAL multi-pattern BGP JOINs are NOT proved in one circuit; cross-pattern join consistency is verifier-side over disclosed rows + the Q6 cross-graph bnode-join guard — but a single hidden cross-credential equi-JOIN IS proved in-circuit (join_eq, single-prover, the join term stays private). No aggregation. (4) Compiled circuit members are fixed buckets only: scan k∈{1,2}, n∈{16,64}, r∈{4,8} (all eight (k,n,r) combinations compiled); filter_int_d∈{1,2,3,4}, filter_f64_d∈{1,2,3,4}, filter_signed_int_d∈{2,4} + filter_decimal_i3_f2; join_eq n_a,n_b∈{16,64} (all four combinations); revoke_unset_d10 (≤1024 status indices); hidden_issuer_d4 (≤16 issuers); holder_pok / holder_set_d4 (hidden-holder PoK / set membership). A proof shape outside a compiled member yields None from build_scan/build_join/derive*id. (5) Issuer attestation is verified verifier-side in the CLEAR by default (reveals WHICH issuer signed). The hidden-issuer (set-membership) and hidden-index revocation circuits are ADDITIVE opt-in privacy layers (KeySet::with_hidden_issuer_depth / RevocationPolicy::with_hidden_index_depth); the clear path always still runs. (6) RevocationStatus.index is disclosed in the clear (linkability channel) unless the hidden-index proof is used. (7) Subprocess proving via nargo/bb (no embedded prover); concurrent provers against the SAME compiled member MUST use the tagged entry points (prove_in/gen_witness_tagged) or they race on shared Prover.toml/witness files. (8) The status-list bitstring and trusted issuer key-set are EXTERNAL relying-party trust anchors (RevocationPolicy.with_snapshot / KeySet::from_hex_keys), never taken from the prover's manifest. (9) Use FileSeenNonces (durable, flock+fsync, single-host) in production; InMemorySeenNonces is test-only. (10) Toolchain pinned to nargo 1.0.0-beta.21 + bb 5.0.0-nightly.20260324, bb target noir-recursive — other versions may change the public-input byte layout the verifier reconstructs against. Both crates are publish=false, non-default workspace members; nothing in sparq depends on them and the wasm artifact is byte-identical with or without them.)
• verifiable-credentials — Verify and sign RDF graphs with W3C Data Integrity (eddsa-rdfc-2022: standard Ed25519 over RDFC-1.0 + SHA-256) and resolve did:key/did:web to a verification key, using the opt-in sparq-vc crate. Use when you trust the SIGNER's key but need tamper-evidence / non-repudiation over an RDF answer (audit trails, data marketplaces, journalistic provenance, agent-to-agent exchange) — the standards-interop, vetted-crypto complement to the in-circuit ZK estate. Covers sign/verify (sign_graph/verify_graph), ProofConfig (verificationMethod/purpose/created/domain/challenge binding), DidKeyResolver, and the pluggable did:web DidDocumentFetcher. (status: NEW, [OPUS-4.8] sq-ylbrq / issue #908; verified against crates/sparq-vc source (14 tests + 1 doctest: did:key round-trip, a published W3C did:key vector decoded + re-encoded byte-identically, non-Ed25519 multicodec rejected, sign→verify round-trip, isomorphism-stability, tamper / wrong-key / wrong-config / wrong-cryptosuite all fail closed, the real store path via Graph::load_str, and the did:web in-memory-fetcher path under the did-web feature). HONEST BOUNDARY (load-bearing): eddsa-rdfc-2022 is authenticity + integrity + non-repudiation ONLY — NOT confidentiality, NOT zero-knowledge, NOT selective disclosure; it reveals the full signed content to the verifier and this crate makes NO privacy claim (unlinkable/selective-disclosure presentation is the ZK estate's job, or a later bbs-2023 phase). did:key is self-certifying (the DID IS the key) — a stable identifier, NOT a trust anchor; did:web's root of trust is whoever controls the host + its TLS. v1 operates over the RDF-dataset form of a credential + its proof config (exactly what RDFC-1.0 canonicalizes) — JSON-LD→RDF context expansion is the caller's job, to keep the lean build free of a JSON-LD context processor. Opt-in standalone crate (publish=false): nothing in sparq's default build or wasm artifact depends on it; did:web behind the off-by-default did-web feature, and the crate ships no HTTP client (the fetch is a pluggable seam). Distinct from sparq-trust::did, which resolves to a sparq-private Baby-JubJub key for the ZK scheme and rejects standard Ed25519 did:key — this is the W3C-interoperable resolver.)
• genai-retrieval — Use when an LLM/agent needs to answer natural-language questions over a sparq RDF Graph with SPARQL, or to build token-budgeted retrieval/grounding context (schema card, VoID, characteristic-set join hints) about an unknown RDF dataset. Covers the sparq-nlq NL→SPARQL loop (ground→generate→validate→execute→repair, offline record/replay, optional live Anthropic backend) and sparq-introspect (schema/VoID/seed-scoped summaries + planner join hints). (status: Both crates are opt-in and at workspace v0.1.0; verified against the source on branch main (2026-06-19 [OPUS-4.8]). sparq-introspect is the more mature surface (22 in-crate unit tests + 1 integration test under tests/olympics.rs; run its benches for the non-canonical scan latency rather than baking a number here). sparq-nlq's ENGINE-SIDE loop is validated end-to-end (grounding, spargebra validation, budgeted execution, repair plumbing, record/replay) against the real olympics dataset with hand-written fixtures; for the LLM-excluded ask() latency run the crate's bench (non-canonical). The design-doc accuracy gate (QALD-style answer-F1, oracle-vs-end-to-end and grounded-vs-ungrounded reported separately, with the "grounding pays for itself" inequality) has LANDED as the offline sparq_nlq::eval harness (sq-05rv), proven deterministically on scripted/recorded completions. NOT yet validated: live-model exec-accuracy NUMBERS — the committed completions are hand-written (as a competent model would answer given the grounding prompt); the real-model measurement runs through the same harness with --features live + RecordingLlm and is #[ignore]'d OFF the gate. Entity/relation linking from sparq-sim IS now wired (sq-uw40 / #647): the EntityLinker expands each linked entity with its sparq_sim::Sim::most_similar structural siblings during grounding. Not yet wired: grammar-constrained decoding. AnthropicLlm (live feature) is compile-checked in CI but never called there; no test touches the network. to_void omits void:distinctObjects by design (no global de-duplicated object count is tracked).)
• agent-tools — Use when an LLM/agent should access a sparq RDF dataset over the Model Context Protocol (MCP) as first-class tools. Covers the opt-in sparq-mcp crate: a JSON-RPC 2.0 MCP server (initialize → tools/list → tools/call) exposing query (SELECT/ASK → SPARQL-JSON), construct (CONSTRUCT/DESCRIBE → N-Triples), introspect (effective schema as JSON or token-budgeted text), stats, prefixes (namespace declarations + term counts), void (W3C VoID N-Triples), and a gated update tool that is OFF by default; the transport-agnostic handle_message dispatch core plus the optional stdio feature; and the honest trust model (local agent-tool server, no built-in auth, read-only by default, queries bounded by a QueryBudget). (status: opt-in crate at workspace v0.1.0; prefixes verified against the source on branch main (2026-07-13 [GPT-5.6], sq-kx5b0); void verified 2026-07-12 [GPT-5.6], sq-2kkym. It is a THIN wrapper — query/construct go through the real sparq-engine query path, introspect/stats/prefixes/void through the real sparq-introspect miner; no new engine logic. Tested by a real in-memory MCP round-trip (default features) AND a real stdio serve-loop round-trip (feature stdio). The update tool is neither advertised nor callable unless ServerConfig::allow_update is set; a read-only server is proven fail-closed (a disabled update call returns -32601 and does not mutate). NO authentication/authorization/sandboxing is built in: the MCP transport is the trust boundary. Optional MCP transports beyond stdio (SSE/HTTP) are NOT implemented (follow-up beads); only stdio + embeddable handle_message ship today.)
• structural-similarity — Training-free structural entity similarity over a sparq RDF graph with the opt-in sparq-sim crate: build an entity's (direction, predicate, neighbor) signature straight from the store's permutation indexes, score predicate-IDF-weighted Jaccard between two entities, and retrieve the top-k most-similar entities (most_similar) via index-driven candidate generation — no embeddings, no model, no training, correct under incremental updates. Covers Sim/SimConfig/SignatureMode, the two similarity notions (shared-context PredicateNeighbor vs predicate-profile/role Predicates), and hybrid (structural + text-vector) fusion with sparq-vectors. Use for co-citation/shared-context similarity, role similarity, entity/relation linking. (status: Verified against crates/sparq-sim source + tests on 2026-06-16. Opt-in (GenAI phase 1), workspace v0.1.0, #![forbid(unsafe_code)], no SPARQL-level integration (call the Rust API over a Graph). The two AUCs differ by design — PredicateNeighbor measures shared context (most same-class pairs tie at 0), Predicates measures role/class separation; the most_similar ranking task is high-precision@10. max_pair_frequency is an approximation on candidate generation only — never on a returned score. Quality/latency gates live in the olympics_eval example (non-canonical numbers). [OPUS-4.8] pending re-review.)
• graph-analytics — Graph analytics over a sparq RDF graph with the opt-in sparq-algos crate: project the graph onto a directed node view (NodeGraph) and run PageRank, centrality, community detection, default-OFF directed SCC, or canonical topological sorting — read directly from sparq-core's permutation indexes, deterministic, no model, no network. Use to rank entities, find clusters/communities, classify directed cycles, or order a DAG. (status: Directed topology added by [GPT-5.6] for sq-lsp7k.20 behind default-OFF topology; iterative Tarjan SCC + Kahn topological sorting. Extended centrality from sq-lsp7k.15 remains behind centrality-extended, exact and unweighted. No SPARQL-level integration. HONEST scope: topology-only — predicates erased, parallel edges collapsed, edges unweighted; SCC/topological sort preserve direction, while extended centrality uses weak topology. Literals are not nodes by default, but subjects whose only objects are literals remain isolated entity nodes. PageRank redistributes dangling mass; WCC is exact; label propagation is heuristic. In-memory CSR view.)
• access-control — Graph-level Solid-style access control over a sparq RDF dataset with the opt-in sparq-solid crate: pods as named-graph-per-document, WAC (.acl) / ACP (.acr) documents kept as queryable triples, their semantics materialized to a queryable authorization view (<urn:sparq:auth>) by N3 rules (sparq-reason), then every query/update filtered per (WebID, client) session to the authorized graph set — fail-closed. Covers PodStore (materialize_wac/acp, query_as/update_as, the zero-copy DatasetView path + the FROM NAMED rewrite portability path), Session/Mode, and the WAC/ACP support matrix. Use to gate which named graphs a session may read/append/write. (status: RESEARCH / architecture track. IMPORTANT honesty: this is the authorization LAYER only — it does NOT authenticate (the Session.agent WebID is caller-asserted, no WebID-OIDC/token verification) and it is NOT a Solid Pod HTTP server (no HTTP/LDP/OIDC resource protocol). Shipped + tested (zero-copy view path, write-gating, auto-re-materialize on .acl/.acr writes); fail-closed (absent grant ⇒ graph invisible; reasoner is fed only ACL/ACR + structural facts, never pod content; urn:sparq: rejected on input). Verified against crates/sparq-solid source + research/solid-access-control-design.md on 2026-06-16. Opt-in, depends on nothing in the workspace. [OPUS-4.8] pending re-review.)
• usage-control-policy — Evaluate W3C ODRL 2.2 usage-control policies over RDF with the opt-in sparq-policy crate: parse an ODRL Set/Offer/Agreement into a typed Policy of Permission/Prohibition rules (action, target, assignee, Constraint, Duty), then evaluate an access Request to a fail-closed ALLOW/DENY Decision. Use when gating a query/asset by purpose, recipient, time window, count, or a duty obligation; when mapping ODRL to the sparq-solid WAC/ACP allow-deny model; or when wiring usage control above access control. (status: single-node base case; the federated ODRL→MPC disclosure split is deferred and inherits the MPC honest-majority/LAN envelope + the open ZK-soundness remediation. See research/feature-research-odrl-policy.md.)
• federated-planning — Cost-based federated SPARQL source selection + bind-vs-hash join planning over already-fetched source descriptors, plus an ANAPSID-style non-blocking streaming join with operator spill, via the opt-in sparq-fedplan crate. Use when planning a federated BGP across multiple SPARQL endpoints from their served statistics (VoID property/class partitions + mined scs: characteristic sets) — HiBISCuS recall-safe source pruning, CostFed skew-aware cardinality, per-join bind-vs-hash-vs-streaming selection, a memory-bounded symmetric hash join (StreamJoin, spill to a backing store), and live adaptive RE-planning at stage boundaries (opt-in adaptive-replan feature). Pure + deterministic planning, no network I/O; off by default. (status: opt-in, does not touch the lean sparq-core/sparq-engine build.)
• gpu-kernels — Experimental GPU (wgpu/WGSL) compute kernels for sparq's hot-path primitive shapes — FILTER + count, hash-join probe, GROUP BY COUNT+SUM — over plain ValueId-shaped columns (&[u32]/&[f64]) in the opt-in sparq-gpu crate. Covers the Gpu handle (Option from Gpu::new() — no adapter ⇒ None), resident ColU32/ColF64/HashTable types, and the four kernels (incl. the no-f64 IEEE-754 bit-order comparison trick). Use to measure whether a GPU beats the CPU for sparq's columnar primitives once the host→device transfer tax is charged. (status: EXPERIMENTAL measurement prototype (roadmap T24d), publish = false, PARKED per research/gpu-verdict.md (GPU only wins the hash-probe shape on M1 unified memory; one winning kernel class does not pay for a residency cache + scheduler + second backend). NOT wired into the engine and nothing depends on it; no sparq-core dep; wgpu never enters the wasm build. group_aggregate caps at 512 groups; hash tables built host-side, only the probe offloaded; f64 arithmetic (vs comparison) unsupported. Verified against crates/sparq-gpu source on 2026-06-16; bench numbers non-canonical (in the verdict).)
• substrate — The opt-in shared zero-overhead evaluation substrate for the sparq engine and the reasoners (crate: sparq-substrate): the id-tuple row/key/posting vocabulary (rows), the XSD numeric value tower with EXACT integer/decimal arithmetic (numeric), the four id-tuple join kernels — sorted merge, hash build+probe, bind-combine, leapfrog trie-join/WCOJ (join), and the SPARQL term total order generic over a CompareTerm trait (compare). All features DEFAULT-OFF; opt in only to the slice your consumer needs. Use when wiring a new sparq reasoner that shares joins or numeric ops with the engine. (status: publishable (sq-qonbz.4 [SONNET-4.6]). All four modules behaviour-neutral vs the pre-move engine baseline. Verified against source on 2026-07-03.)
• mpc — Run a SPARQL query across multiple mutually-distrusting RDF data holders (federated MPC): per-holder local sub-evaluation, crypto-free disclosed-key global-IRI joins, and an honest-majority Shamir secret-sharing backend for secure cumulative aggregates + hidden-value (private-key) joins. EARLY/RESEARCH, native-only; honest-majority, tamper-detecting reconstruction (detect-and-abort; robust correction where redundancy allows), semi-honest the floor where no redundancy exists (degree-2t equality open at n=2t+1 is detection-only) — NOT dishonest-majority / not full malicious security; collaborative ZK proof of correctness/attestation is still a stub. (status: EARLY / RESEARCH (milestones M0-M3 of the RQ2 plan). Native-only: sparq-mpc is deliberately NOT a dependency of sparq-wasm (no browser/MPC-crypto story yet). What is REAL and tested today (40 unit tests pass): (1) per-holder local SPARQL sub-evaluation (Holder); (2) the crypto-free disclosed-key global-IRI equi-join (DisclosedKeyJoin) with a differential test vs union-store evaluation; (3) an honest-majority, SEMI-HONEST Shamir t-of-n backend over F_p = 2^61-1 (ShamirBackend) computing the cumulative-sum aggregate (free local addition) and the secret-shared equality-to-zero primitive; (4) the hidden-value (private-key) join (HiddenValueJoin) built on that equality test. Honest limitations, stated in-code, NOT papered over: malicious-security is now SURFACED via BackendInfo.malicious_security (a MaliciousSecurity enum): the RS-consistency-checked reconstruction detects/robustly-corrects tampered shares at degree t (always redundant for the honest-majority t), but the degree-2t equality open in the hidden-value join is NOT hardened at n=2t+1 (needs an info-theoretic MAC, deferred) and dishonest-majority is future work; the hidden-value join is naive O(|L|x|R|) all-pairs (no cuckoo/oblivious-hashing PSI optimisation), feasibility envelope is minutes-not-seconds for ~10^3-10^4 rows/holder on a LAN; the multi-party protocol runs as an IN-PROCESS SIMULATION (one process plays all parties) -- there is no real network transport; the field key encoding for hidden joins is the caller's responsibility and must be injective. STUBBED / returns MpcError::NotYetImplemented with the gating milestone named (no fake crypto): the collaborative ZK proof of correctness + issuer-attestation (CollaborativeProof, Attestation, ProofStatement) -- gated on the single-prover ZK soundness remediation (#3/#4/#5/#6/#8/#9/#12) and the open Q1 distributed-signature-over-secret-shared-witness research spike (M4). new_seeded / InsecureTestRng are behind cfg(test)+feature=insecure-test-rng and produce PREDICTABLE masks (no security) -- never enable that feature in a deployment.)
• academic-paper — The sparq paper-factory PROCESS (epic sq-gum8): turn a sparq contribution into an academic paper — identify a genuinely-novel contribution, classify it and pick a venue (ISWC/ESWC Resource for the engine, PVLDB/SIGMOD/EDBT for DB-systems, arXiv/workshop for not-yet-sound ZK/MPC), draft a single-source Typst (.typ) paper whose eval section binds to live benchmark data, build both a PDF and an in-site HTML page from that one source, and review claims↔evidence under sparq's empirical-honesty mandate (canonical vs indicative numbers; the ZK/MPC not-yet-sound disclaimer). (status: factory machinery, not an engine capability; design record research/paper-factory-design.md.)

Discoverability

These follow the Agent Skills SKILL.md convention (YAML frontmatter name/description). Any agent that supports the standard can load a skill directly from a clone of this repo, or you can copy/symlink one into ~/.claude/skills/. They are also packaged as a Claude Code plugin (see .claude-plugin/): /plugin marketplace add jeswr/sparq then /plugin install sparq@sparq-tools.

Maintenance

When you change a public API, update the matching skills/<surface>/SKILL.md in the same change. See AGENTS.md for the rule and the TODO/beads policy.

Writing or editing docs? Use terminology — the authoritative RDF/SPARQL terminology guide that keeps every surface consistent with the W3C specs (say RDF 1.2 / SPARQL 1.2 and triple term / reifier / reified triple, never the community-era "RDF-star" / "quoted triple"). It is enforced by the terminology HARD gate in CI.