scix-agent

Build, test, and CI runbook for SciX Experiments: recreate the venv and the optional-extras matrix (dev/qdrant/graph/mcp/embed/ner_pass/viz/...), run make check vs make check-ci, understand the "not integration and not network" marker filter, the optional-dependency collection guards in tests/conftest.py, the ci/scix_test_schema.sql snapshot the CI Postgres loads, the MCP contract-drift test, and the pinned black/ruff formatters. Load when setting up a fresh checkout, when CI is red and local is green (or vice versa), when tests skip silently or whole test modules vanish, when pre-commit or fmt-check fails, or when test_mcp_contract_conformance fails. NOT for prod-DSN/write-test safety semantics — use scix-db-safety-and-telemetry. NOT for changing the MCP tool surface itself — use scix-mcp-tool-surface. NOT for running heavy jobs on this host — use scix-memory-and-batch-discipline. NOT for what the project is — use scix-orientation.

How change is gated in SciX: which changes require an ADR (retrieval/vector/storage/dimensionality/quantization axes), the 15-tool MCP cap and contract regeneration, migration discipline (append-only, no auto-runner, the hand-applied 069-072 gap), which changes HALT at branch-ready for sign-off, where project truth lives (beads/ADRs, not git history), and what is intentionally retired vs parked. Also documents the Gas City bead/dispatch machinery (internal-orchestration). Load BEFORE proposing or landing any change to retrieval, vectors, storage, the tool surface, the schema, or prod data, or before re-landing anything from an old branch. NOT for how the retrieval stack works (scix-retrieval-architecture), DSN/prod-DB guards (scix-db-safety-and-telemetry), running heavy jobs safely (scix-memory-and-batch-discipline), or tool-surface internals (scix-mcp-tool-surface).

The SciX citation-graph layer: graph_metrics.py (PageRank/HITS/Leiden), the paper_metrics and communities tables, the (signal, resolution, community_id) model, the 299M-row citation_edges table, and the MCP citation/provenance tools (citation_traverse, forward_citations incl. the find_replications/cited_by_intent aliases, claim_blame, graph_context) with citation-intent classification. Load when computing or debugging PageRank, HITS, Leiden communities, community labels, citation traversal, claim provenance, replication finding, or citation intent. NOT for the RRF search stack (use scix-retrieval-architecture), NOT for the Qdrant dense lane or embeddings ingest (use scix-vector-serving-qdrant / scix-embedding-pipeline), NOT for MCP tool-cap/contract mechanics (use scix-mcp-tool-surface), NOT for the entity graph (use scix-entity-ner-system), NOT for query-side agent usage of the tools (use the existing scix-mcp skill).

Production-database protection and query_log telemetry for SciX. Load this BEFORE any command that opens a PostgreSQL connection in this repo: running pytest, setting SCIX_DSN / SCIX_TEST_DSN, calling is_production_dsn, passing --allow-prod to a script, wiping/seeding test tables, or analysing the query_log table (success/error_msg semantics, failure rates, guard-block counts). Triggers: "which database am I pointed at", "tests all passed but wrote nothing", "SCIX_TEST_DSN", "--allow-prod refused", "query_log says success", "failure rate by tool". NOT for memory/cgroup sizing or the scix-batch wrapper itself — use scix-memory-and-batch-discipline. NOT for schema-migration or index-build discipline — use scix-index-and-storage-discipline. NOT for what gets ADR-gated — use scix-change-control.

The INDUS paper-embedding ingest path: src/scix/embed.py, scripts/embed.py, daily_sync.sh Step 5, the paper_embeddings table and its PG-to-Qdrant outbox sync, and the s7cy live fire (paper_embeddings was DROP TABLEd out-of-process; committed HEAD embed code targets the dropped table; ~83K papers have no dense vector). Load when embedding new papers, debugging "relation paper_embeddings does not exist" / daily_sync aborts, backfilling the dense-vector gap, touching any embed/outbox/watermark code, or asking which embedding model/pooling/input format SciX uses. NOT for Qdrant collection config, serving reads, or payload indexes — use scix-vector-serving-qdrant. NOT for RRF fusion or lane weighting — use scix-retrieval-architecture. NOT for index DDL / disk reclamation — use scix-index-and-storage-discipline. NOT for DSN guards and prod-DB protection mechanics — use scix-db-safety-and-telemetry. NOT for scix-batch/OOM discipline details — use scix-memory-and-batch-discipline.

Maintainer guide to the SciX cross-discipline entity graph and NER system: the entities / document_entities schema (57.7M paper-entity links), the three extraction lanes (LLM batch extraction, lexical linking, the GLiNER dbl.3 zero-shot pass), the INDUS post-classifier, the per-bucket precision profile and lower-bound honesty discipline, the eq95 denylist, the JIT resolver lane (bulkhead / canary / stubs), and the ZFC rule (classify via the entity graph, never keyword heuristics). Load when working on src/scix/extract/, src/scix/jit/, entity_resolver, resolve_entities, document_entities, NER precision, GLiNER, ner_quality_profile, or the entity MCP tool's internals. NOT for using the MCP tools as a literature researcher — use scix-mcp. NOT for the MCP tool cap / contract regen — use scix-mcp-tool-surface. NOT for citation communities / PageRank — use scix-citation-graph. NOT for embedding ingest — use scix-embedding-pipeline.

How retrieval and extraction quality is measured and what counts as evidence in SciX: the gold sets (50q curated, 1200q recall gold, claim extraction, lexical stress), the fusion-calibration sweep and its honest verdict (dense_only < bm25_only), nDCG@10 / Recall@K / MRR, Wilson 95% CIs, the OAuth persona/UMBRELA judges, the claim_blame gold-set plan (bead 6ajy), and the reporting rules (read-only harness, lane provenance, per-bucket numbers, null results stated plainly). Load when running or interpreting an eval, adding a gold set, judging relevance, choosing an acceptance threshold, or writing up a result. NOT for RRF fusion internals (scix-retrieval-architecture), Qdrant mechanics (scix-vector-serving-qdrant), CI/pytest (scix-build-test-ci), whether a change needs an ADR (scix-change-control), or query_log telemetry (scix-db-safety-and-telemetry).

How index and storage changes are made safely in SciX. Load BEFORE building, rebuilding, dropping, or benchmarking any Postgres index (HNSW, GIN, DiskANN, expression indexes); before any halfvec/quantization change; before reclaiming disk (DROP INDEX vs DELETE vs DROP TABLE, VACUUM FULL, pg_repack); before placing new data on DS (local NVMe) vs NAS (/mnt, NFS); and when working the ADR-015/016 reclamation line or the dqfe quantization spike. Covers the four don't-trust-a-new-index rules (bead 12rp, the 56-hour DiskANN loss), the halfvec shadow-column cutover and its planner traps, the disk-at-99% crisis, and never-live-write-on-NAS. NOT for Qdrant serving (scix-vector-serving-qdrant), the embed pipeline (scix-embedding-pipeline), DSN guards (scix-db-safety-and-telemetry), or change approval (scix-change-control).

Maintainer guide to the SciX MCP tool surface — the 15-tool visible cap and its three enforcement layers, deprecated-alias routing via _ALIAS_TRANSFORMS, the published contract (contract/scix_mcp_v1.json) and the gen_mcp_contract.py regen workflow, the closed error_code catalog, and the add/consolidate-a-tool checklist. Load when adding, renaming, merging, hiding, or retiring an MCP tool; when the contract conformance test or the import-time cap guard fails; when touching mcp_server.py, mcp_tool_specs.py, mcp_handlers/, mcp_errors.py, or mcp_contract.py; or when deciding whether a new capability deserves a new tool. NOT for using the tools to do literature research — use the scix-mcp skill (query-side). NOT for search/RRF internals — use scix-retrieval-architecture. NOT for query_log analysis — use scix-db-safety-and-telemetry. NOT for CI mechanics — use scix-build-test-ci.

Load BEFORE running any heavy or long-running command in this repo on the production host: embedding/ingest passes, NER (GLiNER) runs, chunk passes, index or graph recomputes, eval/fusion sweeps, backfills, big-table migrations, model training, or anything multi-GB or over ~1 minute. Covers the non-negotiable scix-batch wrapper, the systemd-oomd co-hosting hazard (an unwrapped job gets the gascity supervisor killed as collateral), MemoryHigh/MemoryMax sizing, the two self-enforcement guards (--require-batch-scope/SYSTEMD_SCOPE and --allow-prod/INVOCATION_ID), the SYSTEMD_SCOPE-is-not-auto-set trap, and what the scope does NOT bound (docker, Postgres, Qdrant). NOT for DSN/telemetry semantics (scix-db-safety-and-telemetry), index-build validation (scix-index-and-storage-discipline), what the embed scripts do (scix-embedding-pipeline), or test/CI invocation (scix-build-test-ci).

First-load orientation map for the SciX Experiments repo: what SciX is (agent-navigable knowledge layer over the NASA ADS/SciX corpus, NOT a benchmark), corpus scale, the PostgreSQL + Qdrant dual store, the four surfaces (hybrid retrieval, citation graph, entity graph, provenance tools), source layout, the fastest reading route into the codebase, the viz dashboard layer, and the doc-drift warning (which docs to trust and which are stale). Load this when you are new to the repo, when you need to locate where a subsystem lives, or before deciding which sibling skill to load. NOT for running heavy jobs (use scix-memory-and-batch-discipline), NOT for touching any database (use scix-db-safety-and-telemetry FIRST), NOT for retrieval internals (use scix-retrieval-architecture), NOT for querying the corpus as a research user (use the existing scix-mcp skill), NOT for making gated changes (use scix-change-control).

The executable research campaign on SciX's hardest live problem — retrieval-quality integrity ("does the INDUS dense lane earn its place, and does hybrid survive a skeptic?") — plus the other open frontier (claim_blame has no real-corpus accuracy number) and the ADASS external-positioning bar (what may be claimed, reproducibility standards). Load when: planning or running a fusion-calibration sweep, interpreting dense-vs-BM25 nDCG numbers, evaluating a stronger dense lane, deciding what the ADASS paper can claim, building the claim_blame gold set, or asked "is hybrid retrieval actually better here?". NOT for how the RRF/lane code is wired — use scix-retrieval-architecture. NOT for eval-metric mechanics or the gold-set catalog — use scix-eval-and-evidence. NOT for how changes get gated/approved — use scix-change-control. NOT for running the embed/ingest pipeline — use scix-embedding-pipeline.

How a query becomes ranked results in SciX: the RRF hybrid stack in src/scix/search.py — title/abstract BM25 + body BM25 + INDUS dense (Qdrant) fused at RRF k=60, optional alias/ontology lanes and a cross-encoder reranker (off by default), and the honest eval state (dense 0.38 < bm25 0.43 nDCG@10 on the 1200q gold set). Load when reading or changing hybrid_search / lexical_search / vector_search / rrf_fuse, tuning SCIX_LEXICAL_* / SCIX_RERANK_* flags, or checking a change against the ADR-pinned invariants (768d, halfvec, no paid-API lane, Qdrant-not-pgvector serving). NOT for Qdrant ops (scix-vector-serving-qdrant), the embed/ingest pipeline or s7cy fire (scix-embedding-pipeline), running evals (scix-eval-and-evidence), or index builds (scix-index-and-storage-discipline).

The INDUS dense lane's serving substrate: the Qdrant collection scix_indus_v2_papers_s1, the QDRANT_URL gate in vector_search(), the REST-only client pin, the bibcode-only payload + Postgres post-filter over-fetch pattern, SCIX_QDRANT_EXACT, point-id contracts, ADR-013 and why dense ANN does NOT serve from pgvector/pgvectorscale (the DiskANN catastrophe), the ADR-014 sparse lane (parked, not dead), and the ADR-015/016 storage line as it bears on the dense lane. Load when working on Qdrant collections, vector kNN serving, the dense-lane gate, Qdrant payload schema/backfills, the scix-qdrant container, or anything that would put dense ANN back in Postgres. NOT for the RRF fusion stack or lane weighting — use scix-retrieval-architecture. NOT for the embed/ingest pipeline or the s7cy daily-sync fire — use scix-embedding-pipeline. NOT for Postgres index builds, disk reclamation mechanics, or NAS placement rules — use scix-index-and-storage-discipline.

Use the SciX MCP server for scientific literature research across arXiv and the NASA SMD disciplines (astrophysics, planetary science, earth science, heliophysics, and biological & physical sciences as they relate to space science). Provides hybrid semantic+lexical search (INDUS + body BM25 via RRF), citation graph traversal (299M edges, PageRank, communities), entity extraction, and session-aware working sets. Invoke when the user asks about scientific papers, citations, authors, or wants to explore the literature in these domains.