gain-2026

Orchestrate a structured-records investigation against a large data corpus (XML, JSON, JSONL, CSV) — lobbying filings, FEC reports, court records, IRS 990s, anything tabular or nested where the story lives in relationships. Mirrors the Spotlight orchestration pattern (brief, methodology, execution cycles, adversarial fact-check, gates, handoff) but geared for structured data instead of web OSINT. Pushes ETL, entity resolution, anomaly detection, and aggregation to deterministic Python + DuckDB; reserves the LLM for reasoning over already-filtered candidates. Composes with the nine sibling sub-skills (ingest, resolve, detect, evidence-cards, external-data, similarity-search, report-drafting, spotlight-handoff, vault-ingest). Reuses the spotlight investigator and fact-checker subagents. Triggers on prompts mentioning data-driven investigation, lobbying records, LDA, FEC, FARA, corporate filings, large XML or JSON corpora, structured-records journalism, or finding a story in this dataset.

Pull external public datasets into the same DuckDB index used by data-detective so cross-corpus joins can happen in-database. Current pullers (extensible pattern) — FARA bulk data via the OpenSanctions DOJ mirror (Foreign Agents Registration Act registrants and foreign principals); Congress.gov committee jurisdiction graph via the open unitedstates/congress-legislators dataset (current Congress members, committees, assignments); USAspending.gov DoD contract awards via the federal USAspending API. Use this sub-skill in Phase 3 of a data-detective investigation when an anomaly requires external corroboration. Triggers on prompts mentioning FARA data, Congress.gov committee assignments, USAspending DoD contracts, external data joins, or pulling a public dataset into the index.

Phase 5 synthesis sub-skill for data-detective — draft the journalist-grade deliverables editors actually read: findings-report.md (audit narrative), report.html (public-facing artifact, built from the report-template.html skeleton), and evidence-map.json (machine-readable ledger), with per-finding replication paths + sources strips, confidence/novelty pills, phase-by-phase methodology, and a hard citation-provenance rule with closure script. Use at Phase 5 after Gate 1, before Phase 6 spotlight-handoff. Triggers on draft the report, build the HTML report, journalist-grade output, synthesis phase, findings report, evidence map.

Archive the final data-detective findings into an Obsidian vault (or generic directory) as structured knowledge — one finding note per claim, one entity note per named person/organization/agency, one source note per cited filing/court case/external archive, plus a methodology note describing the data-detective pipeline that produced them. Wikilinks the entire graph. Honors the spotlight:ingest --sensitive flag for source-redacted notes. Limited by design to what is contained in the final report (findings-report.md + evidence-map.json) — not the entire case-trace. Use after Gate 1 approval at Phase 7 of data-detective, or standalone to ingest a single finished report. Triggers on prompts mentioning ingest findings to obsidian, archive case to vault, write findings into the knowledge base, structured-records evidence to obsidian, or sensitive flag for case archive.

Run a deterministic anomaly-detector battery and ad-hoc SQL against a DuckDB index of structured records, with SHA-tagged provenance per query. Includes a catalog of 12+ pre-built detectors (spending spikes, missing-data clusters, revolving-door candidates, foreign filings, single-client juggernauts, PAC contribution flow, issue concentration shifts, new-registrant surges, shell-pattern outliers, FARA-gap narrowing, multi-hop committee-jurisdiction match, committee say-vs-pay). Every query writes a CSV plus a provenance.json (SQL hash, schema, row count, timestamp) so claims can cite a specific query. Use this sub-skill at Phase 3 of a data-detective investigation. Triggers on prompts mentioning anomaly detection, lobbying detectors, revolving door SQL, FARA gap analysis, structured-records anomalies, or "what's weird in this database."

Generate one-page markdown evidence cards from records in a DuckDB index of structured records. Each card is a single source record (filing UUID, House XML filename, press release URL hash, FARA entity ID, etc.) rendered in full, with the canonical public document URL linked at the top — the audit surface an editor uses to verify a claim in under a minute. Used by the data-detective orchestrator at Phase 3 and Phase 5 (synthesis). Triggers on prompts about evidence cards, audit cards, claim-to-source links, "render this filing as markdown," or "give me a one-page summary of this record."

Profile-driven ETL from a structured-records corpus (XML, JSON, JSONL) into a single DuckDB index with deterministic schemas, bridge-key columns preserved, and a manifest of source-file SHA-256 hashes for reproducibility. Use this sub-skill at Phase 1 of a data-detective investigation, or standalone to build a queryable index over any corpus that fits the profile contract (file globs → mapper functions). Used by the data-detective orchestrator. Triggers on prompts mentioning corpus ingestion, building a DuckDB index, ETL from XML/JSON to a database, lobbying-records-to-database, or any "I have a folder of structured files and want to query them as a single database."

Three-pass entity resolution for a structured-records corpus — bridge keys first, exact normalized name second, fuzzy match third. Builds a canonical_registrants view in the DuckDB index that joins across House and Senate filings (or any two source systems with overlapping entity references). Designed to be deterministic where possible; only falls back to fuzzy matching for the residual. Use this sub-skill after data-detective-ingest to deduplicate and cross-link entities. Triggers on prompts mentioning entity resolution, deduplicating names across data sources, joining House LDA to Senate LDA, fuzzy matching with rapidfuzz, or any "match entity X across two databases."

Vector similarity search over any free-text column in the data-detective DuckDB index using fastembed (ONNX, no torch) and DuckDB's VSS extension. Use when keyword detectors can't surface an anomaly — e.g. "find more filings that smell like LOC Nation" — by embedding the activity descriptions or press release texts and asking for nearest neighbors of a known anomaly. The skill ships ready-to-use as a complement to keyword-based detectors; not always needed because keyword detectors are cheaper and more auditable, but valuable when the anomaly is a semantic pattern rather than a string match. Use in Phase 3 of a data-detective investigation when a keyword detector fails to surface a suspected pattern. Triggers on "find similar records to X," "nearest neighbor search," "embed activity descriptions," or "discover anomalies by semantic similarity."

The handoff phase of a data-detective investigation — translate finalized data-side findings into Spotlight-ready briefs and pass leads to the spotlight orchestrator for external OSINT amplification. Each candidate finding becomes a spotlight-brief.md with named entities, URLs already gathered, what data-detective established (with evidence card refs), and suggested external angles. The skill also generates a spotlight-handoff.md ranking sheet so the user can pick which leads warrant Spotlight's web-based investigation. Use in Phase 6 of a data-detective investigation, after Gate 1 approval. Triggers on prompts mentioning passover, handoff to spotlight, generate OSINT briefs, what should we research externally, or "take these findings and dig deeper online."