メニュー
Show the provenance trace — every reported number linked to the SQL that produced it, with a confidence badge. Use after an analysis when someone asks "where did that number come from?"
Codex または Claude でインストール この Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。
SOC 職業分類に基づく
Run the held-out gold suite live against the analyst and score it. Use when the user types "/eval", or asks to "run the eval suite", "score the system", "run the train split", "check the test split", "what's our accuracy on the gold cases", or wants the system-level eval number (accuracy + query-similarity + cost). This DRIVES the analyst on each question (it does not grade hand-supplied answers), keeps the gold blind, and writes a self-describing run record. Pairs with /reliability (stability, no key) and /compare (two configs or two models).
Initialize all knowledge subsystems at session start to load active dataset context, user profile, corrections, learnings, query archaeology, and analysis history into working memory. Use this skill automatically at the start of EVERY new session, immediately after the conversation begins. Also trigger after `/connect-data` or `/switch-dataset` to reload dataset context. Apply when the system detects missing or stale knowledge files, when returning to the tool after a break, during session initialization, at the beginning of any conversation, when Claude Code first starts, when a new Claude conversation opens, or at any session boundary. This skill is critical for contextual awareness — it ensures you understand the user's data environment, preferences, and past work before beginning any analysis. Without running this skill at session start, you lack essential context about what dataset is active, what the user prefers, and what corrections have been logged. Always run this skill first, before responding to a
Check whether an AI analysis answer is STABLE by running the same question several independent times and measuring what holds versus what drifts. Use when the user types "/reliability", or asks to "run this a few times", "is this answer stable / reliable", "check reliability of <question>", "does it give the same answer again", "run it N times and compare", or wants to know if a number can be trusted enough to act on. This is the cheapest eval and the only one that needs no answer key: it tells you about stability, not correctness.
Validate data completeness, consistency, and coverage before any analysis begins, flagging issues with severity ratings so the analyst knows what blocks analysis vs. what to note as a caveat. Use this skill at the start of EVERY new analysis — whenever you're about to query data, explore a dataset, design an investigation, or answer an analytical question. Also apply when connecting to a new data source, when results look suspicious or unexpected, when a user mentions data quality concerns, when switching datasets, when profiling a new table, or any time before drawing conclusions from data. Trigger on phrases like "analyze this data", "explore the dataset", "what's in this data", "investigate why", "check data quality", "is the data clean", "profile this table", "validate the data", "run a quality check", "before we analyze", "what's the coverage", or any request that involves querying or analyzing data. **This skill also fires whenever the user asks about a specific named table** — e.g., "tell me about the
Run a comparison: ask one analytics question two ways, with a piece of context and without it, and measure what changed. Use when the user says "/compare", "run it with and without <the definition/context>", "does adding <X> change the answer", "is this context worth it", or wants to see whether a metric definition (or any context) moves the answer and collapses the drift. This is the context-side companion to /reliability: reliability says stable or drifts under one setup; compare says what changed between two setups.
Independently validate the current analysis with a second model (OpenAI Codex). Codex re-derives the same answer from the same data — blind to Claude's SQL and numbers — and the skill reports AGREE / DISAGREE / PARTIAL per finding. Use when the user types "/codex-review", or says "validate with codex", "codex review", "second opinion from codex", "have the other model check this", "independently verify this analysis", "does codex agree", "cross-check this with gpt/codex", or wants a different model to confirm a result before acting on it. This is multi-model validation: a real independent re-analysis, not a critique of Claude's work. If the Codex plugin or CLI isn't installed, this skill detects that and walks the user through setup first.
| name | trace |
| description | Show the provenance trace — every reported number linked to the SQL that produced it, with a confidence badge. Use after an analysis when someone asks "where did that number come from?" |
Renders one self-contained HTML that ties each reported number (a finding) back to the query
that produced it, labeled by confidence: cited (the agent named the query), value-match (a
query's captured result_value equals the number), or inferred (nearest query in time). Unmatched
findings and orphan queries are shown, not hidden — an unverified number is the most important thing to
surface. This is the on-demand artifact for the V2 provenance demo and for any "prove it" moment.
It reads the provenance infra built in Phase 0.8: the query log (hook-stamped with analysis_id +
result_value), the findings manifest, and the reconciler.
Resolve the analysis. Read the current analysis_id and active dataset:
python3 -c "
import sys; sys.path.insert(0, '.')
from helpers.analysis_context import current_analysis_id
from helpers.eval_driver import _active_dataset
print(current_analysis_id(create=False) or '', _active_dataset())
"
If there is no current analysis, there is nothing to trace yet — say so and stop (or, for a past
run, point build_trace at that analysis_id explicitly).
Build + render the trace. Date is today (date '+%Y-%m-%d'):
python3 -c "
import sys; sys.path.insert(0, '.')
from helpers.trace_viewer import build_trace
print(build_trace('<analysis_id>', '<dataset>', '<YYYY-MM-DD>'))
"
This reconciles (writes working/provenance_<analysis_id>.json) and renders
working/trace_<analysis_id>.html. Both are gitignored working files.
Open it. open working/trace_<analysis_id>.html (macOS). It's self-contained and
projection-friendly — large type, collapsible SQL, colored confidence badges.
Read it out. Walk the findings top to bottom: the number, its badge, the SQL. Call out anything unmatched (a number with no query behind it) — that's the honesty check, and the thing to fix.
value-match is strong (the SQL actually returned that
number); inferred is a hint, not proof — say so when reading it out.