xenon

Use when planning, running, or reviewing activation patching or interchange experiments for causal claims in mechanistic interpretability. Especially useful for choosing patch sites, designing paired examples, adding same-label controls, distinguishing read layers from write layers, and avoiding overclaiming from weak or lossy interventions.

Use when a benchmark already has a validated latent label spec and the next job is to plan or review the actual mechanistic analysis program. Covers feature hypotheses, readout and localization strategy, probe choice, control design, evidence ladder, and first experiments.

Use when converting a benchmark's native labels into a benchmark-specific latent label spec for mechanistic interpretability. Covers prompt-side vs response-side separation, label-type classification, direct vs derived targets, derivability checks, confounds, ontology freeze, and gap-list handoff.

Use when deciding whether a benchmark is worth deeper benchmark-first mechanistic interpretability work. Covers public availability, runnable access, label richness, product relevance, likely mechanistic question richness, scale, and obvious confounds before investing in latent-label work.

Constructs linear and nonlinear probes for LLM interpretability in Python. Covers extracting hidden states from transformer models, training probing classifiers/regressors on activations, PCA visualization of representations, SAE feature analysis, logit lens, and causal interventions. Use when the user wants to probe, analyze, or interpret LLM internal representations, build probing classifiers, extract hidden states, or study what information is encoded in model activations.

Use when designing or editing `pipelines_v2` workflow files in this repo. Covers workflow file structure, runner specs, dataset patterns, row alignment, section metadata, report inputs, and the decision boundary between first-class specs and workspace-local transforms.

Walk the documentation tree, cross-check claims against current code, and produce a pruning punch list. Use when docs feel stale, after a methodology shift, or on a regular cadence to fight drift.

Use when a benchmark cannot support the desired latent labels cleanly and needs rewrites, matched pairs, counterbalancing, response generations, or synthetic augmentation. Covers benchmark repair for confounds, split construction, framing variants, and contrast-set design for mechanistic interpretability.

Use when moving from benchmark-driven analysis into causal or mechanism-oriented interventions. Covers activation patching, interchange, control design, read-vs-write distinctions, and intervention-specific success criteria, while pointing to future attention and routing follow-up work.

Use when operating existing `pipelines_v2` workflows in this repo. Covers planning, running, local run tracking, inspecting runs, resuming failed runs, rerunning a step, rerunning from a step, and regenerating local reports from existing artifacts.

Use when designing or repairing benchmark and prompt data so later experiments are behaviorally sane and experimentally valid. Especially useful for lexical controls, split schemes, matched pairs, prompt role placement, relational task design, and preserving the real decision bottleneck.