메뉴
Use when moving from benchmark-driven analysis into causal or mechanism-oriented interventions. Covers activation patching, interchange, control design, read-vs-write distinctions, and intervention-specific success criteria, while pointing to future attention and routing follow-up work.
Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.
SOC 직업 분류 기준
Use when planning, running, or reviewing activation patching or interchange experiments for causal claims in mechanistic interpretability. Especially useful for choosing patch sites, designing paired examples, adding same-label controls, distinguishing read layers from write layers, and avoiding overclaiming from weak or lossy interventions.
Use when a benchmark already has a validated latent label spec and the next job is to plan or review the actual mechanistic analysis program. Covers feature hypotheses, readout and localization strategy, probe choice, control design, evidence ladder, and first experiments.
Use when converting a benchmark's native labels into a benchmark-specific latent label spec for mechanistic interpretability. Covers prompt-side vs response-side separation, label-type classification, direct vs derived targets, derivability checks, confounds, ontology freeze, and gap-list handoff.
Use when deciding whether a benchmark is worth deeper benchmark-first mechanistic interpretability work. Covers public availability, runnable access, label richness, product relevance, likely mechanistic question richness, scale, and obvious confounds before investing in latent-label work.
Constructs linear and nonlinear probes for LLM interpretability in Python. Covers extracting hidden states from transformer models, training probing classifiers/regressors on activations, PCA visualization of representations, SAE feature analysis, logit lens, and causal interventions. Use when the user wants to probe, analyze, or interpret LLM internal representations, build probing classifiers, extract hidden states, or study what information is encoded in model activations.
Use when designing or editing `pipelines_v2` workflow files in this repo. Covers workflow file structure, runner specs, dataset patterns, row alignment, section metadata, report inputs, and the decision boundary between first-class specs and workspace-local transforms.
| name | mechanistic-interventions |
| description | Use when moving from benchmark-driven analysis into causal or mechanism-oriented interventions. Covers activation patching, interchange, control design, read-vs-write distinctions, and intervention-specific success criteria, while pointing to future attention and routing follow-up work. |
Use this skill when the user wants to:
This skill is the interventions phase of the benchmark-flow pipeline.
It begins after:
This skill owns:
It does not own:
Those belong to:
A readable signal is not automatically a writable signal.
Do not treat a high-AUROC probe or clean direction as sufficient evidence for causal importance.
Define:
Choose intervention sites because:
Not because:
Prefer:
Track:
Separate:
Planning-time heuristics to keep in mind:
This skill should also point to likely next steps:
Those may later justify dedicated skills, but this phase should at least surface them.
Primary reference:
Related references:
Use activation-patching-causal-evals as the main methodology library here.
Use the shared principles doc for:
At minimum, leave behind:
workspaces/<workspace>/[<thread>/]phase_04/docs/04-intervention-plan.md
the main human-readable intervention plan, including:
workspaces/<workspace>/[<thread>/]phase_04/docs/04-site-hypothesis.md
why these intervention sites are plausibleworkspaces/<workspace>/[<thread>/]phase_04/docs/04-pairing-and-controls.md
donor-target design, same-label controls, and failure criteriaworkspaces/<workspace>/[<thread>/]phase_04/docs/04-intervention-summary.json
compact structured summary of intervention target, sites, controls, and next actionsUse simple frontmatter on markdown artifacts:
benchmarkphaseversionfrozen_dateinput_artifactsSomeone inspecting this phase should be able to answer:
If the intervention design is weak because the benchmark lacks proper contrasts:
If the intervention plan is sound:
Update the benchmark sidecar with: