pyvene-interventions

name	pyvene-interventions
description	Provides guidance for performing causal interventions on PyTorch models using pyvene's declarative intervention framework. Use when conducting causal tracing, activation patching, interchange intervention training, or testing causal hypotheses about model behavior.
category	ml-training
version	1.0.0
author	Synthetic Sciences
license	MIT
tags	["Causal Intervention","pyvene","Activation Patching","Causal Tracing","Interpretability"]
dependencies	["pyvene>=0.1.8","torch>=2.0.0","transformers>=4.30.0"]

pyvene: Causal Interventions for Neural Networks

pyvene is Stanford NLP's library for performing causal interventions on PyTorch models. It provides a declarative, dict-based framework for activation patching, causal tracing, and interchange intervention training - making intervention experiments reproducible and shareable.

GitHub: stanfordnlp/pyvene (840+ stars) Paper: pyvene: A Library for Understanding and Improving PyTorch Models via Interventions (NAACL 2024)

When to Use pyvene

Use pyvene when you need to:

• Perform causal tracing (ROME-style localization)
• Run activation patching experiments
• Conduct interchange intervention training (IIT)
• Test causal hypotheses about model components
• Share/reproduce intervention experiments via HuggingFace
• Work with any PyTorch architecture (not just transformers)

Consider alternatives when:

• You need exploratory activation analysis → Use TransformerLens
• You want to train/analyze SAEs → Use SAELens
• You need remote execution on massive models → Use nnsight
• You want lower-level control → Use nnsight

Installation

pip install pyvene

Standard import:

import pyvene as pv

Core Concepts

IntervenableModel

The main class that wraps any PyTorch model with intervention capabilities:

import pyvene as pv
from transformers import AutoModelForCausalLM, AutoTokenizer

# Load base model
model = AutoModelForCausalLM.from_pretrained("gpt2")
tokenizer = AutoTokenizer.from_pretrained("gpt2")

# Define intervention configuration
config = pv.IntervenableConfig(
    representations=[
        pv.RepresentationConfig(
            layer=8,
            component="block_output",
            intervention_type=pv.VanillaIntervention,
        )
    ]
)


intervenable = pv.IntervenableModel(config, model)