transcoder-circuits

name	transcoder-circuits
description	Use this skill when working with transcoder-based circuit analysis of large language models, including training transcoders, analyzing MLP sublayers, reverse-engineering LLM circuits, and creating feature dashboards for interpretability research.

Transcoder Circuits: Reverse-Engineering LLM Circuits with Transcoders

When to Use

Activate this skill when:

• Reverse-engineering circuits inside transformer language models (GPT-2, Pythia, etc.)
• Training transcoders to decompose MLP sublayers into sparse linear combinations of features
• Analyzing interpretable features in LLMs using sparse autoencoders or transcoders
• Building feature dashboards or activation visualizations
• Performing mechanistic interpretability research on neural networks
• Comparing SAE vs. transcoder feature interpretability
• Running circuit analysis, replacement contexts, or activation patching

Keywords: transcoder, SAE, sparse autoencoder, mechanistic interpretability, LLM circuits, MLP features, GPT-2, Pythia, feature dashboard, circuit analysis, activation patching

Quick Reference

• Repository: https://github.com/jacobdunefsky/transcoder_circuits
• Transcoder Weights (HuggingFace): https://huggingface.co/pchlenski/gpt2-transcoders
• SAELens (upstream SAE code): https://github.com/jbloomAus/SAELens
• Walkthrough Notebook: walkthrough.ipynb

Installation / Setup

Prerequisites

• Python 3.8+
• CUDA-capable GPU recommended

Quick Setup (from README)

bash setup.sh

This script installs dependencies and downloads transcoder weights from HuggingFace (pchlenski/gpt2-transcoders).

Manual Installation

pip install -r requirements.txt

Requirements (from requirements.txt)

Key dependencies include:

• transformer_lens — for loading and hooking into transformer models
• torch — PyTorch
• einops
• datasets
• huggingface_hub
• wandb (optional, for training logging)

Core Features

• Transcoder Training: Train transcoders on LLM MLP sublayers to decompose activations into sparse interpretable features (sae_training/)
• Circuit Analysis: Reverse-engineer fine-grained feature circuits within a model (transcoder_circuits/circuit_analysis.py)
• Feature Dashboards: Generate dashboards for exploring transcoder and SAE features (transcoder_circuits/feature_dashboards.py)
• Replacement Context: Swap MLP sublayers with transcoder reconstructions during inference (transcoder_circuits/replacement_ctx.py)
• Activations Store: Stream tokens and generate/store activations during training (sae_training/activations_store.py)
• Geometric Median: Utility for computing geometric median for initialization (sae_training/geom_median/)
• SAE/Transcoder Comparison: Evaluation notebooks comparing SAEs and transcoders on Pythia-410M

Usage Examples

Loading a Transcoder and Running Replacement Context

import torch
from transformer_lens import HookedTransformer
from transcoder_circuits.replacement_ctx import TranscoderReplacementContext

# Load GPT-2 small via TransformerLens
model = HookedTransformer.from_pretrained("gpt2")

# Load transcoder weights (downloaded via setup.sh)
transcoder = torch.load("path/to/transcoder_layer_0.pt")

# Use replacement context to patch MLP with transcoder
with TranscoderReplacementContext(model, {0: transcoder}):
    tokens = model.to_tokens("Hello, world!")
    logits = model(tokens)

Training a Transcoder

python train_transcoder.py

Or programmatically:

from sae_training.config import LanguageModelSAERunnerConfig
from sae_training.train_sae_on_language_model import language_model_sae_runner

cfg = LanguageModelSAERunnerConfig(
    model_name="gpt2",
    hook_point="blocks.0.hook_mlp_out",
    hook_point_layer=0,
    d_in=768,
    expansion_factor=4,
    # ... other config options
)
language_model_sae_runner(cfg)

Circuit Analysis

from transcoder_circuits.circuit_analysis import get_circuit_scores

# Analyze which transcoder features contribute most to a given output
scores = get_circuit_scores(model, transcoders, tokens, metric_fn)

Feature Dashboards

from transcoder_circuits.feature_dashboards import make_feature_dashboard

# Generate a dashboard for a specific transcoder feature
dashboard = make_feature_dashboard(
    transcoder=transcoder,
    model=model,
    feature_idx=42,
    dataset=dataset,
)

Key APIs / Models

Models Supported

• GPT-2 small (primary, weights available at pchlenski/gpt2-transcoders)
• Pythia-410M (used in sweep/comparison experiments)

Core Classes

Class	Module	Description
ActivationsStore	sae_training/activations_store.py	Streams tokens and stores LLM activations for training
RunnerConfig	sae_training/config.py	Base config shared across all training runners
LanguageModelSAERunnerConfig	sae_training/config.py	Config for training transcoders/SAEs on a language model
CacheActivationsRunnerConfig	sae_training/config.py	Config for caching LLM activations to disk
SparseAutoencoder	sae_training/sparse_autoencoder.py	The transcoder/SAE model class

Key Functions

Function	Module	Description
language_model_sae_runner	sae_training/train_sae_on_language_model.py	Main training entry point
get_circuit_scores	transcoder_circuits/circuit_analysis.py	Compute feature circuit attribution scores
make_feature_dashboard	transcoder_circuits/feature_dashboards.py	Generate feature visualization dashboards

Related Tools

Anthropic has open-sourced circuit-tracer, a Python library that builds on the (cross-layer) MLP transcoder formulation to extract feature circuits directly from open-weights models such as Gemma-2-2B and Llama-3.2-1B. Given a prompt, it computes the direct effect of every active transcoder feature, transcoder error node, and input token on every other active feature and output logit, and surfaces the resulting attribution graph through a Neuronpedia-hosted frontend. It pairs naturally with the training and feature-analysis primitives in this skill: train or load transcoders here, then plug them into circuit-tracer's ReplacementModel.from_pretrained to obtain end-to-end circuits.

• Announcement: https://www.anthropic.com/research/open-source-circuit-tracing
• GitHub: https://github.com/safety-research/circuit-tracer

Common Patterns & Best Practices

1. Start with walkthrough.ipynb — the notebook walks through loading transcoders, replacement context, and circuit analysis step by step.
2. Use TranscoderReplacementContext as a context manager to ensure the model is properly restored after patching.
3. Transcoder weights are layer-specific — load each layer's transcoder separately.
4. MLP0 features are particularly important; the "restricted blind case studies" notebook shows how to work without them.
5. Use wandb for tracking training runs when training new transcoders.
6. Circuit analysis is iterative — start by finding the top features, then investigate their upstream inputs recursively.

Demo Scripts

scripts/train_transcoder_example.py

#!/usr/bin/env python3
"""
Training Example: Train a Transcoder on GPT-2

This script demonstrates how to configure and launch a transcoder
training run using the sae_training library.

Prerequisites:
    bash setup.sh
    OR
    pip install -r requirements.txt

Usage:
    python train_transcoder_example.py

Notes:
    - Transcoders are trained on MLP sublayers of transformer models.
    - This is based on the existing train_transcoder.py in the repo root.
    - Uses the LanguageModelSAERunnerConfig and language_model_sae_runner.
"""

import torch


def build_transcoder_config(
    model_name: str = "gpt2",
    layer: int = 0,
    expansion_factor: int = 4,
    total_training_tokens: int = 1_000_000,
    log_to_wandb: bool = False,
):
    """
    Build a LanguageModelSAERunnerConfig for transcoder training.

    Args:
        model_name: HuggingFace model name
        layer: Which transformer layer to train the transcoder for
        expansion_factor: Ratio of transcoder hidden dim to model d_model
        total_training_tokens: Number of tokens to train on
        log_to_wandb: Whether to log metrics to Weights & Biases

    Returns:
        LanguageModelSAERunnerConfig instance
    """
    from sae_training.config import LanguageModelSAERunnerConfig

    # GPT-2 small has d_model=768; adjust for other models
    d_model_map = {
        "gpt2": 768,
        "gpt2-medium": 1024,
        "gpt2-large": 1280,
        "EleutherAI/pythia-410m": 1024,
    }
    d_in = d_model_map.get(model_name, 768)

    cfg = LanguageModelSAERunnerConfig(
        # --- Model & Hook ---
        model_name=model_name,
        hook_point=f"blocks.{layer}.hook_mlp_out",
        hook_point_layer=layer,
        d_in=d_in,

        
        expansion_factor=expansion_factor,
        

        
        lr=,
        adam_beta1=,
        adam_beta2=,
        l1_coefficient=,
        lr_scheduler_name=,
        train_batch_size=,
        context_size=,
        total_training_tokens=total_training_tokens,

        
        dataset_path=,

        
        n_checkpoints=,
        checkpoint_path=,

        
        log_to_wandb=log_to_wandb,
        wandb_project=,
        wandb_entity=,
        wandb_log_frequency=,

        
        device=  torch.cuda.is_available()  ,
        seed=,
        dtype=,
    )

     cfg


 ():
    
     sae_training.train_sae_on_language_model  language_model_sae_runner

    ()
    ()
    ()
    ()
    ()
    ()
    ()
    ()
    ()

    transcoder = language_model_sae_runner(cfg)
     transcoder


 ():
    
     transcoder_circuits.replacement_ctx  TranscoderReplacementContext

    layer = transcoder.cfg.hook_point_layer  (transcoder, )  
    tokens = model.to_tokens(prompt)

    
    hook_point = 
    _, cache = model.run_with_cache(tokens, names_filter=hook_point)
    original_acts = cache[hook_point]

    flat_acts = original_acts.reshape(-, original_acts.shape[-])
     torch.no_grad():
        feature_acts, reconstruction, *_ = transcoder(flat_acts)

    mse = ((flat_acts - reconstruction) ** ).mean().item()
    sparsity = (feature_acts > ).().mean().item()
    l0 = (feature_acts > ).().(dim=-).mean().item()

    metrics = {
        : mse,
        : sparsity,
        : l0,
        : feature_acts.shape[-],
    }

    ()
     k, v  metrics.items():
        ()

     metrics


 ():
    
    ( * )
    ()
    ( * )

    
    ()
    cfg = build_transcoder_config(
        model_name=,
        layer=,
        expansion_factor=,
        total_training_tokens=,  
        log_to_wandb=,
    )

    ()
    ()
    ()
    ()

    
    ()
    :
        transcoder = run_training(cfg)
        ()

        
        ()
         transformer_lens  HookedTransformer
        model = HookedTransformer.from_pretrained()
        model.()
        evaluate_transcoder(transcoder, model)

     Exception  e:
        ()
        ()
        


 __name__ == :
    main()

scripts/transcoder_usage_example.py

#!/usr/bin/env python3
"""
Usage Example: Transcoder Circuits - Loading and Analyzing LLM Circuits

This script demonstrates how to use the transcoder_circuits library to:
1. Load a pretrained GPT-2 model via TransformerLens
2. Load a pretrained transcoder
3. Run inference with transcoder replacement context
4. Perform basic circuit analysis

Prerequisites:
    bash setup.sh   # installs deps and downloads weights
    OR
    pip install -r requirements.txt

Transcoder weights: https://huggingface.co/pchlenski/gpt2-transcoders
"""

import torch
import os
from pathlib import Path


def load_model(model_name: str = "gpt2"):
    """
    Load a HookedTransformer model using TransformerLens.

    Args:
        model_name: HuggingFace model name (e.g., "gpt2", "pythia-410m")

    Returns:
        HookedTransformer model instance
    """
    try:
        from transformer_lens import HookedTransformer
        model = HookedTransformer.from_pretrained(model_name)
        model.eval()
        print(f"Loaded model: {model_name}")
        return model
    except ImportError:
        raise ImportError("transformer_lens not installed. Run: pip install transformer_lens")


def load_transcoder(weights_path: str, device: str = "cpu"):
    """
    Load a pretrained transcoder from a .pt weights file.

    Args:
        weights_path: Path to the transcoder .pt file
                      (downloaded via setup.sh to ./transcoder_weights/)
        device: torch device string ("cpu" or "cuda")

    Returns:
        Loaded transcoder object
    """
    if not os.path.exists(weights_path):
        raise FileNotFoundError(
            
            
        )
    transcoder = torch.load(weights_path, map_location=device)
    ()
     transcoder


 ():
    
    :
         transcoder_circuits.replacement_ctx  TranscoderReplacementContext
     ImportError:
         ImportError()

    tokens = model.to_tokens(prompt)
    ()

     TranscoderReplacementContext(model, transcoders_by_layer):
        logits = model(tokens)

    ()
     logits


 ():
    
    last_token_logits = logits[, -, :]  
    probs = torch.softmax(last_token_logits, dim=-)
    top_probs, top_indices = torch.topk(probs, k)

    results = []
     idx, prob  (top_indices.tolist(), top_probs.tolist()):
        token_str = model.tokenizer.decode([idx])
        results.append((token_str, prob))
        ()

     results


 ():
    
    tokens = model.to_tokens(prompt)
    hook_point = 

    _, cache = model.run_with_cache(tokens, names_filter=hook_point)
    activations = cache[hook_point]  
    ()
     activations


 ():
    
    
    flat_acts = activations.reshape(-, activations.shape[-])

     torch.no_grad():
        feature_acts, reconstruction, *_ = transcoder(flat_acts)

    ()
    ()
    ()

    
    mean_acts = feature_acts.mean(dim=)
    top_vals, top_idxs = torch.topk(mean_acts, )
    ()
     idx, val  (top_idxs.tolist(), top_vals.tolist()):
        ()

     feature_acts, reconstruction


 ():
    
    :
         sae_training.config  LanguageModelSAERunnerConfig
     ImportError:
         ImportError()

    cfg = LanguageModelSAERunnerConfig(
        
        model_name=,
        hook_point=,
        hook_point_layer=,
        d_in=,

        
        expansion_factor=,  

        
        lr=,
        l1_coefficient=,
        train_batch_size=,
        total_training_tokens=,

        
        dataset_path=,

        
        log_to_wandb=,
        wandb_project=,
    )

    ()
    ()
    ()
    ()
    ()

     cfg


 ():
    
    ( * )
    ()
    ( * )

    
    ()
    :
        cfg = training_config_example()
     Exception  e:
        ()

    
    ()
    :
        model = load_model()
     Exception  e:
        ()
        ()
        

    
    ()
    prompt = 
    :
        activations = cache_activations_example(model, prompt, layer=)
     Exception  e:
        ()
        

    
    ()
    
    weights_path = 
    :
        transcoder = load_transcoder(weights_path)

        
        ()
        feature_acts, reconstruction = demonstrate_feature_activation(transcoder, activations)

        
        ()
        logits = run_with_replacement_context(model, {: transcoder}, prompt)
        ()
        get_top_predicted_tokens(model, logits)

     FileNotFoundError  e:
        ()
        ()
        ()
        tokens = model.to_tokens(prompt)
         torch.no_grad():
            logits = model(tokens)
        ()
        get_top_predicted_tokens(model, logits)

    ( +  * )
    ()
    ( * )


 __name__ == :
    main()