sparse-autoencoder-training

name	sparse-autoencoder-training
description	Provides guidance for training and analyzing Sparse Autoencoders (SAEs) using SAELens to decompose neural network activations into interpretable features. Use when discovering interpretable features, analyzing superposition, or studying monosemantic representations in language models.
version	1.0.0
author	Orchestra Research
license	MIT
dependencies	["sae-lens>=6.0.0","transformer-lens>=2.0.0","torch>=2.0.0"]
platforms	["linux","macos","windows"]
metadata	{"hermes":{"tags":["Sparse Autoencoders","SAE","Mechanistic Interpretability","Feature Discovery","Superposition"]}}

SAELens: Sparse Autoencoders for Mechanistic Interpretability

SAELens is the primary library for training and analyzing Sparse Autoencoders (SAEs) - a technique for decomposing polysemantic neural network activations into sparse, interpretable features. Based on Anthropic's groundbreaking research on monosemanticity.

GitHub: jbloomAus/SAELens (1,100+ stars)

The Problem: Polysemanticity & Superposition

Individual neurons in neural networks are polysemantic - they activate in multiple, semantically distinct contexts. This happens because models use superposition to represent more features than they have neurons, making interpretability difficult.

SAEs solve this by decomposing dense activations into sparse, monosemantic features - typically only a small number of features activate for any given input, and each feature corresponds to an interpretable concept.

When to Use SAELens

Use SAELens when you need to:

• Discover interpretable features in model activations
• Understand what concepts a model has learned
• Study superposition and feature geometry
• Perform feature-based steering or ablation
• Analyze safety-relevant features (deception, bias, harmful content)

Consider alternatives when:

• You need basic activation analysis → Use TransformerLens directly
• You want causal intervention experiments → Use pyvene or TransformerLens
• You need production steering → Consider direct activation engineering

Installation

pip install sae-lens

Requirements: Python 3.10+, transformer-lens>=2.0.0

Core Concepts

What SAEs Learn

SAEs are trained to reconstruct model activations through a sparse bottleneck:

Input Activation → Encoder → Sparse Features → Decoder → Reconstructed Activation
    (d_model)       ↓        (d_sae >> d_model)    ↓         (d_model)
                 sparsity                      reconstruction
                 penalty                          loss

Loss Function: MSE(original, reconstructed) + L1_coefficient × L1(features)

Key Validation (Anthropic Research)

In "Towards Monosemanticity", human evaluators found 70% of SAE features genuinely interpretable. Features discovered include: