saelens

name	saelens
description	Use this skill when working with Sparse Autoencoders (SAEs) for mechanistic interpretability of language models, including training SAEs, loading pre-trained SAEs, analyzing neural network features, or integrating SAEs with TransformerLens, HuggingFace Transformers, or other PyTorch-based models.

SAELens — Sparse Autoencoders for Language Models

When to Use

Activate this skill when:

• Training sparse autoencoders (SAEs) on language model activations
• Loading and analyzing pre-trained SAEs from Neuronpedia or HuggingFace
• Performing mechanistic interpretability research on transformer models
• Investigating neural network features using SAE decompositions
• Generating SAE feature dashboards with SAE-Vis
• Hooking SAEs into transformer forward passes (HookedSAETransformer)
• Evaluating SAE quality (variance explained, L0 sparsity, reconstruction loss)
• Running cache activation pipelines for large-scale SAE training
• Analyzing logit lens features through SAE decomposition
• Working with TopK, Gated, or Standard SAE architectures

Keywords: sparse autoencoder, SAE, mechanistic interpretability, TransformerLens, feature analysis, neural network features, activation patching, SAE training, HookedSAETransformer, Neuronpedia, dictionary learning, GPT-2, language model interpretability

---

Installation / Setup

Prerequisites

• Python 3.10+
• PyTorch (CUDA recommended for training)

Install from PyPI (recommended)

pip install sae-lens

Install from source (development)

git clone https://github.com/decoderesearch/SAELens.git
cd SAELens
pip install -e ".[dev]"

Quick Reference

• Documentation: https://decoderesearch.github.io/SAELens/
• PyPI Package: https://pypi.org/project/sae-lens/
• GitHub: https://github.com/decoderesearch/SAELens
• Pre-trained SAEs list: https://decoderesearch.github.io/SAELens/latest/pretrained_saes/
• Migration guide (v6): https://decoderesearch.github.io/SAELens/latest/migrating/
• Neuronpedia: https://www.neuronpedia.org/
• SAE-Vis Library: https://github.com/callummcdougall/sae_vis
• SAEBench: https://github.com/adamkarvonen/SAEBench
• Colab Tutorials:
  • SAE Lens + Neuronpedia
  • Loading and Analysing Pre-Trained SAEs
  • Logit Lens with Features
  • Training a Sparse Autoencoder
  • Training SAEs on Synthetic Data
  • SynthSAEBench

---

Optional dependencies