steer-features

name

steer-features

description

Use this skill for feature-level steering of models — locating the internal feature that drives a target behavior, scoring and selecting it by its effect on the model's output, and directly amplifying or shrinking that feature's activation during generation to control behavior. Applies to features read from the model's own activations or from a Sparse Autoencoder (SAE); the bundled demo scripts happen to use an SAE, but the method does not require one.

SAEs Are Good for Steering - If You Select the Right Features

Note — an SAE is optional. The method itself only needs to locate the feature that drives the target behavior and scale its activation in place during generation. When the features live in a Sparse Autoencoder (SAE), the same intervention takes the encode → amplify/diminish target feature → decode form (Feature Clamping): the SAE encodes the residual stream into feature space, the target feature's activation is scaled, and the result is SAE decoded back. The demo scripts here use an SAE, but that is a property of this demo, not a requirement of the method.

When to Use

This skill should be activated when:

• Working with Sparse Autoencoders (SAEs) for model interpretability
• Steering language model outputs toward specific concepts or behaviors
• Analyzing and scoring SAE features (input vs output features)
• Implementing unsupervised decomposition of model latent spaces
• Comparing SAE steering with supervised steering methods
• Working with models like Gemma, LLaMA, or Pythia for feature-based steering
• Evaluating feature effectiveness for model control

Keywords: SAE, sparse autoencoder, model steering, feature selection, interpretability, latent space decomposition, output scoring, input scoring, neural network control

Quick Reference

• Paper: SAEs Are Good for Steering -- If You Select the Right Features
• Website: Project Page
• Repository: GitHub
• Related Tool: Neuronpedia API for feature data

Installation/Setup

# Install required packages
pip install -r requirements
pip install accelerate
pip install sae-lens

# For development/testing
git clone https://github.com/technion-cs-nlp/saes-are-good-for-steering
cd saes-are-good-for-steering

Prerequisites

• Python 3.8+
• CUDA-capable GPU recommended for model inference
• Sufficient RAM for loading language models

Core Features

• Output Score Calculation: Characterize features based on their effect on model output
• Input Score Calculation: Analyze features based on input token activation patterns
• Feature Filtering: Remove low-scoring features to improve steering effectiveness
• Multi-Model Support: Works with Gemma (2B/9B), LLaMA 3.1, and Pythia models
• SAE Hook Integration: Amplify specific SAE features during model inference
• Generation Analysis: Evaluate steering success with various metrics
• Data Processing: Handle feature data from Neuronpedia and custom sources

Default Intervention Mode (IMPORTANT)

Unless the user explicitly asks for additive steering-vector injection, all SAE experiments MUST use the encode → amplify target feature → decode flow (i.e. Feature Clamping / Encoder-Clamp-Decoder):

1. Encode the residual-stream activations into SAE feature space.
2. Amplify (clamp) the target feature to the desired value.
3. Decode/reconstruct the activations and write them back.

Additive injection (hidden_states += steering_factor * vector) is only allowed when the user specifically requests it. Note: the placeholder body in scripts/steer_model.py currently performs additive injection for demonstration only — it is NOT the default and should not be treated as the reference behavior.

Usage Examples

Calculate Output Scores

python ./src/output_score.py --model_type=<model_type> --features_file=<features_json> --cache_path=<filename_to_load_and_save>

Calculate Input Scores

First, download feature data from Neuronpedia, then run:

python ./src/input_score.py --model_type=<model_type> --features_file=<features_json> --cache_path=<filename_to_load_and_save> --feature_data_path=<path>

Example Model Types

• gemma_2b
• gemma_9b
• gemma_9b_it (instruction-tuned)
• llama31
• pythia70

Key APIs/Models

Supported Models

• Gemma 2B: Base language model for lightweight steering
• Gemma 9B: Larger capacity model for complex steering tasks
• Gemma 9B-IT: Instruction-tuned variant with enhanced steering capabilities
• LLaMA 3.1: Meta's language model for comparative analysis
• Pythia 70M: Small-scale model for rapid experimentation

Core Classes/Functions

• AmlifySAEHook: Hook class for amplifying SAE features during inference
• get_output_score(): Calculate output scores for features
• get_generation_success(): Evaluate steering effectiveness
• init_hook(): Initialize SAE hooks for model steering

Configuration Files

• Feature definitions: JSON files specifying layer, feature indices, and metadata
• Cache files: Stored scores and generation results for efficiency
• Instruction sets: Curated prompts for Concept500 evaluation

Common Patterns & Best Practices

Feature Selection Strategy

1. Calculate both input and output scores for all features
2. Filter out features with low output scores (< threshold)
3. Select features with high output scores but moderate input scores
4. This approach yields 2-3x improvement in steering effectiveness

Efficient Processing

• Use caching to avoid recomputing scores
• Process features in batches when possible
• Store intermediate results for iterative analysis

Steering Factor Optimization

• Test multiple steering factors (0.2 to 20.0)
• Start with lower factors for subtle steering
• Increase gradually to find optimal balance between effect and fluency

Data Organization

Available Datasets

• Feature Files: Pre-selected features for each model variant
• Generated Texts: Outputs at various steering factors
• LLM Scores: External evaluation of concept adherence, instruction following, and fluency
• Axbench Instructions: 131 instruction prompts for comprehensive evaluation

File Structure

Demo Scripts

scripts/compute_scores.py

#!/usr/bin/env python3
"""
Compute Input and Output Scores for SAE Features

This script demonstrates how to calculate input and output scores for SAE features
to identify which features are most effective for model steering.

Requires: pip install sae-lens accelerate transformers torch
"""

import json
import argparse
from pathlib import Path
from typing import Dict, List, Tuple, Optional
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from sae_lens import SAE
import numpy as np


def load_features(features_file: str) -> Dict:
    """
    Load feature definitions from a JSON file.
    
    Args:
        features_file: Path to JSON file containing feature specifications
        
    Returns:
        Dictionary containing feature definitions
    """
    with open(features_file, 'r') as f:
        features = json.load(f)
    print(f"Loaded {len(features)} features from {features_file}")
    return features


def compute_output_score(
    model: AutoModelForCausalLM,
    tokenizer: AutoTokenizer,
    layer: int,
    feature_idx: int,
    test_prompts: Optional[List[str]] = None
) -> :
    
     test_prompts  :
        test_prompts = [
            ,
            ,
            ,
            ,
            
        ]
    
    scores = []
    
     prompt  test_prompts:
        
        inputs = tokenizer(prompt, return_tensors=)
        
        
         torch.no_grad():
            baseline_outputs = model(**inputs)
            baseline_logits = baseline_outputs.logits[, -, :]
        
        
        
        
        
        
        
        score = torch.rand().item()  
        scores.append(score)
    
     np.mean(scores)


 () -> :
    
    
     (feature_idx)   feature_data:
         
    
    activation_data = feature_data[(feature_idx)]
    
    
    
    
    
    
    
    
    
    
    input_score = np.random.rand()  
     input_score


 () -> []:
    
    selected_features = []
    
     feature_idx  features.keys():
        idx = (feature_idx)
        
        
         output_scores.get(idx, ) < output_threshold:
            
            
        
         input_scores.get(idx, ) > input_threshold:
            
            
        selected_features.append(idx)
    
    ()
    ()
    
     selected_features


 ():
    
    scores_data = {
        : output_scores,
        : input_scores,
        : {
            : (output_scores),
            : np.mean((output_scores.values())),
            : np.mean((input_scores.values()))
        }
    }
    
     (output_file, )  f:
        json.dump(scores_data, f, indent=)
    
    ()


 ():
    parser = argparse.ArgumentParser(description=)
    parser.add_argument(, =, default=,
                        =)
    parser.add_argument(, =, required=,
                        =)
    parser.add_argument(, =, default=,
                        =)
    parser.add_argument(, action=,
                        =)
    parser.add_argument(, action=,
                        =)
    
    args = parser.parse_args()
    
    
    features = load_features(args.features_file)
    
    
    output_scores = {}
    input_scores = {}
    
     args.compute_output:
        ()
        
        tokenizer = AutoTokenizer.from_pretrained(args.model_name)
        model = AutoModelForCausalLM.from_pretrained(
            args.model_name,
            torch_dtype=torch.float16,
            device_map=
        )
        
        
         feature_idx  features.keys():
            idx = (feature_idx)
            layer = features[feature_idx].get(, )
            score = compute_output_score(model, tokenizer, layer, idx)
            output_scores[idx] = score
            
             idx %  == :
                ()
    
     args.compute_input:
        ()
        
        tokenizer = AutoTokenizer.from_pretrained(args.model_name)
        
        
        feature_data = {}  
        
        
         feature_idx  features.keys():
            idx = (feature_idx)
            score = compute_input_score(tokenizer, feature_data, idx)
            input_scores[idx] = score
    
    
     output_scores  input_scores:
        selected = filter_features_by_scores(features, output_scores, input_scores)
        ()
    
    
     output_scores  input_scores:
        save_scores(output_scores, input_scores, args.output_file)


 __name__ == :
    main()

scripts/steer_model.py

#!/usr/bin/env python3
"""
Steer Language Model Output using SAE Features

This script demonstrates how to use selected SAE features to steer model outputs
toward desired concepts or behaviors.

Requires: pip install sae-lens transformers torch accelerate
"""

import json
import argparse
from typing import List, Dict, Optional, Tuple
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from dataclasses import dataclass
import numpy as np


@dataclass
class SteeringConfig:
    """Configuration for steering experiments."""
    model_name: str
    feature_idx: int
    layer: int
    steering_factor: float
    max_length: int = 100
    temperature: float = 0.7
    top_p: float = 0.9


class SAESteeringHook:
    """
    Hook for amplifying specific SAE features during model inference.
    
    This class demonstrates the core steering mechanism where specific
    features are amplified to influence model behavior.
    """
    
    def __init__(self, layer: int, feature_idx: int, steering_factor: float):
        """
        Initialize the steering hook.
        
        Args:
            layer: Model layer to hook into
            feature_idx: Index of the SAE feature to amplify
            steering_factor: Amplification factor (typically 0.2 to 20.0)
        """
        .layer = layer
        .feature_idx = feature_idx
        .steering_factor = steering_factor
        .hook_handle = 
        
     ():
        
        
        
        
        
        
        
         (output, ):
            hidden_states = output[]
        :
            hidden_states = output
            
        
        
        batch_size, seq_len, hidden_dim = hidden_states.shape
        
        
        steering_vector = torch.randn(, , hidden_dim, device=hidden_states.device)
        steering_vector = steering_vector * .steering_factor
        
        
        hidden_states[:, -, :] = hidden_states[:, -, :] + steering_vector.squeeze()
        
         (output, ):
             (hidden_states,) + output[:]
         hidden_states
    
     ():
        
        
         (model, ):  
            layers = model.model.layers
         (model, ):  
            layers = model.transformer.h
        :
            layers = model.layers
            
        target_layer = layers[.layer]
        .hook_handle = target_layer.register_forward_hook(.steering_hook)
        
     ():
        
         .hook_handle:
            .hook_handle.remove()
            .hook_handle = 


 () -> [[, ]]:
    
     (scores_file, )  f:
        scores_data = json.load(f)
    
    output_scores = scores_data.get(, {})
    input_scores = scores_data.get(, {})
    
    
    combined_scores = {}
     feature_idx, out_score  output_scores.items():
        in_score = input_scores.get(feature_idx, )
        
        combined = out_score * ( - (in_score - ))
        combined_scores[(feature_idx)] = combined
    
    
    sorted_features = (combined_scores.items(), key= x: x[], reverse=)
     sorted_features[:top_k]


 () -> :
    
    
    hook = SAESteeringHook(
        layer=config.layer,
        feature_idx=config.feature_idx,
        steering_factor=config.steering_factor
    )
    hook.register(model)
    
    :
        
        inputs = tokenizer(prompt, return_tensors=)
        inputs = {k: v.to(model.device)  k, v  inputs.items()}
        
        
         torch.no_grad():
            outputs = model.generate(
                **inputs,
                max_length=config.max_length,
                temperature=config.temperature,
                top_p=config.top_p,
                do_sample=,
                pad_token_id=tokenizer.eos_token_id
            )
        
        
        generated_text = tokenizer.decode(outputs[], skip_special_tokens=)
         generated_text
        
    :
        
        hook.remove()


 () -> [, ]:
    
    results = {}
    
     factor  factors:
        config = SteeringConfig(
            model_name=model.config.name_or_path,
            feature_idx=feature_idx,
            layer=layer,
            steering_factor=factor
        )
        
        output = generate_with_steering(model, tokenizer, prompt, config)
        results[factor] = output
        
        ()
        (output)
    
     results


 ():
    parser = argparse.ArgumentParser(description=)
    parser.add_argument(, =, default=,
                        =)
    parser.add_argument(, =, default=,
                        =)
    parser.add_argument(, =, =)
    parser.add_argument(, =, default=, =)
    parser.add_argument(, =, default=,
                        =)
    parser.add_argument(, =, =)
    parser.add_argument(, action=,
                        =)
    
    args = parser.parse_args()
    
    
    ()
    tokenizer = AutoTokenizer.from_pretrained(args.model_name)
    model = AutoModelForCausalLM.from_pretrained(
        args.model_name,
        torch_dtype=torch.float16,
        device_map=
    )
    
    
     args.scores_file:
        best_features = load_best_features(args.scores_file, top_k=)
        ()
        
          args.feature_idx:
            
            args.feature_idx = best_features[][]
            ()
    
     args.compare_factors:
        
        results = compare_steering_factors(
            model, tokenizer, args.prompt,
            args.feature_idx, args.layer
        )
        
        
         (, )  f:
            json.dump(results, f, indent=)
        ()
    :
        
        config = SteeringConfig(
            model_name=args.model_name,
            feature_idx=args.feature_idx,
            layer=args.layer,
            steering_factor=args.steering_factor
        )
        
        output = generate_with_steering(model, tokenizer, args.prompt, config)
        
        ()
        (output)


 __name__ == :
    main()